Agents Need Plumbing

Remy and the Agent Stack Beneath It

Google I/O has historically been a developer tools conference. This year it was an agent infrastructure announcement disguised as a product launch. Remy, Google's new agent framework, sits at the center of the Android 17 integration story. It handles multi-step task execution across apps, maintains conversational context between sessions, and delegates sub-tasks to specialized agent endpoints. The model underneath is Gemini. But the engineering surface that matters is everything around Gemini: the orchestration layer, the memory store, the tool registry, the permission model.

The New Stack reported that enterprise architects are already rethinking their AI stacks in response to Remy's architecture. The reasoning is straightforward. Google did not release a better model. Google released a better agent runtime. The runtime handles the hard parts that model intelligence alone cannot solve: persistent memory across sessions, tool invocation with error recovery, state checkpointing that survives process restarts, and multi-agent coordination protocols.

This distinction matters because most enterprise agent projects fail at the infrastructure layer, not the intelligence layer. The model can reason about what to do. The model can generate the right API call. But the system around the model cannot remember what happened yesterday, cannot recover from a failed tool invocation gracefully, and cannot hand off state to a different agent when the task crosses domain boundaries. These are plumbing problems. Pipes, valves, and pressure gauges. Not cognitive architecture.

The Signal for Enterprise Teams

Google committed an entire I/O keynote to agent infrastructure. That is a resource allocation signal worth reading carefully. It means Google's internal teams concluded that model capability is sufficient for most agent use cases today. The gap is the system around the model. Enterprise teams building their own agent deployments should take the same inventory. If your agent demo works in a notebook but fails in production, the problem is almost certainly in the plumbing: state management, error handling, memory persistence, tool authentication, or routing logic. Not model quality.

Tack and the Agent Storage Layer

Inference Room shipped Tack this week and committed to releasing at least one agent infrastructure product per month. Tack is specifically an agent-native storage and memory layer. Not a vector database. Not a key-value store. A purpose-built persistence system designed for the access patterns that agents produce: high-frequency reads of recent context, sparse retrieval of historical interactions, structured state snapshots, and multi-agent shared memory.

The fact that a company can build a business around agent memory tells you where the pain is. Current approaches cobble together vector databases, Redis caches, and prompt stuffing to give agents something resembling memory. It works for demos. It breaks under load, over time, and across sessions. Agents that handle real workflows need to remember what they did last Tuesday, what failed, what the user corrected, and what constraints apply going forward. That is a storage problem with specific durability, consistency, and retrieval requirements.

Inference Room's monthly release cadence is also a signal. There are enough unsolved infrastructure problems in the agent stack that a company can ship a new product every 30 days and still find greenfield territory. Memory. Authentication delegation. Billing attribution for multi-agent workflows. Audit logging that traces decisions across agent handoffs. Each of these is a distinct infrastructure product that does not exist in mature form today.

Cold Starts and the Inference Layer

The infrastructure gap extends below the memory layer into the inference engine itself. Modal published techniques this week for cutting inference cold starts by 40x using a combination of linear programming, FUSE filesystems, checkpoint/restore, and CUDA-level checkpointing. A 40x reduction in cold start latency transforms what agents can do. An agent that takes 12 seconds to wake up is useless for interactive workflows. An agent that takes 300 milliseconds to wake up can handle real-time decision support.

This matters for enterprise deployments because most organizations will not run agents on dedicated, always-warm GPU instances. The cost is prohibitive for the long tail of agent use cases. Serverless agent inference. Where GPU resources spin up on demand, execute the agent step, and release. Is the economically viable architecture. But serverless only works if cold starts are fast enough to be imperceptible. Modal's 40x improvement moves that from theoretical to practical.

Logistics as the Agent Proving Ground

Logistics companies are deploying agentic AI to transform passive record-keeping systems into active operational decision-makers. This is one of the first industry verticals where agents are making consequential decisions in production. Not generating text. Not summarizing documents. Making routing decisions, adjusting inventory allocations, and renegotiating supplier timelines autonomously.

Logistics is a natural proving ground for agent infrastructure because the domain has clear constraints, measurable outcomes, and high tolerance for incremental automation. A supply chain agent that reroutes a shipment based on port congestion data produces a measurable cost delta. You can compare agent decisions to human decisions on the same inputs and calculate the ROI within a single quarter. This feedback loop is what most enterprise agent deployments lack.

The infrastructure demands of logistics agents are also instructive. These agents need real-time data feeds from IoT sensors, port systems, weather APIs, and carrier networks. They need to maintain state across multi-day workflows. A container shipment takes weeks. The agent managing it needs persistent context that survives far longer than a chat session. They need to coordinate with other agents handling parallel shipments. And they need audit trails that regulators and insurers can inspect. Every one of these requirements maps to an infrastructure component that most agent frameworks do not provide out of the box.

The Dell Deskside Signal

Dell launched deskside agentic AI hardware this week, claiming an 87% cost reduction over two years compared to cloud-hosted agent inference. The 87% figure deserves scrutiny. It likely assumes high utilization rates and specific workload profiles. But the directional signal is clear: hardware vendors see agent infrastructure as a distinct product category, separate from general-purpose AI servers.

A deskside agent appliance places the plumbing physically inside the enterprise perimeter. Memory, state, and inference all run on hardware the organization owns. For industries with strict data residency requirements, this solves the compliance problem and the latency problem simultaneously. AI infrastructure is already straining under sovereignty demands. Dell's product is a response: if you cannot send agent data to the cloud, bring the cloud to the desk.

The Workforce Impact Is Here

These are not theoretical deployments. Standard Chartered announced 7,000 job cuts driven by AI adoption. Microsoft's AI chief projected that accounting, legal, marketing, and project management roles could be automated within 18 months. The character-based AI agents market alone is projected to reach $5.45 billion by 2032, growing from $0.55 billion this year. A 10x growth projection over six years. These numbers reflect organizations that have moved past the proof-of-concept phase and are committing capital to production agent deployments. The plumbing investments are what enable that transition.

The pattern across this week's signals is consistent. Google built a runtime. Inference Room built a memory layer. Modal solved cold starts. Dell built an appliance. Logistics firms built domain-specific agent pipelines. Every one of these efforts addresses the same gap: the systems between the model and the real world.

Most enterprise teams evaluating agents focus on model selection. Which LLM has the best reasoning? Which scores highest on benchmarks? OpenAI's o3 delivers 20% fewer errors than its predecessors. That matters. But a 20% improvement in model accuracy is worthless if the agent loses its memory between sessions, cannot recover from a failed API call, or has no way to hand off a multi-day workflow to a colleague agent.

The infrastructure stack that production agents require looks like this. A persistent memory system with both short-term context and long-term retrieval. A tool registry with authentication, rate limiting, and fallback paths. A state management layer that can checkpoint and resume agent workflows across process boundaries. A routing system that can dispatch tasks to specialized sub-agents. An audit layer that records every decision, every tool invocation, and every state transition. And an inference layer with cold starts fast enough to support on-demand agent activation.

None of these components require frontier model capability. All of them require careful systems engineering. The organizations deploying agents successfully in 2026 are the ones that recognized this distinction early and invested in plumbing before they invested in more powerful models.