AI Agents Hampered by Legacy Infrastructure, Not Models, Experts Say
Leading infrastructure experts from LinkedIn, Walmart, and Zendesk convened at VB Transform 2026 to discuss the primary obstacles in deploying AI agents at scale. Their collective conclusion was that legacy infrastructure, built for human operational speeds, is the main bottleneck, rather than the AI models themselves. The challenge lies in bridging the speed gap between human-centric systems and the millisecond-level demands of AI agents.
At LinkedIn, the issue was Kubernetes' slow container provisioning, which was resolved by shifting to pre-provisioned container pools. They also addressed LLM hallucination in self-orchestrating agents by scripting 80% of workflows and reserving LLMs for reasoning tasks, with evidence logged at each step. Walmart faced a surge in internal agent development by 'citizen developers,' leading to duplication. Their solution involved implementing governance to manage and promote agent efficiency, preventing engineering bottlenecks. Zendesk's challenge stemmed from data volume, with 20 billion customer conversations requiring robust data pipelines and infrastructure rather than simply large context windows for LLMs.
All three companies emphasized the importance of owning core components while leveraging external advancements strategically. LinkedIn developed an AI gateway and a model-independent memory subsystem for flexibility. Walmart created an internal gateway for vendor neutrality across different workflow types. The experts advised investing in evaluation frameworks early, owning agent harnesses from the outset with monitoring capabilities, and building for model and context independence to ensure long-term adaptability.
AI agent deployment is currently constrained by the inertia of existing enterprise IT architectures, which were not designed for the rapid, autonomous operations characteristic of AI. This highlights a systemic mismatch: organizations are attempting to integrate high-speed AI capabilities into slow-moving, human-centric infrastructure. The reported solutions, such as pre-provisioning resources, implementing robust governance, and developing independent data and memory subsystems, represent efforts to retrofit these legacy systems. However, the underlying tension between the speed of AI innovation and the pace of infrastructure modernization suggests a need for more fundamental architectural shifts. Over the next decade, organizations will likely face increasing pressure to decouple AI operations from traditional IT, potentially leading to the rise of specialized AI infrastructure or a complete reimagining of enterprise computing paradigms to accommodate AI's distinct operational requirements.
AI-generated to prompt reflection — not editorial opinion, not advice, not a statement of fact. How this works.