Enterprises Race to Buy AI Infrastructure Faster Than They Can Track Costs
A recent survey of 107 enterprises reveals a significant "compute gap" where AI infrastructure spending is rapidly outpacing the ability to monitor and control associated costs. While most organizations currently rely on hyperscalers and model provider APIs for their AI workloads, a majority plan to switch or add specialized compute providers within the next year, with many targeting the next quarter. Purchasing decisions are primarily driven by integration with existing systems and total cost of ownership, rather than headline token prices. This focus on TCO is notable given that most enterprises lack clear visibility into their unit economics. Specifically, 83% of organizations report GPU utilization rates of 50% or less, and fewer than half (44%) rigorously track their compute expenses. This indicates a trend of heavy, fast-moving investment in AI infrastructure that is outpacing the necessary visibility and control mechanisms. The research highlights that only 21% of enterprises are running AI in production at scale, yet spending intentions are accelerating ahead of this maturity. A substantial 45% plan to evaluate AI-specialized clouds, a segment currently used by very few. Furthermore, a clear majority (64%) intend to switch or add an infrastructure provider within twelve months, with 38% planning changes within the next quarter, signaling unusually high churn for such a foundational technology category. When selecting providers, integration (41%) and total cost of ownership (35%) are prioritized over per-token pricing, which influences only 8% of decisions.
AI compute investment is accelerating rapidly, driven by enterprise ambition, but this growth is not matched by robust financial oversight. Organizations are prioritizing future capabilities and vendor diversification over current cost-efficiency, leading to potential overspending and underutilization of existing resources. This dynamic suggests a market where the perceived strategic advantage of early AI adoption outweighs immediate financial prudence. The high intent to switch providers indicates a search for optimal solutions, but the lack of cost visibility could lead to suboptimal choices or a cycle of continuous vendor evaluation without achieving stable, cost-effective operations. As AI inference scales, the shift from raw compute power to memory bandwidth will become critical, a factor currently underestimated by many enterprises, potentially creating future infrastructure bottlenecks and unplanned expenses.
AI-generated to prompt reflection — not editorial opinion, not advice, not a statement of fact. How this works.