Future of Software Development Retreat Highlights: Harnesses, Self-Hosting, and Agent Control

Recent discussions at the Thoughtworks Future of Software Development Retreat reveal a growing focus on "harnesses" for managing AI model interactions, particularly context management. While context windows are expanding, ensuring models focus on relevant information remains a challenge. Strategies include limiting context size and employing more robust validation techniques like property-based testing and formal methods, even if only for verification by domain experts. The utility of these "harnesses" is debated, with some questioning if future, more sophisticated models will render them obsolete, while others believe they will remain crucial for efficiency and enabling weaker models, including locally hosted open-weight options.

The increasing cost of AI model tokens is driving interest in self-hosting open-weight models, which are rapidly closing the gap with frontier models. Beyond cost savings, a desire for "model sovereignty" and enhanced information security are significant motivators, especially given potential government interventions. Attendees noted that companies have been self-hosting for up to two years. The complexity of managing GPU inference data centers and the required talent present challenges, mirroring past issues with private cloud adoption. Professional services firms may find opportunities in assisting with this infrastructure. Cost control also involves selecting appropriate models for specific tasks, potentially with AI brokers guiding the choice. Fine-tuning models for particular domains is expected to increase, leading to reduced reasoning needs, lower token consumption, and operational cost savings.

Kief Morris presented a unifying narrative for the retreat's diverse sessions, centering on the core question of how much autonomy to grant AI agents and how to maintain confidence in their outputs. This theme spans discussions on code review, incident management, and the spectrum of operational models. The fundamental choice revolves around the "unit of work" delegated to an agent: its size, scope, preparation, verification, and the safeguards implemented. Separately, Sam Ruby's session on "Bring me a Rock" explored how LLMs transform traditional management dysfunction into a viable strategy. When applied to tireless AI agents, iterative refinement becomes efficient. The discussion also addressed whether non-engineers should direct LLMs, reframing the issue as a management decision to "hire" an AI rather than a tool, aligning with Peter Drucker's principle of managing by objective when workers possess specialized knowledge.