AI Agents Need Fiduciary Duty, Not Just Capability, to Ensure Trust

In dynamic environments, the trustworthiness of AI agents has become a critical runtime problem, moving beyond static pre-deployment evaluations. Vin Sharma, Founder and CEO of Vijil, highlights that organizations often treat AI agents like traditional software, failing to account for their ability to perceive, reason, act, and learn from real-world interactions. This is problematic because the underlying models are trained on static data, making their understanding of the world outdated by the time they are deployed.

Traditional AI benchmarks fall short in assessing agent trustworthiness because they are static, imperfectly model reality, and can be memorized by models, thus failing to predict real enterprise behavior. Sharma proposes the concept of a "fiduciary agent," which, similar to professions with a formal duty of care, must operate with duties of competence, care, and loyalty to the enterprise. Trustworthiness is defined by whether the benefit of delegating a task outweighs the risk of failure, with risk comprising reliability, security, and safety. Vijil's testing methodology focuses on purpose, personas, and policies to adapt to specific workflows, simulate diverse user and threat profiles, and enforce organizational rules.

Failures often emerge in production due to environmental changes like data and concept drift, new attack vectors, and emergent issues in multi-agent systems, such as collusion. Continuous trust management involves discovery of ungoverned agents, assigning distinct workload identities with restricted permissions, and enforcing policies. New key performance indicators include "time to trust" and "time to recovery." Sharma emphasizes that trust in AI systems must be built into the infrastructure, becoming continuous, trackable, and measurable for ongoing improvement.