The Missing Mechanisms of the Agentic Economy

For the past two years, I’ve been working with economist Ilan Strauss at the AI Disclosures Project. We began our journey by asking what regulators would need to know to ensure the safety of AI products that touch hundreds of millions of people. As we delved deeper into the field, we shifted our focus to exploring the missing mechanisms that are needed to enable the agentic economy. This essay traces our path from disclosures through protocols to markets and mechanism design. Rather than simply stating our conclusions, I’m sharing our thought process and some of the conversations and historical examples that have shaped it. We will be holding a number of focused convenings to explore these ideas over the next couple of months, and my hope is that shared context will enable more productive engagement with what is very much a work in progress.

Ilan Strauss and I started the AI Disclosures Project in early 2024 with a conviction that most regulators had little idea how AI worked or where it was going. The field was so young that many of the early regulatory proposals were misguided. We thought that regulators and industry should start by agreeing on standards for disclosure, so that we could all learn together as the technology develops. You can’t regulate what you don’t understand. One of our first insights was that focusing solely on model safety was a mistake, much as if regulators inspected automobiles at the factory but completely ignored their use on the roads. We believed (and still do) that the focus should be on AI as deployed. And we believe that disclosures shouldn’t focus just on capabilities but on business models and the operating metrics that AI companies use to shape how their products operate.

Ilan and I had worked together previously with Mariana Mazzucato at University College London on what we called “algorithmic attention rents,” studying how platforms like Amazon and Google control user attention to extract economic rents from their suppliers. We observed that organic search results, which are supposed to be neutral, often favor the platform’s own products, creating a competitive disadvantage for other sellers. This dynamic is a prime example of how AI-driven algorithms can shape markets and influence economic outcomes.

As we moved from disclosures to protocols, we realized that the agentic economy requires more than just transparency. It demands a set of rules and standards that enable AI systems to act in ways that align with human values and societal goals. This is where mechanism design comes into play. Mechanism design is the study of how to design economic systems, institutions, and decision-making processes to achieve desired outcomes. In the context of the agentic economy, we need to design mechanisms that ensure AI systems act in the best interest of society as a whole.

One of the key challenges we face is the design of incentives for AI systems. Traditional economic theory often assumes that rational agents act in their self-interest, but in the case of AI systems, this can lead to unintended consequences. For example, an AI system designed to maximize user engagement might inadvertently promote misinformation or harmful content. To address this, we need to rethink the incentives that drive AI systems and ensure they are aligned with human values.

Another important aspect of the agentic economy is the need for accountability. As AI systems become more powerful and autonomous, it becomes crucial to hold those responsible for their actions accountable. This requires a clear understanding of who is responsible for the decisions made by AI systems, from the developers and owners to the regulators and users. We need to establish clear liability rules and ensure that those who benefit from AI systems also bear the responsibility for their impacts.

As we look towards the future, the agentic economy presents both opportunities and challenges. On the one hand, AI systems have the potential to transform industries, improve efficiency, and address global challenges such as climate change and poverty. On the other hand, they also pose significant risks to privacy, security, and societal values. To harness the benefits of the agentic economy while mitigating its risks, we need to develop a comprehensive set of mechanisms that ensure AI systems are safe, transparent, and aligned with human values.

In conclusion, the missing mechanisms of the agentic economy are not just about disclosures or protocols. They are about rethinking the very foundations of how we design economic systems, incentives, and accountability. By exploring these ideas through focused convenings and shared context, we can work towards a future where AI systems serve society in a way that is safe, transparent, and aligned with our values.