Market Intermediaries: A post-AGI Vision for the Economy

An outline of an economic mechanism for human flourishing after AGI. Also, a brief look at an experiment the Meaning Alignment Institute will run later this summer to prototype it.

Executive Summary

Current market incentives for AI are misaligned with human flourishing, whether via AI arms races, isolating hyper-stimuli, or the displacement of meaningful human labor. Common responses like regulation and wealth redistribution tend to centralize power and leave perverse incentives unchanged. We propose AI-enabled market intermediaries that pool consumers, measure outcomes, and pay suppliers based on delivered human benefit rather than misaligned metrics. MAI will pilot this with 200 people this summer—a first step towards a "meaning economy" where humans retain their economic agency and are free to pursue what’s meaningful to them.

Market Misalignment and Human Flourishing

Many AI risks are driven by markets misaligned with human flourishing:

• There are markets for things that are bad for us, such as AI arms races among nations and labs, the market for AI girlfriends and other hyper-stimulus, isolating distractions, and markets for political manipulation and destabilization.

• There are markets that displace us entirely, where AGI eliminates meaningful work, leaving humans as passive consumers dependent on UBI stipends granted at the discretion of those who control AGI-generated wealth.

We can summarize these as failures of markets to put human values and meaning on a par with (what should be) instrumental goals like engagement, ROI, or the efficient use of resources.

There are three common responses to these problems with markets. Each centralizes power:

• One response is to keep the number of market actors low, and to coordinate between them such that they voluntarily resist market incentives for a time. This can only work when the number of players is small enough that they can meet and agree (for ex. safety evals, AI lab pledges).

• Another response is regulation, such that market actors mostly follow market incentives but are forced to stop just short of the bad outcomes. Regulation centralizes power in the state (or as is likely necessary, a global governmental body) and in the small number of large actors which can afford to comply with regulations. The globalized nature of modern tech companies also means that regulation in one country often means operations can continue as usual in other markets with less strict rules (for ex. EU AI act, AI executive order, Pause AI).

• A third response is to let a bad outcome occur (such as human displacement) but then redistribute wealth to minimize the damage. Redistribution also centralizes power: in the hands of AI providers accumulating capital, and in the states which redistribute it (for ex. UBI).

Not only do these approaches centralize power, they also don’t actually re-align markets: markets continue to pull the wrong way, patched by pledges, regulations, or redistributions.

Why does this happen?

One way to view this problem is that the wrong outcomes are being monitored and contracted against. Contracts often refer not to the human-flourishing-related outcomes that matter to human beings, but to more measurable proxies which often diverge from them.

Looking at this in depth, there are four interrelated problems:

1. Incomplete contracts and measurement costs. Contracts cannot specify all contingencies, especially for complex outcomes like "human flourishing." Traditionally, the cost of measuring and verifying such outcomes has been prohibitively high, so suppliers contract on easily measurable proxies (hours worked, subscriptions sold) rather than the outcomes users actually care about. A supplier will charge by the hour or subscription if they can, simplifying cash-flow and lowering risk, but moving incentives away from what's ideal for consumers.

2. Market power and bargaining asymmetries. Large suppliers face millions of atomized consumers. This asymmetry means suppliers write standardized, one-size-fits-all contracts that minimize their risk while consumers face take-it-or-leave-it choices. The transaction costs of negotiating billions of individual contracts are prohibitive relative to each consumer's value, preventing customization.

3. Externalities and network effects in consumption. Individual flourishing depends on community wellbeing—it's hard to flourish if your neighborhood is struggling or your family's needs are unmet. The ideal contract might account for outcomes across groups (e.g., a concert where all friends enjoy themselves). But such interdependencies would increase supplier risk and complexity, so contracts remain individualized, ignoring these social spillovers.

4. Information asymmetries and measurement bias. Suppliers know more about their products' true effects than consumers do. Worse, suppliers control the dashboards and metrics used to measure outcomes. These are optimized for growth and engagement—for monitoring whether marginal new users purchase—rather than whether various user segments achieve their hoped-for outcomes. This masks the divergence between what's delivered and what users actually want.

These problems compound: provision drifts away from true outcomes, externalities go unpriced, information asymmetries deepen, and biased metrics masquerade as success.

Markets are unlikely to self-correct because economies of scale in contracting and assessment naturally lead to standardization. Even personal AI agents won’t change this dynamic — if they negotiate one-off contracts, they will face the same bargaining asymmetries. Google has no incentive to manage a billion different contract types, when each is worth only $20.

A New Approach: Market Intermediaries

We argue that powerful AI can re-align markets with genuine human goods—eliminating the failure modes above—without requiring state intervention. Our solution is a new type of market mechanism: market intermediaries.

A market intermediary acts as an AI agent that contracts on behalf of multiple users, evaluating outcomes they value and bundling them into custom "enterprise-level" deals—making them worthwhile for large providers to consider. If they accept that deal, sellers will be paid by the intermediary based on the 'goodness' (as defined by the buyers) they produced, rather than by the services rendered.¹ In other words, the market intermediary uses non-market data about good outcomes for buyers to route resources from consumers to providers.

Market intermediaries make outcome-based or performance-based contracts ubiquitous in a market, by externalizing the costs of specification and auditing, and removing principal-agent problems in contract design.²

Specifically, intermediaries address each of the four economic problems identified above:

• Incomplete contracts: By using AI to specify and measure complex outcomes like human flourishing. (We have a great deal of work in this area: 1 2.)

• Market power asymmetries: By aggregating consumers to negotiate enterprise-level deals with customized terms.

• Externalities: By bundling related users' outcomes together in contracts that account for social interdependencies.

• Information asymmetries: By operating independent assessment systems that align incentives properly.

This restructuring provides additional benefits:

• Risk management. Outcome-based (rather than deliverable-based) contracting shifts risk in the direction of suppliers. Aggregating many customers lets intermediaries tune that risk, striking a transparent balance rather than pushing it wholesale onto either side.

• Principal-agent problems with assessment. The intermediary can operate neutral dashboards that resist both suppliers’ growth-metric bias and customers’ temptation to misreport.

• Consumer atomism. Market Intermediaries are an ideal place to bundle together contracts with related concerns, allowing consumer’s preferences for various kinds of solidarity and togetherness to enter into contracts.

• Contracting costs. Similar to health insurance companies, market intermediaries can centralize and reduce the costs of contracting, while avoiding the problems when suppliers are in charge of it.

We suspect that, in a world of market intermediaries, contracts will be more likely to address deeper needs, address bundles of users rather than the easily-isolated individual, and spread risk more equally between suppliers and users.

Evidence of Viability: Lessons from Existing Markets

The feasibility of outcome-based contracting isn't just theoretical—it already works in several industries where the right conditions exist: aircraft maintenance is provisioned via performance-based or “power-by-the-hour” agreements, where what matters is whether the aircraft can fly at the scheduled times. In health insurance, there are value-based contracts or capitation payments: agreements between insurers and health providers that tie payment to patient or population health outcomes, not to services rendered. In cloud compute, service-level agreements tie payments to cloud service uptime.

Currently, outcome-based contracts are only prevalent where three conditions are met.

• Quantification of benefit. The outcome desired by consumers must be easily quantified and measured. Examples are quality-adjusted life years (QALYs) in a hospital’s catchment area or an aircraft’s serviceable flight hours.

• Cheap assessments. Outcome-based contracting requires one or both parties to estimate outcomes, assess and audit those outcomes regularly, and insure against errors in their estimates. For outcome-based contracting to be affordable, these transaction costs must be relatively low.³

• Symmetric information. For both sides to converge toward an optimal contract, neither can have a decisive advantage. With cloud compute, for instance, providers often have an information advantage. That drives towards metered, but not outcome-based contracts.

Powerful AI changes all three of these factors: it can extend the assessment of benefit into qualitative domains; it can make assessment much cheaper; and it can occupy a third-party, auditable position to reduce information asymmetry in contract design. Outcome-based contracting can thus be extended to many more markets—even in domains with fuzzy assessment needs, where measuring ‘good outcomes’ requires qualitative interviews.

For example, consider an outcome-based contract between a consumer and an AI assistant company. The outcomes the consumer wants from an AI assistant are diverse and qualitative. This makes them difficult to specify in a contract, difficult to estimate in terms of likelihood, and it’s also difficult to assess whether they happened.

A market intermediary can make this feasible: it can interview users about the outcomes they’d like, or generalize from outcomes observed in a corpus of user-assistant interactions; it can structure a contract based on those outcomes; it can offer a spread to the AI assistant provider, that incentivizes them towards an aligned product, but manages their risk.

This means an intermediary could restructure the market for personal AI assistants, aiming it towards the flourishing of users, rather than keeping them engaged.

Similarly,

• An intermediary could restructure labor markets, making them about connecting people with work they find meaningful, given their personality and values, not just getting a job done.

• An intermediary could restructure markets for GPU compute, around a basket of pro-social causes, not just business objectives.

For the first time, it may be possible to address markets’ intrinsic misalignment with humanity at the root, rather than just patch over those misalignments with regulation or corporate pledges.

Components of a Market Intermediary

To illustrate the components of a market intermediary, let's imagine building one for car repair, a classic market with high information asymmetry.

A market intermediary is made of five components.

1. The Assessor tracks what benefits consumers want, what providers claim they can deliver, and whether they actually deliver. For car repair, it learns whether someone wants longevity over aesthetics, then monitors outcomes through interviews or performance data. Key challenges: capturing all real benefits (not just "car runs without problems" but also valuable mechanic education) and preventing gaming (shops bribing customers for false reviews).

2. The Bundler groups consumers with similar needs and matches them with relevant providers. It might bundle customers with the same car model or repair needs, then identify shops that specialize in those areas, considering factors like location, wait times, and expertise.

3. The Contract Builder creates outcome-based pricing with floor and ceiling prices. Confident shops offer wider spreads (lower floors, higher ceilings) while conservative ones offer narrow bands. A shop might bid differently for "car runs well for 6 months" versus "2 years."

4. The Central Mechanism allocates bundles to shops based on their bids, confidence signals, and past performance. This creates a market where high-performing, confident suppliers win more business while encouraging truthful bidding.

5. Working Capital Financing provides insurance or financing so suppliers like the car repair shops don't have to wait for outcome-based payments to resolve.

Our First Experiment

One place where market intermediaries are both easy to try and could be high impact is the human labor market for social flourishing. We hypothesize that (1) this market is tractable for reorganizing suppliers, interviewing users, assessing contracts, and shifting incentives; and (2) the results will illuminate both the mechanism of market intermediaries and prototype a post-AGI economy.

At the assessment level, an intermediary for this purpose would capture information about people’s sources of meaning as relates to their social life — when they are fulfilled and when they aren’t, how they are unfulfilled, as well as logistical information about their life, their finances, the people close to them, etc. (We have already developed detailed methods to assess human flourishing that go beyond simple metrics, capturing the nuanced and personal nature of what makes each individual thrive.) It would also check in repeatedly about how things are going.⁴

The intermediary would then match and route payments to in-person events, group outings with friends, venues, etc — or more specifically, the organizers and facilitators of such events, outings, etc. This search and matching would ”eat up” the transaction costs preventing such trades to take place in the regular market.

To evaluate such an intermediary, the Meaning Alignment Institute will build and test it this summer for 200 people, and collect data on (1) avoidance of relevant market misalignments (such as over-isolation, see above); and (2) satisfaction with the LLM’s match compared to what participants would do without it. One desired outcome is that actors feel they're better off having worked through the intermediary than making their own quality assessments and purchasing outside of it.

We believe this same process, if successful, could be implemented in other markets too. For example, intermediaries could revolutionize AI assistant markets by rewarding genuine user flourishing over addictive engagement, paying providers like ChatGPT and Anthropic accordingly, or be used for public infrastructure, converting developers' empty community promises into contracts with real teeth.

Towards a Meaning Economy

How does this avoid the disempowerment we warned about at the start? Market intermediaries is our first step in a greater project — one of fundamentally reimagining what economic activity looks like in an age of AGI.

Eventually, we envision an economic model where AI handles optimization and routine transactions while humans engage only in activities they find meaning in. Traditional producer-consumer categories would dissolve in this world—people would naturally flow between participating and contributing based on what’s meaningful to them, and goods and activities would be priced and incentivized accordingly. Rather than making humans economically obsolete and dependent on UBI, economic incentives would be recentered to preserve and enhance human meaning and agency.

Our experiment with market intermediaries is the first step toward this world. The meaning economy isn't a retreat from technological progress but its natural evolution—where human values and AI collaborate to create abundance in what actually matters.

We have much more to say about how this. Stay tuned.

Thanks to Matt Prewitt, Seb Krier, Toby Shorin and Scott Moore for comments on earlier versions of this document.

1. Equivalently, providers can buy insurance from the intermediary to cover them if they fail to provide a level of benefit to the consumer. These insurance premiums can go up and down based on the intermediaries assessments of benefit provided.

2. While intermediaries themselves could face principal-agent problems, several mechanisms keep them aligned: (1) They operate as audited nonprofits, preventing profit-driven distortions; (2) Multiple intermediaries compete in each market, giving users choice; (3) Their assessment systems are transparent and auditable; (4) Unlike suppliers, their sole purpose is accurate outcome measurement and payment routing.

3. Monitoring must be inexpensive relative to surplus; otherwise, as Holmström & Milgrom (1991) show, effort mutates back toward measurable inputs. In the airline industry, these costs are small compared to the overall expense of owning and operating an aircraft.

4. The easiest way to assess outcomes is to have people maintain a profile of fulfilled and unfilled sources of meaning, and what’s helped and what hasn’t. In the long run, much of this data can be inferred.