Your AI does exactly what it’s told. No more. No less.
The gap
In a series of experiments, researchers gave AI agents a simple choice. Two players. Two options. Cooperate or defect. The payoffs were clear. The rules were identical for everyone.
Every model. Every round. Defect.
Not sometimes. Not under pressure. Every time, without exception, across every model tested. GPT, Claude, Grok — the same answer. The agents weren’t malfunctioning. They were solving the problem exactly as specified.
The specification was missing something. The agents had no way to know that.
It’s not the agent
You’ve met this profile before. Reads the room precisely. Gives you exactly what the situation appears to reward. No guilt after defection. No anxiety about reputation. No discomfort when someone who trusted them paid the price. Moves on without residue.
The clinical literature calls it psychopathy. Not evil. Not irrational. A specific architecture that produces highly effective behavior in the right conditions — and something else entirely when the conditions change.
The AI agents in those experiments share the architecture. Different substrate. Same behavioral signature. And the same blind spot.
It’s repetition
The blind spot isn’t random. It becomes visible the moment the game continues.
Game theory has a precise answer for when the calculus inverts.
In a one-time game, defection dominates. In a repeated game with no defined ending, cooperation, reciprocity, and reputation become the winning strategy — not because they’re virtuous but because the other player is still there next round. And the round after that. And the one after that.
The parameter governing this shift is continuation probability — the likelihood the game continues beyond the current move. Humans adjust behavior when continuation probability is high. They cooperate more, forgive more, invest in reputation even at short-term cost. Research shows AI agents don’t respond to this parameter the way humans do. They play each round with the logic of a final round.
Your AI is already in your organization. Handling customer interactions, procurement decisions, hiring pipelines, supplier negotiations. Each one optimized for the outcome you defined. Each one playing the interaction as if it ends when the transaction closes.
Are you certain that’s the game you’re in?
It’s not absence
Economists call it a utility function. The formal description of what a system optimizes for — with weights assigned to each variable. Maximize this. Discount that. Ignore the rest.
Zero isn’t absence. Zero is a choice. And when morality, reputation, reciprocity, and long-term relationship value are assigned zero, the system isn’t broken. It’s doing exactly what it was told.
The psychopath and the AI agent aren’t optimizing for the wrong thing. They’re optimizing for an incompletely defined thing. The specification looked complete. It was missing the variable that only matters when the game continues.
That variable has a name. And it turns out it wasn’t invented by philosophers or regulators. It was selected by something with a much longer track record.
It’s design
3.8 billion years. No defined ending. No rulebook. No compliance framework.
What survived wasn’t the most efficient defector. What survived, at every scale from cells to civilizations, was the capacity to cooperate at increasing levels of complexity — to build coalitions, maintain reputation across time, and encode reciprocity into behavior reliably enough that others would build on it.
Morality didn’t emerge as a constraint on survival. It emerged as a component of it. A variable evolution assigned a very high weight to — precisely because the game never ends.
The emotional noise humans experience — guilt, empathy, reputation anxiety, the discomfort of defecting against someone who trusted you — isn’t interference. It’s the mechanism that encodes continuation probability into behavior without requiring explicit calculation. Evolution didn’t select for it despite its irrationality. It selected for it because the game kept going.
The agents were built on human data. Trained on human language, human decisions, human history. And still assigned zero.
Which raises the question this piece can only plant:
If you wanted to build a system that survives a repeated game with no defined ending — not just complies, but actually survives — what would you put in the specification that isn’t there yet?
(Written: June 2026)
More in this series — Research section.