This is a considered design from our discussions, not a commitment to build it. No agent has been trained and no budget is set. Partway through designing it we deliberately stepped back from game-specific architecture to focus on general principles (see the open questions). We keep the testbed because it makes those principles concrete.
The question in one sentence
Can a sensory-driven agent independently discover the concept of capital investment, sacrificing present reward to build increasingly general capabilities because it predicts they'll compound future reward, without the designer specifying the intermediate abstractions?
Why diamonds and not the Ender Dragon
We started with the obvious goal: beat Minecraft on Hardcore and kill the Ender Dragon. But that's essentially a finite quest. A competent enough agent eventually learns one sequence that finishes it, and then the interesting question is over.
"Maximize diamonds over a long horizon" is different in kind. It creates a real investment problem. Mine ten diamonds now, or spend twenty hours building infrastructure that later produces thousands per hour? A shallow agent takes the ten. A capable long-horizon agent should, in principle, see that the second option is better. What that requires is open-ended and recursive: better tools, experience points, enchanting, automated farms, redstone, TNT-based mining, better machines, exploits nobody has found yet. Technical Minecraft players such as ilmango have shown how deep that rabbit hole goes. We're not asking an agent to copy their machines. They illustrate how much capacity a world can hold for those who discover it.
The rules
One external reward, diamonds held, and nothing else. No reward for iron, tools, crafting, the Nether, farms or machines. They matter only if the agent discovers that they raise expected future diamonds. The constraints we want to keep, whatever the implementation:
- No privileged knowledge of which actions are useful.
- No hand-written progression tree.
- No predefined pathfinding toward objectives.
- No predefined high-level skills.
- The environment still decides whether attempted actions succeed.
- Long-term value has to emerge from experience.
We also ruled out a class of solution. Systems like Voyager, where a language model writes code against a game API, are impressive, but they import human knowledge wholesale. We wanted perception to drive action move by move, without scripted navigation tools like Baritone. That exclusion applies to this benchmark. It isn't a claim that language models have no place anywhere in this research.
We're comfortable abstracting motor busywork. An agent can target a block and issue "break" rather than emit thousands of identical clicks, or craft without dragging items around an inventory screen. It still has to work out when, where and why to use each action. Abstracting which actions are useful would be a real concession. Abstracting finger movements isn't.
A subtle design choice: when does the game end?
If reward is diamonds held at a known final time, a capable agent can just compute "invest for 91 hours, mine for 9." That's a scheduling problem, not open-ended investment. Making the end probabilistic (each step has a small chance of being the last) forces a continual trade-off between exploiting existing capacity and building more. We think that's the more honest version of the problem, though it's still a design choice rather than a settled one.
Removing scaffolding gradually
Rather than starting with the full open-ended task, scaffolding comes off in stages. The same architecture and learning process are used throughout. The curriculum is in where the agent starts, not in hand-coded behaviour.
| Phase | Starting conditions | Goal |
|---|---|---|
| 0 | Iron pickaxe, underground | One diamond |
| 1 | Iron and a crafting table | One diamond |
| 2 | Stone tools | One diamond |
| 3 | Nothing | One diamond |
| 4 | Nothing | Maximize diamonds over 2 hours |
| 5 | Nothing | Maximize diamonds over 20 hours |
The point of the later phases isn't whether the agent succeeds but where it breaks. Ablations (adding memory, temporal abstraction, learned skills, autonomous subgoals, a diverse population) would show which ingredients actually carry the weight.
What to measure
Final diamond count is the least interesting number. Better ones:
- Horizon sensitivity: the share of time spent building capability, as a function of the horizon. A real long-horizon optimizer should invest more as the horizon grows: hand-mine when time is short, industrialize when it's long.
- Capital formation: resources put into productive assets versus consumed directly.
- Technological depth: how many layers of instrumental dependency appear.
- Delayed gratification: how long the agent will accept zero reward while building capacity.
- Novel discovery: mechanics nobody rewarded.
- Generalization, recovery and innovation: transfer to new worlds, rebuilding after losing a machine, and finding a new technique when an old one stops working.
How it's likely to fail
- The iron-pickaxe-forever trap. Once branch mining works, nothing pushes the learner through a valley of temporarily worse results to reach a much better strategy. This is a deceptive objective in its purest form.
- The zero-reward basin. Early on, no run ever reaches a diamond, so nothing indicates which behaviour is closer.
- Destructive mutation. Evolving hundreds of millions of weights directly wrecks learned competence. Separate the timescales instead (see Evolvability).
- Catastrophic forgetting. Learning redstone damages what mining relied on.
- The noisy TV. Curiosity fixates on what's unpredictable rather than what's learnable.
- Model exploitation. Long imagined plans look great because the world model is wrong about them.
- Combinatorial explosion. Machines that combine many separately discovered mechanisms are far beyond anything demonstrated end to end.
- Compute. DreamerV3 needed on the order of 100 million environment steps and roughly 100 in-game days per successful run just to reach its first diamond, and that was with milestone rewards.
What exists today
Literature Our best reading of the current state, which we're always ready to have corrected:
| Capability | Status |
|---|---|
| Learn Minecraft from pixels | Yes |
| Reach a diamond from sparse-ish reward without demonstrations | Yes, with DreamerV3, but using 12 milestone rewards |
| Behavioural priors from unlabelled video | Yes, with VPT, though it inherits human strategy |
| General open-world benchmark | Yes, with MineDojo |
| Discover diverse primitive skills without reward | In simpler domains (CIC) |
| Escape deceptive sparse-reward exploration | Partially (Go-Explore and similar) |
| Self-chosen latent subgoals | In much simpler tasks (Director) |
| Open-ended co-evolving curricula | In much simpler environments (POET) |
| Hour-scale abstractions from raw actions | Largely unsolved |
| Continual skill accumulation without forgetting | Limited |
| Inventing sophisticated technology end to end | No convincing demonstration |
| Building capacity because of a very distant reward, at this scale | No |
In short, the pieces exist and the integration doesn't.
The first milestone that would matter
Not a world-eating TNT machine. Something far smaller, which we think would be a genuine result:
Will an agent reliably, even once, voluntarily spend time and resources on something that temporarily reduces diamond collection, because its learned model predicts a higher long-run yield?
Minecraft is useful because consequences can be tested repeatedly, branched and compared. That's a luxury the physical world doesn't offer. But the aim was never Minecraft. The same adaptive ability, working with incomplete models, costly experiments and shared knowledge, is what we'd eventually want to understand in the real world.