The human head start
One observation from our discussions keeps coming back. People find Minecraft hard to master without tutorials, wikis or experienced friends, and that's with enormous prior knowledge. We know from real life how movement works, that a tool called a "pickaxe" is probably for breaking rock, that iron is tougher than stone, that valuable minerals are deep underground, and that following a cave down is easier than digging straight through.
A learner without that background, a baby or a blank neural network, faces a far harder problem. That makes the results of systems like DreamerV3 more impressive, not less. It also sharpens the question. If we want a learner that finds solutions we haven't already found, it needs to connect dots on its own, building abstractions from experience that transfer to situations it hasn't seen.
Prediction as a route to abstraction
Working synthesis
A useful world representation encodes relationships and possible futures, not just compressed snapshots. One plausible route to abstraction: recurring events share predictive structure across space or time, and a learner that captures that shared structure has, in effect, formed a concept. Two different caves "mean" the same thing if they predict the same kinds of futures.
Internal replay or imagined experience could then let a learner reuse and connect discoveries without repeating every external interaction. That's cheaper than doing everything for real, and it's one way experience could compound.
What neuroscience suggests
Literature
- The hippocampus as a predictive map (Stachenfeld, Botvinick and Gershman, 2017). Using the successor representation, this work explains hippocampal place cells as encoding expected future occupancy rather than location alone. Representations can be about where you're likely to go next.
- Schemas and the medial prefrontal cortex (Bein and Niv, 2025). A perspective linking schema learning, reinforcement learning, dimensionality reduction and memory reactivation. It's a bridge from recurring environmental structure to reusable knowledge.
- Planning and replay (Jensen, Hennequin and Mattar, 2024). A recurrent network model that learns when to spend computation on internal rollouts, relating them to human behaviour and hippocampal replay. It's an example of adaptively allocating thinking time.
Where prediction falls short
We want to be careful here, because "it's all prediction" is a tempting overreach.
- A successor representation is policy-dependent. It describes what happens under the behaviour you've been following, not a complete causal model of the world.
- Accurate prediction under observed behaviour doesn't establish counterfactual understanding, knowing what would happen if you did something you've never done.
- A neuroscience model that explains some replay data doesn't demonstrate general autonomous invention.
- Prediction, planning, causal intervention and goal generation are separate capabilities. A system can be excellent at one and poor at the others.
There's also the engineering hazard of model exploitation. A planner optimizing inside a learned model will find the places where the model is wrong in its favour. Work such as When to Trust Your Model (Janner et al., 2019) suggests short imagined rollouts branched from real experience. That's a reminder that useful imagination doesn't mean simulating everything.
Selective attention
An earlier idea of ours was attention-like retrieval over past observations and actions: rather than processing everything a learner has ever seen, retrieve what's relevant. That addresses cost. But learning what's relevant under delayed reward is part of the original problem. Selective attention alone doesn't solve credit assignment.
Combinatorial discovery
The hardest version of this question is combinatorial. The most impressive Minecraft machines (technical players' TNT-duplication world-eaters, for example) combine many separately discovered mechanisms: TNT destroys blocks, redstone triggers TNT, pistons move blocks, slime carries piston motion, certain configurations repeat. The space of possible combinations grows explosively with every mechanism known.
Humans manage this with abstraction, analogy, and a lot of shared culture. We know of no system that does this kind of combinatorial recombination end to end from raw experience. Brian Arthur's account of technology as combinatorial evolution suggests this is how real technological progress works too.
What we don't know
- Whether prediction alone is enough for useful abstraction, or whether intervention, memory or selection must contribute.
- How abstractions get composed into new ones, and what keeps that composition from exploding combinatorially.
- How a learner can know where its own model is reliable, and when it's worth paying for real evidence instead.