World models

Evidence that AI trained only on text builds an internal picture of how the world works, rather than only matching patterns of words.

A world model is an internal representation of how things are and how they work, which a system can use to predict what happens next. The question is whether language models have one, or only statistics about which words tend to follow which.

Several studies have looked inside models and found such representations. A model trained only on lists of moves in the board game Othello, and never shown a board, turned out to keep track of the board internally, and editing that internal board changed the moves it predicted (Li et al., 2023). Llama-2 models were found to represent where places are and when events happened, with individual “space neurons” and “time neurons” (Gurnee and Tegmark, 2023). Anthropic found that Claude, asked for the capital of the state that contains Dallas, first represents “Texas” and then “Austin”, and that when writing a rhyming poem it picks the rhyming word before writing the line.

These are partial models, not full understanding, and the AI Safety Atlas notes that some knowledge in models looks more like a lookup table. But the evidence goes against the idea that these systems only repeat surface patterns. For safety it cuts both ways: world models make systems more capable, and they are what interpretability research tries to read.