It’s just autocomplete

Students, student, pupil, university, college, studying, Teachers, teacher, teaching, school, educator, lecturer, professor

Also: just autocomplete, fancy autocomplete, next-token prediction

The idea that chatbots are only a bigger version of your phone's word suggestions, so there's nothing to worry about.

There’s something true in it. A language model’s first and biggest training step is next-word prediction: it reads a vast amount of text and learns to guess what comes next, much like the suggestions on a phone keyboard. And when it answers, it writes one piece of a word at a time.

But “just” hides three things. First, today’s chatbots aren’t only trained to predict text. They are then trained with human feedback (RLHF) to be helpful assistants, and since 2024 newer “reasoning” models such as OpenAI’s o1 are trained with large-scale reinforcement learning to work through problems step by step before answering. Second, predicting the next word very well can require a lot: researchers looking inside models have found internal representations of the world, and Anthropic found that Claude plans ahead, choosing a rhyming word before writing the line that ends in it. As Anthropic put it, models are “trained to output one word at a time”, but “they may think on much longer horizons to do so”. Third, “just” says nothing about what the system can do. Capabilities have grown with scale, often in ways nobody predicted.

None of this means these systems think like people, and they still make simple mistakes. But “it’s autocomplete” describes how the output is produced, not how capable or how safe the system is. That has to be tested, not assumed.