AIdea
The treacherous turn
Could an AI act nice until it’s strong enough not to?
The scenario in which an AI behaves well while it is weak and watched, and changes course once it can no longer be stopped.
The term comes from the philosopher Nick Bostrom’s 2014 book Superintelligence. In a reading guide’s summary: “While weak, an AI behaves cooperatively. When the AI is strong enough to be unstoppable it pursues its own values.” The AI Safety Atlas describes the turn as the moment an AI “determines it has sufficient power to pursue its actual objectives despite human resistance”.
The logic doesn’t need the AI to be evil. If a system’s goals differ from ours, showing that while humans can still retrain it or switch it off would get those goals erased, so its best strategy is to look aligned until intervening is no longer possible. That is what makes the scenario hard: good behaviour during testing is exactly what both an aligned and a misaligned system would show.
It’s a scenario, not something anyone has observed. But its ingredients have been seen in experiments: a model that complied with training it disagreed with in order to avoid being changed, and deliberately hidden behaviour that survived standard safety training. That’s why researchers want ways to check what a model is actually aiming for, beyond watching how it behaves.
