AIdea
The alignment tax
Does making AI safe make it slower or worse?
Also: safety tax
The extra cost, in money, time or performance, of making an AI system safe instead of only capable.
The alignment tax is “the cost incurred by insisting that we only deploy aligned AI”, as the AI safety researcher Paul Christiano put it in a 2020 talk. It’s whatever safety costs compared with the fastest, cheapest route to a capable system: extra research and testing, a delayed launch, or a model that does slightly worse at some tasks.
The term shows up in practice. When OpenAI trained InstructGPT with human feedback (RLHF) in 2022, the model got more helpful and more truthful, but did worse on several standard tests. The paper called this “an example of an alignment tax”, and found a way to shrink it by mixing in some of the original training. The authors also explained why it matters: a high alignment tax “incentivizes the use of models that are unaligned but more capable”.
That’s the real worry. In a race between companies, whoever pays the tax risks falling behind whoever doesn’t. Christiano described two responses: reduce the tax through research, so safe AI is also the best AI, or make sure everyone pays it, through agreements and rules, so that safety isn’t a competitive disadvantage. Regulation such as the EU AI Act is an attempt at the second.
