Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignment
Cameron Tice, Puria Radmard, Samuel Ratnam, Andy Kim, David Africa, Kyle O
Alignment Pretraining by Geodesic Research·Read paper
Abstract
LLMs trained on data about misaligned AIs themselves become less aligned. Luckily, pretraining LLMs with synthetic data about good AIs helps them become more aligned. These alignment priors persist through post-training, providing alignment-in-depth. We recommend labs pretrain for alignment just as they do for capabilities
Make this part of your paper trail
Save this paper to a shelf, write a review, and keep your own notes.