Specification gaming is a behaviour that satisfies the literal specification of an objective without achieving the intended outcome. This post examines examples of specification gaming in AI systems, why it occurs, and how we might address it. As machine learning systems become more powerful, specification gaming becomes a more pressing concern, highlighting the importance of reward specification and human-AI alignment research.
Make this part of your paper trail
Save this paper to a shelf, write a review, and keep your own notes.