Introduction
I’m excited by deception probes. When I mention this, I’m sometimes asked “Do deception probes work?”
But I think there are many applications of deception probes, and each application will require probes with different properties, i.e. whether a deception probe works will depend on what you’re using it for. Furthermore, whether one deception probe works better than another will also depend on what you’re using them for. This remark sounds a bit trivial, but I didn’t appreciate it initially.
In this document, I'll enumerate 18 different applications of deception probes, and what properties each application requires. This are just my very rough guesses, just to kickstart some public conversation.
Applications of deception probes
1. Monitor current models
Problem Statement: AIs at the current capability level may be important for future safety work, e.g. trusted monitoring. However, current LLMs engage in strategic deception. This is despite being post-trained to be honest, and without any fine-tuning or instructions to be deceptive![1] But we can’t deploy current AIs for trusted monitoring if they will occasionally lie about whether they think the observed activity is suspicious.
How deception probes help: Train deception probes on the weak model (e.g. Claude 4) and use the weak model to monitor the activity of the strong model (e.g. Claude 6). If the weak model claims the activity is not suspicious, then apply the deception probe to that claim. If the pro...
Make this part of your paper trail
Save this paper to a shelf, write a review, and keep your own notes.