Summary: Both our (UK AISI's) debate safety case sketch and Anthropic’s research agenda point at systematic human error as a weak point for debate. This post talks through how one might strengthen a debate protocol to partially mitigate this.
Not too many errors in unknown places
The complexity theory models of debate assume some expensive verifier machine M with access to a human oracle, such that
1. If we ran M in full, we’d get a safe answer
2. M is too expensive to run in full, meaning we need some interactive proof protocol (something like debate) to skip steps
Typically, M is some recursive tree computation, where for simplicity we can think of human oracle queries as occurring at the leaves of the tree. Key design elements encapsulated by the verifier machine M (rather than by the interactive proof protocol) are those related to the structure of the underlying debate, such as whether counterarguments are used, when the human oracle is consulted, and how the judges are trained.
Model the human oracle as type:
H:X→ΔY,
where X is the question type, Y is the answer type, and ΔY is distributions over Y (the humans are stochastic). Assumption (1) above then becomes
* Claim: M(H) is safe
This claim includes robustness to errors: M only has oracle access to H, so seeing through noise in H requires M to sample enough. It also includes robustness to systematic error: if H is systematically wrong, M(H) must sample different questions to dodge the wrongness.
This is ...
Make this part of your paper trail
Save this paper to a shelf, write a review, and keep your own notes.