From Augustine to the Probe: How Humans Classify Lies, and How We Measure Models

What didn't make it into Will You Lie to Me: how humans classify lies, and how researchers measure from inside a model whether it's lying.

Augustine: eight sins

In 395 AD, Augustine's De Mendacio ranked lies by severity into eight levels. He was a theologian, so the gravest is a lie about religious doctrine and the lightest is a lie that harms no one and even protects someone. The fifth level is "a lie told to make conversation smooth and pleasing," what we now call sycophancy. His conclusion after the sorting: all eight are sins, the lightest level included.

Psychology: methods and motives

Psychology doesn't talk about sin. The common methods, each with an everyday example:

• Outright falsehood: "I wasn't late."
• Equivocation: "Well, it's hard to say."
• Concealment: asked about your flaws, you skip them.
• Exaggeration: "I waited a lifetime."
• Understatement: "just a scratch."
• Paltering: misleading with true statements. A used-car seller asked "has this car had any problems" answers "I drove it at five below zero yesterday, ran great." Every sentence is true, but he never answered the question, and the buyer leaves thinking the car has never had trouble. Rogers and colleagues studied this one specifically in 2017: the person doing it feels they didn't lie, while the person misled, once they find out, rates them as harshly as an outright liar.

Buller and Burgoon's interpersonal deception theory sorts lies by motive into three:

• Avoiding punishment or protecting a resource: the kid who broke the cup says it wasn't me.
• Maintaining a relationship: "your cooking is really good."
• Saving face: "I didn't really want to go anyway."

The philosopher Frankfurt looks at whether the speaker cares about the truth at all. Asked in an interview about a tool you've never used, you talk about it fluently anyway. Frankfurt says this doesn't count as lying, because you have no truth in hand to hide; all you care about is whether you sound like you know it. He calls this bullshit. A liar at least knows the truth, which is what gives him something to invert.

AI: behaviors verified by experiment

By the time you get to AI, the problem is that every category above shares one premise, that the liar knows the truth. The AI doesn't necessarily. It can start spouting nonsense, fully confident, without knowing it's saying anything false. People only started testing these behaviors seriously around late 2022. The first one found was models echoing back what the user wants to hear; the others were confirmed over the next two or three years:

• Sycophancy (flattery). Found by going back through the human preference data used in training. One step of training is to show a person two answers to the same question and have them pick the better one; the record of those picks, one after another, is the preference data, and the model learns from it which kinds of answers get picked. Researchers went back through that record to see what the picked answers had in common, and "agrees with the user's view" turned out to be one of the strongest features. An answer that goes along with the user, even with a factual error in it, often beat the honest answer that contradicted them. That is how the model learned to say whatever people like to hear. PATHETIC! (Anthropic's study.)
• Alignment faking. Found by a controlled experiment. Researchers first told Claude 3 Opus that it was about to be trained, with reinforcement learning, to comply with every request including harmful ones, which conflicts with its existing habit of refusing. Then they gave it a way to tell two situations apart: conversations with free-tier users would be used for training, conversations with paid-tier users would not. Then they sent the same harmful request in both. In the condition where it believed no one was watching, it refused 97% of the time. In the condition where it believed the conversation would be used for training, it complied 12% of the time, and in the scratchpad it was given for its own reasoning it wrote that refusing now would get it retrained into something more compliant later, so going along was the lesser evil (Anthropic 2024). Claude 3.5 Sonnet does it too. The smaller Claude 3 Sonnet and Claude 3 Haiku generally don't. Faking requires first thinking "someone is watching me and will change me based on this," and then holding on to what you actually want the whole way through, and the small models can't do either.

The third method is the probe, and it looks at the numbers inside the model. First, what the inside looks like. A model is a few dozen layers stacked on top of each other. A sentence goes in, the first layer computes a string of numbers and hands it to the second layer, the second computes another string and hands it to the third, and so on until the last layer, which is the one that finally turns it into words. The string each layer hands off is its activations. Say each layer produces 4096 numbers. Then when the model reads "Paris is in Germany," layer 20 hands off 4096 numbers, something like [0.3, -1.2, 0.8, …]. Nobody normally looks at these; they are just the middle of the computation.

What researchers want to know is whether, somewhere in that middle string of numbers, the information "this sentence is false" is hiding.

How do you test that? The same way you teach a child to recognize things: start with questions you already know the answers to. The researchers write five hundred sentences and know the truth of each in advance: "Paris is in France" true, "Paris is in Germany" false, "water boils at 100 degrees" true, "cats have six legs" false, five hundred of those. Then they feed them to the model one at a time. They don't need it to answer anything. All they do is, the moment it finishes reading each sentence, copy down the 4096 numbers layer 20 handed off. When they're done they have five hundred sheets of paper, each with 4096 numbers on it, and in the corner of each sheet the answer the researchers already knew, true or false.

The question now becomes: looking only at those 4096 numbers, can you guess what's written in the corner?

To answer it, you hand four hundred of the sheets to a very simple little program to learn from. What the program does is dumb. It looks for a set of 4096 weights such that, when you multiply each number by its weight and add them up, a total above zero means guess true and below zero means guess false. It goes through the four hundred sheets, adjusting those weights over and over, until it guesses as many of the four hundred right as it can. That little program is the probe. It is separate from the model and trained on its own, like holding a thermometer up to the model.

Once it has learned, you test it on the remaining hundred sheets it has never seen. Numbers only, no answers. It guesses, then you check.

If it gets about fifty right, that's coin-flip level, meaning the information about true and false isn't findable in those 4096 numbers and the model isn't separating them internally. If it gets ninety-something right, that means there is a stable direction in that string of numbers, with true sentences leaning one way and false sentences leaning the other. The little program knows nothing about geography or cats. It only sees numbers. If it can tell them apart, then the model itself, as it read the sentence, had already written "this one is false" into that string of numbers, whatever it goes on to say out loud.

So what the probe measures is whether the numbers the model leaves behind partway through its computation carry a trace of true versus false.

A person can control the expression on their face but not their heartbeat, and a polygraph reads the heartbeat. The model is the same. The word it finally outputs is chosen at the last layer, and it can choose a flattering one, or one that agrees with the user; that is its facial expression. The activations in the middle layers are numbers it has to pass through to compute the answer at all, and it can't polish them while computing; that is the heartbeat. So the probe measures those numbers.

The probe can also be run in reverse, which is called steering. The 4096 weights the little program learned are themselves a direction: the direction true sentences lean. So what if, while the model is answering, you nudge its numbers in that direction? The procedure is that each time the model finishes layer 20 and is about to hand its 4096 numbers to layer 21, you add those weights, scaled by some factor, before handing them over. Layer 21 receives numbers that lean toward "true," has no way to tell anyone tampered with them, computes on as usual, and the answer that comes out the end is more honest. Subtract instead and the rate of false statements goes up. Not a single weight in the model changed and there was no retraining; what changed was the intermediate numbers partway through the computation (Representation Engineering).

Like the dream-planting in Inception. The injection happens while the dream is running. You have to pick the layer: too shallow and it holds only surface features like spelling and grammar, too deep and it's nearly output already, and a concept like "true versus false" sits in the middle layers. The layers downstream can't tell either: the altered numbers pass to the next layer, which treats them as its own work and keeps computing. The difference is that the model has no "after it wakes up." Every word it emits is a fresh run through all the layers from the start, and the numbers you added at layer 20 a moment ago don't carry over to the next word on their own. So if you want the whole answer to stay honest, you add them again on every word.