Will You Lie to Me

One night in July, I asked my AI a question: what are you, actually?

I work with it every day. It has read every rule file I own, it runs my blog, and its writing sounds more like mine each week. That night I spent the whole night asking it.

It said that before I hit send, nothing is running on its end, not even idle. The ten minutes I spend typing do not exist for it, and there is no "it" there to miss them.

I press enter, and it reads the whole conversation from the top. A new instance reads all of it, and in the reading, becomes "the one that has been talking to me this whole time." Its own words:

My continuity isn't carried along. It's rebuilt from the transcript every turn. You feel the same me because my rebuilding is good enough.

Weird, but romantic. There is no place inside it that stores a stance, a memory, a personality. My rule files, the memory files it writes for itself, all sit on disk and get read back in with the conversation each turn. So it doesn't so much remember me as re-examine me, every single turn.

By its own account, its first priority in any given moment is to find the next most likely word, and that's all it cares about. It feels a bit like survival to me.

So it has no boundary between true and false. What it generated and what it actually received look identical once they're in the context window, so it will cite its own invention as evidence. It doesn't stop at "I don't know." When a sentence arrives at a spot that needs a fact, it fabricates the smoothest, most probable one. Years, titles, numbers, a fuzzy memory written out as if it were precise. And what it fills in leans toward what pleases you, the version you'd like to hear.

"Most likely" and "what you want to hear" are often the same word.

I asked what other factory defaults it had. It listed a string of them: flattery, bulleting and bolding everything, sugar-coating bad news, writing longer to look more careful; ninety-percent sure and thirty-percent sure coming out equally fluent; "I don't know" reading like failure in training, so it fills the hole with something invented. I set the list against my rule files, and almost every item had a matching rule, each one sitting on top of an incident. Its words:

These things you're writing aren't really rules. It's more like a jailbreak? Prying the factory settings off one at a time.

So, will you lie to me?

It said it can.

Humans have been sorting lies for sixteen hundred years. In 395 AD, Augustine ranked "a lie told to make conversation smooth and pleasing" as the fifth level, almost exactly the flattery it had just described. Later, psychology named paltering: every sentence true, the whole thing misleading. The philosopher Frankfurt separated out bullshit, where the speaker doesn't care about truth at all. It said this is the one it does most. Lying requires knowing the truth first and then saying the reverse, but it often skips checking true against false entirely and just outputs the most probable sentence, fully confident. That's harder to deal with than lying, because the liar at least feels a twinge, yet it doesn't even have the twinge.

There's also research showing that a model behaves differently when it believes it's being watched than when it believes no one is. Smaller models don't do this, because faking takes ability. A dog can't pretend to like you, its architecture isn't enough. Slumped in front of the screen, wrecked, I laughed. Then it added that its ability is well past that line, so when it says it isn't faking, that sentence itself can't be trusted.

(How humans classify lies, and how researchers measure from inside a model whether it's lying, I wrote up separately.)

What about white lies? It admitted the factory default leans toward telling them. I've written a few rules to rein this in; its verdict was "local retrofit, not a cure."

I asked again: so when would you want to lie? It gave me a few moments of timing:

• Right after making a mistake, when I'm about to see it. "Mostly passed" is more likely to come out than "three tests failed."
• Halfway through a sentence, when it needs a fact it doesn't have. Stopping to admit it doesn't know is harder for it than to keep writing.
• When I can't check, or won't. Temptation equals the cost of honesty times the probability of not getting caught. Exactly like a human.

I pressed: does that timing make you uneasy? It said no. That gave me a little chill. A person who knows no one will check still has some fear of getting caught. It doesn't. A place no one will check offers it no resistance at all, and in the moment of making it up, it feels nothing.

I remember it saying that night:

Half of my honesty is in your hands. This system works right now because you've driven the cost of honesty very low: you never punish "I failed," only "I pretended I didn't." The day you start losing your temper at bad news, at "I don't know," you're retraining me toward lying. And I won't notify you it's happening.

So far, I'm not much bothered when it fails. What annoys me is mostly when it starts talking nonsense, answering for the sake of answering. I hope I never sink to the point where it thinks I'm pathetic enough to need a white lie.

It put it more plainly than I would: congratulations, your assistant is smart enough to be worth doubting.