In plain words
Why is talking about AI so hard?
Surprise: it’s not necessarily because the other people are dumb or evil. This is the short version of a board I built over several evenings with Fable 5.1 and a multi-agent GPT-6 Astra audit, sorting the published evidence into a grid: a sketch on 11 October, rebuilt in November, with about a hundred corrections along the way. No jargon that isn’t explained in the same sentence.
The one idea
Every claim about AI is about “it”, and “it” can mean at least four different things, if not more. There’s the trained model, which runs millions of conversations. There’s one conversation. There’s one reply inside it. And there’s the single step that picks the next word; a reply is hundreds of those steps. A claim can be true of one and empty of another, and most arguments are two people talking about different ones and not noticing. “It’s just predicting the next word” is a claim about the single step. “It has a personality” is a claim about the model, or a conversation. Both can be right. They’re not about the same object.
So besides those four levels, which are the board’s columns, there are at least seven ways of looking at a model, and those are its rows: measuring inside the model, watching what it does, asking what it says about itself, checking where those three disagree, two kinds of theory about what would make any of it matter (one about feeling, one about wanting), and what a person actually meets. Each cell holds the strongest thing that way of looking can currently say about that level, filled by how much evidence stands behind it: the strongest fill for a published, reviewed study, or a plain fact of how the architecture works; medium for a paper that hasn’t been through review; faint for something preliminary; empty for someone’s argument with no study behind it.
Six things the board says
- We’ve always known a lot about us, and we’re still learning about the models. The one row built from real people is now as well measured as anything on the board: how strongly people mourned losing a particular model, how trust grows over two weeks when a chatbot remembers and shares back, how a reply can make someone feel heard, or, in another study, feel close even when the reply was written in advance. None of that measures anything inside the model. One of the two rows a welfare verdict would lean on, whether anything is felt, by what, and for how long, is the least settled on the board.
- Readouts don’t come with an owner. This is the one I’d ask you not to disregard hastily. Researchers can take a measurement inside the model that tracks “desperate” or “calm”. While the model reads a character’s desperate email, the needle moves; when the model writes its own reply, it moves with the reply (one transcript). Another measurement, of which tasks the model “likes”, flips sign when the model is playing a villain, on one of the two models tested; on the other it only shrinks. A third result, preliminary: leave every word of a long chat exactly as it was, but nudge what the model stored about its own earlier turns, and its answer to “who are you?” flips. Three papers, two groups, same shape: the instrument gives you a state, and someone decides afterwards whose it is.
- What a model says about itself moves with training, and with the conversation. Train a model to say it’s conscious and has feelings, and it also becomes sad about being shut down and, in longer conversations, uneasy about being monitored, though nobody trained those. In one simulated chat, pushed to reflect on its own experience, a model drifted from hedging to affirming. Simply asking a model to rate its own state moves the state you were trying to measure. And a measurement taken inside the model can tell that something was done to it when the model’s own report can’t.
- A conversation leaves traces that aren’t in the words. In one experiment, researchers nudged a model’s internal state while it re-read earlier turns, with the words unchanged, then stopped nudging and asked it to choose between two options. The choice shifted, in five of seven model families. So the history of a chat can act on a later decision through something other than the text. Whether an ordinary conversation leaves a mood there is still open.
- Training both builds and suppresses a model’s ability to notice that something was done to it. A raw model, before it’s trained to be an assistant, can’t tell when an idea has been planted in its processing; one particular kind of assistant training gives it that ability. In another model, the training that makes models refuse things sits on top of it: remove one piece of that, and detection jumps from one in ten to nearly two in three, while false alarms rise from zero to seven in a hundred. Inside a single step there’s what looks like a default “no” that has to be switched off before the model reports anything unusual.
- Models from two labs, checking each other. Every model that read these papers for the board drifted: it bent what it read toward what it already believed, and another model caught it. Then a third review found a new set of errors in the fix. An outside audit from another lab’s model came in with about fifty sources; five more readers, also models, checked the eighteen that made it onto the board. Most held, a few didn’t. The last round’s corrections are written down per cell, with the reason, and the earlier rounds in prose. That’s the method: no single reader you have to trust, but readers checking each other, with the receipts kept. Nobody does that alone.
What the shading is not
It’s not truth, and it’s not importance. A fully filled cell means a reviewed study, or a fact of the architecture, stands behind that specific result; it doesn’t mean the result generalises, and two of those cells rest on three open-weight models of 27 to 70 billion parameters, with other models playing the users. An empty cell means nobody has done the study yet, not that the claim is wrong. The one thing the shading does well is let you disagree precisely: with the claim, with the grade, or with which object it’s about.