The short answer
Detecting AI text is not a matter of looking at a sentence and declaring "a machine wrote this." None of the methods that work produce certainty; they all produce probability. Which is why "real or fake" is not even an answerable question in practice — the more useful question is how much of a text came from AI.
Max Spero, co-founder and CEO of the detection startup Pangram, makes exactly this point on TechCrunch's Equity podcast: working out how much AI went into something is both harder and more useful than labelling it AI or human. The company recently raised $9 million for its detection system and struck a partnership with Substack, which now shows readers which of the authors they follow use AI to write their newsletters.
There are really three separate questions
Much of the confusion comes from treating three different questions as one:
- Did a model produce this text? A binary classification. The most-asked question and the least informative.
- How much of the text came from a model? A writer who had a model edit a draft is not in the same position as someone who pasted a prompt and published the output unchanged. The question of degree tells you more than a binary label.
- Which model produced it? This is what actually matters for copyright, contracts and compliance reviews — and it is the hardest to answer.
An organisation cannot choose a method without knowing which question it is asking. The families of methods below answer these three to different degrees, and none of them serves all three well. For a school the second question usually decides things, for a publisher the first, for a legal team the third — and that choice also determines up front which kind of error you can live with.
Detection methods: four families
| Method | What it looks at | Weak point |
|---|---|---|
| Statistical signature | How "expected" the text is according to a language model; the distribution and uniformity of word choices | Collapses on edited, short or translated text |
| Trained classifier | A probability from a separate model trained on human and machine text | Can be blind to output from a new model absent from its training data |
| Watermark | An invisible statistical trace deliberately embedded at generation time | Works only for the provider that adds it; can be stripped |
| Provenance metadata | Generation information attached to the file — who, with what tool, when | Lost on copy-paste; barely survives in plain text at all |
The first two look at the text itself and can work on anything; the last two require cooperation from whoever produced it. That distinction is decisive in practice: a university or an employer does not get to choose which model a student uses, so any policy resting on watermarks is incomplete from the start.
Why watermarking is not enough on its own
The idea is sound: while generating output, the model alters the source of randomness in its word choices so the result can later be recognised statistically. Anthropic described this method for Claude's text in detail and opened a detection interface for it. But it has three limits.
The first is coverage: only the model of the provider that adds the watermark can be recognised. Someone running an open-weight model on their own hardware never adds one. The second is durability: code that removed Claude's invisible mark was published within four hours. The third showed up on the image side — Google made the visible watermark on generated images removable, keeping it mandatory only in countries where the law requires it.
Watermarks are still not worthless. Where a provider needs to recognise its own product — abuse reviews, corporate compliance, breach investigations — they work. They just cannot carry the claim of labelling every piece of text on the internet.
Identifying which model wrote it
This was the hardest of the three questions: naming the model a text came from. Statistical signature methods partly help here, because every model has its own tendencies in word choice — the same prompt has a different "flavour" from each. But those tendencies shift with every version; a distinction that worked a week ago may not hold after a new release.
Watermarking gives the clearest answer to this question, but only for the provider that applies it. So the answer to "which model" splits in two in practice: near-certain for watermarking providers, guesswork for everything else. That asymmetry grows as open-weight models spread.
An extra difficulty in Turkish
Most detection tools are trained predominantly on English text, and their accuracy figures are usually reported on English too. For Turkish that creates a two-sided problem: there is less training data, and the structure of the language makes statistical signatures harder to read. In an agglutinative language the same root takes dozens of forms through suffixes, so a method measuring predictability at the word level finds a far less clean signal than it does in English.
The practical consequence: carrying an accuracy figure reported for English straight over to Turkish text is wrong. An organisation deploying a detector on Turkish content needs to ask for that tool's Turkish performance separately — and most providers do not publish that number.
The real issue: the cost of a false positive
A detector's accuracy rate means little on its own; what matters is the direction in which it errs. A false negative — missing AI text — is tolerable in most settings. A false positive means accusing a person of cheating.
That asymmetry produces its heaviest consequences exactly where detection tools are used most: in education. A student writing in a second language, working with a limited vocabulary and flat sentence structures, looks under statistical-signature methods very much like generated text — because what both share is predictability. The same problem applies to technical writers who deliberately keep their style plain.
Another point Spero underlines is the cost of false positives on sensitive images. For a piece of text, an "AI" label creates a reputational problem; for a personal image, the same label turns into a far heavier accusation.
Why doesn't this get easier?
There is a structural asymmetry between generation and detection. With every release the generation side tries to produce less predictable, more varied, more "human" text — which is to say it deliberately weakens the signal detection depends on. The detection side has to recalibrate for every new model. This is a race in which the defender is always a step behind: when a new model ships, the detector does not recognise it, and recognising it first requires collecting enough samples of what that model produces.
Four practical difficulties sit on top of that:
- Short text: The statistical signal strengthens with length. A comment or a product review often does not carry enough data to decide on.
- Rewriting: Handing one model's output to another with "put this in your own words" largely erases the signature.
- Mixed authorship: This is the most common usage pattern today, and where the draft is human and the editing is machine, a binary label is meaningless to begin with.
- Targeted evasion: Someone who knows how a detector works can tune the output to pass it.
What to do in practice
Using a detector as a decision mechanism on its own loads it with weight it cannot carry. A more defensible arrangement looks like this:
- Treat the tool's output as a signal, not as evidence; it should never be grounds for sanction by itself.
- Set the threshold by use case: a publisher's content pipeline and a university's disciplinary process cannot absorb the same margin of error.
- Ask for a proportion rather than a binary label; "what percentage" is both more honest and more useful than "yes or no."
- Build the process on disclosure: an arrangement where use is declared openly is cheaper and fairer than trying to detect it.
- Leave a route to appeal a false positive; a detection policy without one will eventually punish someone unjustly.
How close is the "dead internet"?
Spero's broader claim is that the internet is dangerously close to the dead internet theory becoming reality within a few years. What he means is not only feeds filling with AI content, but AI-generated text and images making their way into job applications, product reviews and even insurance claims. What the internet then needs is a "trust layer" — the gap startups like Pangram are trying to fill.
His prediction about the labour side runs in two directions: he thinks the "bottom tier" of writing jobs is gone for good, while genuinely good human writing may become more valuable. That second half shows why the detection debate is not purely a technical matter: the question of who wrote a text is inseparable from the question of how much that text is worth.
In summary
Detection tools work, but they do not deliver certainty; what they deliver is probability, and how that probability gets used is entirely the organisation's choice. The most defensible position today is to treat the tool not as an evidence machine but as a signal feeding a human decision — and to put the possibility of a false positive at the centre of the policy.
Over the longer term the debate will probably shift from detection to disclosure. Measuring from the outside whether a text was machine-written gets harder with each release, while building an arrangement where use is declared up front is technically far cheaper. Substack showing readers an author's AI use is a move in exactly that direction: it turns the problem from one of detection into one of transparency.