How Do AI Detectors Work? Signals, Scores, and Limits

An AI detector does not recover a document's history. It estimates whether the text resembles examples the detector associates with human or machine writing. That distinction matters: a score can guide a closer review, but it cannot prove who wrote a passage.

The short answer

Most AI writing detectors are classification systems. They receive a passage, extract statistical or learned signals, and return a label or probability-like score. A detector may compare the passage with patterns learned from collections of human and model-generated text. Other methods examine how probable the passage is under a language model, or how that probability changes when the text is perturbed.

The result depends on the detector's training data, target models, supported languages, minimum text length, and decision threshold. Two detectors can therefore disagree without either one experiencing a software failure. They may be answering slightly different statistical questions.

What signals can a detector use?

1. Token probability and predictability

A language model assigns probabilities to possible next tokens. Some detection methods look for sequences that are unusually probable under a model. This idea is often simplified as "perplexity," but low perplexity is not a signature of AI authorship. Instructions, definitions, formulaic reports, and writing by people who use a limited vocabulary can also be highly predictable.

2. Probability curvature

The research method DetectGPT tests whether a passage sits in a region of a source model's probability function where small rewrites tend to receive lower probability. Its authors reported strong results in specific experiments involving known models and datasets. That is useful research evidence, but it is not a universal authorship test for every model, genre, or edited passage.

3. Learned stylistic patterns

A trained classifier can learn combinations of vocabulary, syntax, repetition, transitions, paragraph shape, and other features that separated human and generated samples in its training set. Commercial systems do not always disclose their full feature set. Broad labels such as "burstiness" are therefore better understood as informal descriptions than as a complete account of how every detector works.

4. Document-level context

Some systems split a document into qualifying text segments and aggregate signals across the document. Length matters because short passages offer less evidence. Lists, quotations, code, tables, and heavily templated language can also behave differently from ordinary prose.

What does an AI score mean?

A displayed percentage is easy to misread. It may represent a calibrated probability, a share of qualifying text classified as likely generated, or a product-specific confidence score. Those are not interchangeable. A "70%" result does not automatically mean there is a 70% chance that a particular person used AI.

Before acting on a score, check the detector's documentation for four details:

• Unit of analysis: Is the score for the whole document, individual sentences, or only qualifying prose?
• Supported inputs: Is the tool validated for this language, genre, and length?
• Threshold: What score changes the label, and what false-positive rate accompanies that threshold?
• Intended use: Is the output designed for triage, editorial review, academic investigation, or another setting?

Why false positives and false negatives happen

Detection is a tradeoff between catching generated text and avoiding false accusations. Raising sensitivity can identify more generated samples while also flagging more human writing. Lowering sensitivity can protect more human writing while allowing more generated samples through.

OpenAI retired its 2023 text classifier because of its low accuracy. In the published evaluation, the classifier identified 26% of AI-written challenge-set text as likely AI-written and incorrectly labeled 9% of human text that way. OpenAI also warned that performance varied with passage length, language, predictability, and distance from the training data. See the archived classifier explanation and limitations.

Model drift creates another problem. A detector trained on yesterday's generators may encounter a newer model, a different sampling method, translated prose, or a mixed human-and-AI draft. Editing can further change the signals. Performance measured on one benchmark should not be assumed to transfer unchanged to every real document.

Fairness deserves separate attention

A peer-reviewed study by Liang and colleagues found that several evaluated GPT detectors frequently misclassified essays written by non-native English writers. The paper links this result to constrained linguistic variation and cautions against uncritical use in evaluative settings. Read the open-access study, GPT detectors are biased against non-native English writers.

That does not mean every detector has the same error pattern today. It does mean organizations should test their chosen system on the languages and populations they serve, document an appeal path, and avoid treating a single score as a verdict. Our guide to detector bias and non-native English writing covers this issue in more detail.

How to review a detector result responsibly

1. Confirm the input is suitable. Check language, word count, excluded material, and whether the pasted text matches the original document.
2. Record the exact result. Keep the detector name, date, score, and version information when available. Scores can change as products update.
3. Look for independent evidence. Draft history, source notes, revision records, citations, and the writer's explanation are more directly connected to authorship than stylistic probability alone.
4. Use a second review only when it adds context. Running many detectors until one supplies a preferred answer is not a reliable method. Compare results deliberately and note disagreement.
5. Let a person make the consequential decision. Turnitin's own current guidance states that its model may misidentify human, AI-generated, and AI-paraphrased text and should not be the sole basis for adverse action. See Using the AI Writing Report.

What current evaluations tell us

NIST evaluates generators and detectors under controlled conditions rather than assuming one accuracy number applies everywhere. Its 2024 GenAI text-to-text pilot report uses measures such as area under the ROC curve and Brier scores to examine both discrimination and calibration. The broader lesson is practical: detector quality is a measured property of a system on a defined dataset, not a permanent badge attached to a product.

We saw that instability directly in our 2026 AI humanizer benchmark. One unchanged historical control moved from 28% to 73% on the same detector service, while another moved from 19% to 3%. The benchmark publishes the frozen-holdout results, quality gate, controls, and limitations.

A useful way to think about the output

Treat an AI detector like a smoke alarm, not a security camera. It can surface a pattern worth checking. It cannot show you the drafting process, identify the author, or explain every reason the pattern appeared. The higher the stakes, the more important it is to combine the signal with provenance, context, and human review.

You can use the AI Undetectable text detector for an initial likelihood signal, then use the humanizer workspace to revise structure, clarity, and reader fit. In both cases, the final judgment belongs to the person reviewing the work.