AI-text detectors have a bad reputation, mostly earned. Plenty of them are black boxes that flag a nervous student's essay as a robot and offer no way to check the claim. This page is the opposite: the corpus, the calibration, the operating point, and the parts it gets wrong, with the false-positive number that matters put first.
The detector reads prose and scores how much it looks like current-model AI writing, from a set of measurable features: the density of the words and stock phrases these models overuse, sentence-length uniformity, connective-opener overuse, chat-UI artifacts left in a paste, and a few dozen more. Those features feed a calibrated model that returns a probability. It is not a vibe and it is not a language model judging another language model; it is a scorer whose every input you can inspect.
Every number on this page is self-reported: we ran the eval, we picked the sets, nobody audited it. Saying that plainly is the point. For a tool that flags contributions, the number that matters is how often it is wrong about a human, and the hardest version of that is careful non-native English, which sits in the same formal register the models write in. On a held-out set of 4,300 non-native-English essays, real writing from real ESL students, the false-positive rate is 7.8% at the shipped operating point. Not zero. That is the honest ceiling on trusting any single flag, and it is why the extension never calls a flag proof.
Measured on data the calibration never saw, split by how current the generator is. These are the detector's real reported numbers, not a small in-house sample:
| Held-out set | What it is | AUROC | Catch rate at 5% FPR |
|---|---|---|---|
| DetectRL-X | 2025-era generators (DeepSeek-V3 and similar), plus paraphrase and zero-width-space attacks | 0.983 | 92% |
| RAID-test | 2022-23 generators (GPT-2/3, Cohere, MPT, Mistral) | 0.89 | 67% |
| MAGE (paraphrase) | GPT text run through a paraphraser, out of domain | 0.81 | 39% |
| humanizer-attack | Ghostbuster "undetectable" essays | - | 99% caught |
For comparison, GPTZero's public RAID submission is 0.9724 catch rate at 5% false positives: ahead of this detector on RAID's older models, behind it on the ESL false-positive axis, where that class of detector runs around 60%. The RAID gap on old base-model text is real and named, not hidden.
A separate 40-document labeled set (19 human, 21 AI) rides along in the repo, stacked with the writing most likely to trip a detector: formal 19th-century essayists (Twain, Thoreau, James, Cather), journalism, kernel commit messages, a translated legal code. It is not a benchmark and is not quoted as one; it is the calibration guard that catches a change that would start flagging Mark Twain as a robot before it ships.
The detector is tuned to a 5% false-positive operating point on a broad mix of English text: it would rather miss a bot than accuse a person. That is the target the threshold is set from, not a promise that every group lands on it. Non-native English comes in above it, at 7.8%, which is the whole point of putting that number first. Below roughly twenty words, or on text it reads as anything other than English, it does not guess at all. It says it abstained and shows you why. A one-line "lgtm" gets no verdict, because no honest detector can give one.
Two failure modes, named rather than buried. First: text a person drafted and then had a model polish for tone or grammar. The words on the page really were shaped by the model at that point, and a style detector built to catch AI phrasing has no principled way to tell "polished" from "written," because in the features it measures they can look the same. Second, and the one to take most seriously: non-native and heavily formal English. The classifier abstains outright on anything it doesn't read as English, but a non-native speaker writing carefully in English, or anyone writing in a dense, formal register, can still land close to the same stylistic territory as current AI prose. The 7.8% false-positive rate on 4,300 real ESL essays above is exactly this failure mode measured at scale rather than papered over, and it is not zero. It is a claim that some non-native and formal writers will be flagged, which is why a flag is a signal to look, never proof. Here is a concrete example I will not hide: text written densely about the vocabulary these models overuse trips the style signals, so some of my own writing about this detector scores as AI-ish. That is the failure mode, shown rather than papered over. It is exactly why every result in the extension is labeled a signal at a stated false-positive rate, never proof, and why the advice is always the same: judge the contribution, not the author. The detector points you at text worth a second look. You do the looking.
The held-out generator sets above (DetectRL-X, RAID-test, MAGE, the humanizer-attack set, and the 4,300-essay ESL gate) are licensed for evaluation use and are not redistributed, so this page cannot hand you a one-line command to reproduce them the way an open dataset would. What is committed to: they are the same frozen sets on every release, the fitting never touches them, and the numbers are the direct output of the eval against the shipped model. The 40-document calibration set that guards against regressions does ship in the repo and runs with one command:
python3 eval/run_eval.py
That prints the per-sample table for the small guard set. It is the linter's tripwire, not the detector benchmark, and it is labeled that way here so the two are never confused.