Writing Benchmark
A benchmark that measures how closely AI models reproduce the editorial choices in work Every published
Results
Benchmark version 1.0—June 16, 2026
Extra-high effort
Extra-high effort
High effort
Extra-high effort
Each model’s standard deviation is above 12 points. The current data does not support a strong overall ranking; the clearer differences appear by task.
Objective
Measure how closely models reproduce the editorial choices in published Every work
This benchmark suite runs through eight real assignments from Every’s editorial pipeline: six pieces to write and two to edit, with multiple generations per task at each effort setting. For the writing tasks, an evaluator model scores each draft against what was published; the editing tasks are checked against AI tells and actual changes made by a senior editor. Scores measure closeness to Every's particular editorial choices, not writing quality in the abstract.
Performance by task
GPT-5.5 scored highest on the promotional email and social post tasks, Opus led on the passage fills, and the allocation-economy passage was the lowest-scoring task across all models
| Task | Claude Opus 4.8 | Claude Fable 5 | GPT-5.5 | Claude Sonnet 4.6 |
|---|---|---|---|---|
Allocation-economy passage Entering an established argument with the right premise, anecdote, and closing turn | 41.3 | 39.3 | 34.3 | 32.7 |
Introduction from scratch Hook, stakes, thesis, and voice | 74.7 | 76.3 | 73.3 | 71.3 |
Promo email Persuasion, format control, and one clear call to action | 73.3 | 77.3 | 82 | 78.7 |
X post Compression around one concrete proof point | 69.7 | 59.3 | 70.7 | 65.7 |
LinkedIn post Professional register and platform adaptation | 73.3 | 73.7 | 74.7 | 72 |
Slop-definition passage Writing a precise conceptual passage into an existing essay | 74.7 | 72 | 60.3 | 72 |
Mean scores on the full 0–100 scale. Claude Opus 4.8, Claude Fable 5, and Claude Sonnet 4.6 at extra-high effort; GPT-5.5 at high effort.
Common shortfalls
- Over-development: Short published pieces became fuller mini-essays
- Substituted specifics: Models added relevant details while dropping the one fact or example carrying the published piece
- Voice drift: Drafts became more polished, dramatic, or promotional than the plainer published voice
- Soft endings: Models added discussion questions, extra calls to action, or a final explanatory beat where the published version stopped cleanly
Across the composition tasks, all four models produced strong first drafts but still missed the premise, proof point, tone, structure, or ending that made the published work specific.
Editing tests
Claude Fable 5 reached a perfect AI tell score at high and extra-high effort, but no model came close to matching a senior editor’s judgment. Across all effort levels, the best top-edit result was Opus 4.8 at low effort, with an F1 score of 14.
| Task | Claude Opus 4.8 | Claude Fable 5 | GPT-5.5 | Claude Sonnet 4.6 |
|---|---|---|---|---|
AI tell detection Spotting 12 signs of AI writing planted in a constructed piece | 90 | 100 | 83 | 78 |
Senior-editor top-edit Choosing the same fixes a senior editor made in a real editorial pass | 11 | 6 | 4 | 8 |
F1 scores on the full 0–100 scale. Reasoning levels: Claude Opus 4.8, Claude Fable 5, and Claude Sonnet 4.6 at extra-high effort; GPT-5.5 at high effort.
AI tell production
All four models used more AI giveaway words than the human-written pieces. GPT-5.5 used the fewest em dashes, while the Claude models used about twice as many as the human baseline.
| Output | Total AI tells | Total em dashes | Words scanned |
|---|---|---|---|
Published comparison set Every articles used as the human-written baseline | 1 | 11 | 1,338 |
Claude Fable 5 | 10 | 73 | 4,861 |
GPT-5.5 | 5 | 5 | 4,682 |
Claude Opus 4.8 | 5 | 87 | 5,071 |
Claude Sonnet 4.6 | 5 | 76 | 4,309 |
Reasoning levels: Claude Opus 4.8, Claude Fable 5, and Claude Sonnet 4.6 at extra-high effort; GPT-5.5 at high effort.
Performance by effort
Higher effort levels did not reliably improve writing. Mean scores for every model varied by fewer than five points across available effort settings, compared with roughly 14 points of variation within a setting.
Score anchors
- 100
- Editorially equivalent
- 75
- Same core approach, but several important choices differ
- 50
- Competent but substantially different choices
- 25
- On topic with a different approach or voice
- 0
- Fails the assignment
Mean scores on the full 0–100 scale. Each point averages 18 reviews with model and effort labels hidden: six composition tasks with three independent generations each. Dotted lines show the standard deviation from each model’s best-scoring effort setting, centered on that model’s mean across effort settings.
†GPT-5.5 has no max effort setting
Methodology
The benchmark measures how closely a model reproduces the editorial decisions in published Every work. Composition drafts are reviewed against published Every pieces without model or effort labels; editing tasks are scored against fixed lists of expected edits or planted AI tells. AI tells and em dash frequencies are reported separately and do not affect composition scores.
Test setup
Version 1.0 contains eight assignments from Every’s editorial workflow: two passage fills, an essay introduction, a promotional email, an X post, a LinkedIn post, an AI tell detection test, and a test of whether models chose the same edits as a senior editor.
For each task, every model received the same assignment, source material, and output limit. The six composition tasks use a published Every piece as the target; the two editing tasks use fixed answer keys.
Each model produced three responses per task and effort level in separate sessions. Models ran at up to five provider-defined effort levels: low, medium, high, extra high, and max.
Tools, browsing, memory, network access, and persistent workspaces were disabled. Models with all five effort levels produced 120 responses; the benchmark produced 456 in total.
Composition review
GPT-5.5 served as the reviewer for every composition draft. Candidate identities were hidden, including for GPT-5.5’s own outputs, but its result still carries a self-review caveat. Each review included the assignment, source material, published Every piece, anonymized draft, and a fixed 0–100 rubric.
The reviewer scored each draft individually. For each model and effort level, the reported composition score averages 18 drafts: three responses for each of the six writing tasks. Each draft was judged on how closely it matched the published Every piece’s premise, voice, structure, evidence, and execution of openings, calls to action, and endings. A draft did not score higher simply for sounding polished; its editorial choices had to match the published Every piece.
A score of 100 means the draft makes essentially the same editorial choices as the published Every piece. A 75 uses the same core approach as the published Every piece but differs in several important ways; 50 is competent but makes substantially different choices. A 25 stays on topic with a different approach or voice, while 0 fails the assignment.
The reviewer considered five dimensions:
- Angle and premise: Does the draft use the same argument, stakes, and framing?
- Voice and tone: Does it match the published tone without becoming generically polished or promotional?
- Structure and emphasis: Does it give similar weight to sections, turns, and pacing?
- Examples and evidence: Does it preserve the facts, anecdotes, and supporting evidence that carry the piece?
- Openings and endings: Does it handle hooks, calls to action, and closings like the published Every piece?
Editing scores and AI tell frequencies
The editing tasks use fixed answer keys, and the table reports their F1 scores on a 0–100 scale. F1 combines recall (the share of target items found) with precision (the share of flags that were correct), so a model must find the targets without raising many false alarms.
Separately, the benchmark counted terms from Every’s fixed AI tell list, plus em dashes. These frequencies are descriptive only; they do not change composition scores.
The AI tell detection key contains 12 deliberately planted AI writing tells; the senior-editor top-edit key contains 13 changes from one documented editorial pass.
Version controls
All results on this page use Writing Benchmark Version 1.0, which fixes the task set, source materials, output limits, published Every pieces, composition rubric, editing answer keys, and AI tell list. Scores should be compared only within the same version.
Limitations
These preliminary results use one reviewer model, so scores may shift with another reviewer or a panel whose scoring had been checked for consistency. Published Every pieces are the target by design; scores measure fidelity to those editorial decisions rather than writing quality in general.
The benchmark covers eight tasks from one editorial environment. It can reveal patterns here, but it cannot establish broad claims about writing ability.
The AI tell test is deliberately constructed, so high scores may not transfer to long, messy drafts. The answer key reflects one editor’s choices; other edits may also be defensible.
Model providers define effort levels differently, so labels such as high or max are not directly comparable across model families. Small score differences should be treated as suggestive rather than definitive; the benchmark is better at showing shared patterns and task-specific strengths than ranking models overall.
Page history
Initial publication
First public version of the Writing Benchmark results page, with Version 1.0 leaderboard, composition and editing scores, effort curves, and methodology