AI in content creation: 37% faster drafts, 68% shipped unedited

AI in content creation does one thing well and one thing badly, and the research separates them cleanly. In a preregistered MIT experiment with 444 college-educated professionals, access to ChatGPT cut the time spent on a writing task by 10 minutes, a 37% drop against a control group that averaged 27 minutes. Blinded graders scored the assisted work 0.45 standard deviations higher. So the drafting gain is real, and it’s larger than most vendor claims.

What doesn’t travel is everything after the draft. In that same experiment, 68% of treated participants reported submitting ChatGPT’s first output without editing it, and their submissions didn’t score higher than raw model output the graders also saw. Meanwhile an audit of 111 million references across 2.5 million papers found a conservative 146,932 citations in 2025 pointing at work that doesn’t exist. That’s the shape of the job: the model is fast at sentences and indifferent to whether they’re true, and the person who closes that gap is an editor.

What the trial measured, and what it left out

The speed number comes from Shakked Noy and Whitney Zhang’s experiment, circulated as an MIT working paper in March 2023 and preregistered at the AEA RCT Registry. They recruited 444 experienced, college-educated professionals: marketers, grant writers, consultants, data analysts, HR professionals and managers. Each wrote two occupation-specific pieces of 20 to 30 minutes, things like press releases, short reports, analysis plans and delicate emails.

Half were walked through signing up for ChatGPT between the two tasks, so performance on the first task doubled as a baseline. The control group was pointed at the LaTeX editor Overleaf instead, because the design needed a placebo signup that didn’t write anything.

Pay was tied to quality. Participants earned $1 for each point on a 1 to 7 grade, with a bonus arm paying $3 extra for a 6 or a 7. Blinded professionals in the same occupations graded every submission, three evaluators per piece, with an average within-essay cross-evaluator correlation of 0.44.

Of the treated group, 92% managed to sign up and 81% chose to use the tool on the second task, rating it 4.4 out of 5 for usefulness. Mean grades went from 3.79 in the control group to 4.54 in the treatment group on the second task, so the gap shows up in raw scores and not just in standardised units.

That matters because of what the rubric covered. Evaluators scored writing quality, content quality and originality. Nothing in the design asked whether a figure in a press release matched a source, because the tasks carried no external claims to check.

So the experiment is strong evidence about drafting speed and perceived quality, and silent on accuracy. It’s also a working paper rather than a peer-reviewed article, which the authors state on the title page, so the figures carry the usual preprint caveat.

Study	Sample	Task	Headline result
Noy and Zhang, MIT working paper, March 2023	444 professionals	Occupation-specific office writing, 20 to 30 minutes	Time down 10 minutes (37%), grades up 0.45 SD
Doshi and Hauser, arXiv preprint v3, March 2024	293 writers, 600 evaluators	Eight-sentence short story, optional GPT-4 story ideas	Novelty up 6.7%, stories more similar to each other
Zhao and co-authors, arXiv preprint, May 2026	111m references in 2.5m papers	Audit of citation existence across four preprint and journal corpora	146,932 non-existent citations estimated for 2025

The drafting step shrinks and the editing step grows

Before the treatment, participants in that experiment split their time roughly 25% brainstorming, 50% writing a rough draft and 25% editing. Afterwards, the share spent on rough-drafting fell by more than half and the share spent editing more than doubled. The work didn’t disappear, because the total still had to be produced. It moved along the pipeline, away from making sentences and towards judging them, which is the one result in the paper that reads like a job description for an editor.

Here’s the objection, and it’s the strongest one against this whole article. The same authors looked for evidence that human editing improved the machine’s output, and didn’t find it. Treated participants were active for only about three minutes after they first pasted in a large block of text.

There was no correlation between that time and the grade, though the bonus arm gave them a reason to keep working. And treated submissions scored no better than raw ChatGPT output handed straight to the evaluators.

We observe neither of these pieces of evidence, suggesting that ChatGPT is increasing productivity primarily by substituting for worker effort.

Shakked Noy and Whitney Zhang, MIT working paper, March 2023

Take that seriously, then note its boundary. Those were short internal documents with no citations, no named sources and no numbers that could be wrong. The graders were told to treat each piece as work they’d encounter in their own job, and the rubric rewarded prose and ideas.

So an editor adds nothing measurable when there’s nothing to verify, which is true but narrow. The question is what happens when there is.

Everyone who prompts the same model gets handed the same idea

Anil Doshi and Oliver Hauser ran a different experiment, posted to arXiv in December 2023 and revised in March 2024, with the Science Advances DOI listed on the same page. They asked 293 writers for an eight-sentence story. One group worked alone, one could request a single three-sentence story idea from GPT-4, and one could request up to five.

A separate 600 evaluators then rated the results, but weren’t told who had help. Across both assisted conditions, 88.4% of writers called on the model at least once.

Individually it worked. Evaluators rated assisted stories 6.7% higher on novelty and 6.4% higher on usefulness, and the five-idea condition did best at 8.1% and 9.0%. Still, the gain concentrated among writers who scored low on a divergent association test taken beforehand: for them, five ideas lifted novelty by 10.7% and ratings of how well the story was written by up to 26.6%. The authors write that the five-idea condition “effectively equalizes the creativity scores across less and more creative writers”.

Collectively it narrowed. Using embedding similarity, assisted stories sat 5.2% and 5.0% closer to the model’s ideas than unassisted ones, and the 4.29-point shift from a single idea covers 24.3% of the entire range the human-only group spanned. Assisted stories were also measurably closer to each other.

Evaluators noticed something too: once told a writer had been offered one model idea, they assigned that author 25.4% less ownership of the story. So the trade is a better median piece bought with a narrower spread, which is a problem if your competitor is prompting the same model about the same keyword on the same morning.

The fabricated citation is now an editing task

The clearest measurement of what slips through sits in an audit posted to arXiv in May 2026. Zhenyue Zhao and five co-authors checked 111 million references across 2.5 million papers in arXiv, bioRxiv, SSRN and PubMed Central, and found a sharp rise in references to papers that don’t exist, with a conservative estimate of 146,932 hallucinated citations in 2025 alone.

The errors cluster in fields with fast AI uptake, in manuscripts carrying linguistic signatures of AI-assisted writing, and among small and early-career author teams. Their conclusion is that preprint moderation and journal publication capture only a fraction of these errors. It’s a preprint itself, so treat the figure as the authors’ estimate rather than a settled count.

Retrieval doesn’t fix it either. The Tow Center at Columbia tested eight search-connected chatbots across 1,600 queries, 20 publishers by 10 articles by 8 tools, asking each to name the headline, publisher, date and URL behind an excerpt. Klaudia Jaźwińska and Aisvarya Chandrasekar reported in March 2025 that the tools collectively answered more than 60% of queries incorrectly, from 37% for Perplexity to 94% for Grok 3.

More than half of the responses from Gemini and Grok 3 cited fabricated or broken URLs. ChatGPT misidentified 134 articles while signalling uncertainty just 15 times out of 200 responses.

Which means a link in an AI draft is a claim waiting to be checked, not a check that’s already happened. That’s the part that doesn’t automate, and it’s why we price AI writing products by what an editor has to do afterwards rather than by word allowance, as we did when we ran our house checks over four vendors’ own blog posts. Detection tools don’t rescue you here either, for reasons we went through in why most AI text detectors fail: they guess at provenance, not accuracy.

Google scores the output and ignores the method

Search guidance on this is unusually direct. Google’s guidance about AI-generated content, posted on 8 February 2023 by Danny Sullivan and Chris Nelson, sets out the whole position: focus on quality rather than how content is produced, and treat automation used mainly to manipulate rankings as a spam violation. The FAQ at the foot of that post is blunter than most vendor marketing admits.

Using AI doesn’t give content any special gains. It’s just content.

Google Search Central, guidance about AI-generated content

The enforcement side has a name. Google’s spam policies, last updated 28 August 2026, define scaled content abuse as generating many pages primarily to manipulate rankings rather than help users, “no matter how it’s created”, and list “using generative AI tools or other similar tools to generate many pages without adding value for users” as the first example. The same page flags stitching content from different pages together without adding value. So the hinge in both documents is volume without added value, not machine authorship.

That post also answers the disclosure question most publishers ask second. Google’s position is that AI or automation disclosures are useful where someone might reasonably wonder how a piece was created, that bylines should be accurate where readers would expect them, and that giving a model an author byline is probably not the best way to make the process clear.

Nobody owns a draft the model wrote alone

There’s a second reason to keep a person in the loop, and it has nothing to do with quality. In January 2025 the US Copyright Office published Part 2 of its report on copyright and artificial intelligence, covering copyrightability. It drew on more than 10,000 comments filed in response to the Office’s August 2023 Notice of Inquiry, about half of which addressed copyrightability. Two of the report’s conclusions carry the whole question.

Copyright does not extend to purely AI-generated material, or material where there is insufficient human control over the expressive elements. Based on the functioning of current generally available technology, prompts do not alone provide sufficient control.

US Copyright Office, Copyright and Artificial Intelligence, Part 2: Copyrightability, January 2025

The Office also concluded that using AI tools to assist rather than stand in for human creativity doesn’t affect protection for the output, and that human authors keep copyright in their own expression where it’s perceptible in a model’s output, along with the creative selection, coordination or arrangement of material. Each case still gets assessed on its own facts, which means nothing here is a blanket rule. The practical reading is narrow but useful: a prompt log isn’t authorship, and the edit you make afterwards is the thing that carries a claim.

Where newsrooms are moving their people

Publishers have had three years to work this out, and their answer is visible in a survey. The Reuters Institute’s Journalism and Technology Trends and Predictions 2026 polled 280 senior editors, chief executives and digital leaders in 51 countries between 18 November and 20 December 2025. Back-end automation was called important by 97% of them, newsgathering by 82% and faster coding by 81%.

But adoption isn’t the interesting part. The verdict is: 13% described their newsroom AI initiatives as transformational, 44% as promising, 42% as limited, and 67% said no jobs had been cut as a result.

Asked what newsrooms should focus on more or less in the era of AI, the same group produced a ranking that reads as a map of where human judgment still pays. The figures below are net differences between the share saying “more important” and the share saying “less important”, on a base of 264 respondents.

Focus area	Net difference
Original investigations and on-the-ground reporting	+91
Contextual analysis, explanation and framing	+82
Community building, live journalism and events	+76
Human stories	+72
Fact-checking and verification	+63
Opinion and commentary	+55
Breaking news and live coverage	+26
Q&A type articles	-10
Evergreen content	-32
General news	-38
Service journalism	-42

Read the bottom of that table next to the top. The categories being cut are the ones a chatbot reproduces from what’s already published, and general news sits at -38 for exactly that reason. The categories being funded require somebody to go somewhere, ask something, or check a claim.

Fact-checking at +63 is the line that matters most here, because it’s the only item on the list that’s purely a verification cost. Even so, the same survey records confidence in journalism’s prospects at 38%, down 22 percentage points from its 2022 poll, so this is a reallocation inside shrinking budgets rather than a hiring spree.

What would change the conclusion

The market has already moved against the comfortable version of this argument. A WashU Olin study by Xiang Hui and Oren Reshef with Luofeng Zhou, summarised in August 2023, tracked Upwork freelancers after ChatGPT’s release and found writing-related workers took 2% fewer monthly jobs and earned 5.2% less per month. The drop was steeper for the freelancers with higher rates, top-rated badges and long job histories, which is the opposite of what a story about rewarded expertise predicts. The authors called the findings suggestive and short-term, and they were right to.

So here’s what would move us. A trial that graded assisted drafts on source accuracy, not prose, and still found human review adding nothing measurable would undercut the editing case directly, because the MIT experiment never tested it. Models that decline to cite what they can’t retrieve would shrink the verification bill instead of shifting it, though nothing in the error data suggests that’s close. And if the Tow Center’s error rates fell into single digits on a repeat run, the fact-check would stop being the expensive part of the workflow.

Until then the split holds, and it’s narrower than either side of the argument likes. Drafting is cheap, measurably faster and slightly homogenised. Verification, originality of reporting and the copyright claim are still produced by a person, and the published record of fabricated citations is the receipt. If you’re deciding which parts of your own process to hand over, the same test applies that we used when we looked at which workflows actually survive automation: ask what breaks if the output is confidently wrong, and keep a human wherever the answer is expensive.