TL;DR

Research published in Judgment and Decision Making gave 1,682 adults one of six short stories, half written by people and half by ChatGPT. Those who read a machine-written story rated it more absorbing and of better quality. In a follow-up, readers asked to pick the human-written story got it right 40% of the time in one experiment and 52% in another.

The design separates two things that usually get muddled. Participants were told whether their story came from a person or software, but that label was sometimes false. Ratings tracked the label as well as the text: stories badged as human scored better regardless of who actually wrote them. Readers who felt warmly towards AI in general gave higher marks to anything labelled as ChatGPT output.

Senior author Dr Deena Skolnick Weisberg of Villanova University argued the results should shift expectations rather than settle an argument. She suggested the appeal of AI prose may come down to being easier to take in, while the preference for human-labelled work reflects a want for authenticity. Weisberg, a fiction writer herself, was clear that room remains for human authorship as self-expression.

Familiarity with fiction did not help people spot the difference. Familiarity with AI systems did, which fits Weisberg’s view that machine writing carries a recognisable house style you can learn.

Luke Kennard, who teaches film and creative writing as a professor at Birmingham and had no involvement in the study, put the counter-argument bluntly. He rejected the framing of AI as a tool comparable to a spellchecker, pointing to training data taken without permission and the environmental cost of the data centres involved. Given the volume of human work these systems absorbed, he said, competence should be the floor rather than a surprise.

Looking Forward

For UK businesses commissioning written work, the finding that matters is not the quality comparison but the labelling effect: perceived authorship moved scores independently of the writing itself. That cuts both ways for anyone weighing up disclosure policies. Kennard’s objection also lands close to home, arriving in the same fortnight as reporting on rare books being pulped to feed training sets and continued opposition to data centre development across Scotland.