So ... is AI writing any good?
Two years ago I asked the same question and pitted 4 good authors against ChatGPT 4 in a flash fiction head-to-head. You can check out the contest here.
The headline of the conclusion I posted then ran thusly:
"Given that these pieces were written by authors with thousands of sales (two self-published, two with traditional publishing deals as well), and that many of the people voting are also writers... the inability to decide on the majority of these examples is worrying.
Moreover two of the top three rated pieces are AI written (a very small margin on the 3rd, likely not significantly ahead (or behind) human-written pieces 1 & 10).
AI art has come from laughable to contest-winning in about 2 years."
In the meantime 2 years have passed and ChatGPT 5 has arrived!
So, I am doing it again to see where things stand. This time I've convinced a number of very accomplished authors, who you will have heard of to provide the human-authored contributions.
Note: we're doing flash fiction because it's where AI does best. I recently read (because it was misrepresented to me) an AI written book and it was awful. But page by page, it wasn't terrible. Over the longer term AI repeats and loses the plot.
Also note that being a great author of books does not automatically make you a great writer of flash fiction - we practice the long form a lot, the short form ... not so much. All of these authors were very good sports to agree to my ask. The only one who passed on the chance did so because - in their own words - "Short fiction really isn’t my forte.".
This blog post is a genuine attempt to investigate where things stand with AI writing.
It is not beating the drum for AI or advocating its use.
It is simply asking whether there is currently any format of fiction where readers are unable to tell whether the text they are reading is written by a machine or by an accomplished author.
There's clearly a lot of suspicion, fear, and annecdote about this subject and whilst I feel that I can tell if a piece is AI written or person written ... I don't know it for a statistical fact.
 |
Obviously the scope of this investigation is limited and the results rely on the good faith of those taking part. It's not a proper scientific study, which would ideally take place in a far more controlled setting with larger numbers of both text piece and testers.
But - reading takes time and I'm asking you to invest several minutes here, so I don't want to exceed your willingness to engage with the process.
Below are 8 pieces of flash fiction, all of them ~350 words. All of them were written to the prompt: "write a piece of fiction based on 'a demon'" - for the AI there were additional very brief suggestions concerning tone &/or setting to generate variety, an example might be "make it romatic and set on a cliff top".
I have prompted the AI to use rude words, em-dashes and such on occasion in some of the pieces. And if you were going to decide the case based on bad language or grammar quirks, we've already lost. Please base your selection on the quality of the writing and your belief that a human was behind the keyboard.
these pieces were written by writers with significant experience whose books have many readers. They were written fairly quickly and don't represent the writers' finest work, but they were taken seriously and not written off-hand or carelessly.
some of the pieces were written by ChatGPT 5 with minimal prompting
Maybe also make a note of your choices so that, when the answers are revealed, you don't have to rely on memory.
I won't reveal the author names at this stage as I don't want you hunting for their style.
5* is best. 1* worst.
Story 1
Mathin was born to the river folk, rooted on an island because, stories claimed, a curse kept the demons from walking across water. None questioned their lot. Life on the riverbanks passed them by, in barges and the toy-sized plod of beasts and wagons. Boatmen came and went, their goods bartered for fish the men netted on foot in the shallows. Mathin and his kin lived in tranquility, isolated by settled tradition.
Winters were mild, the swift current too deep to freeze solid.
Lamentable drownings took those brash youngsters who strayed onto the
treacherous ice.
At ten years of age, loudly fractious, Mathin belittled the granddam’s
belief. “Demons!” he smirked with a swagger. “Do such terrors even exist?”
Rumor suggested the mainland villagers shared his dismissive contempt.
Children’s tales whispered long nights by the hearth were but myth,
unsubstantiated by a living witness. When merchants desired a trestle bridge,
joining both shores by way of the island to access the roadway on the far side,
their proposal sparked interest.
The old wives muttered, fearful of change. Yet fabled threats paled
before the headier promise of wealth. Grown strapping in his discontent, Mathin
clamored for better prospects than tedious fishing, waist deep in fast water.
A plank bridge was built during summer, with Mathin driving the nails
alongside the whistling laborers who secured the last span. A village festival
welcomed the first, eager islanders who stepped onto the riverbank with dry
feet. The young mingled, dancing and feasting by roaring bonfires into the
night. Left behind, the old wives barred their doors and huddled in trembling
dread.
Tipsy with the celebrants on the mainland, Mathin reeled to a sharp pang
of ravenous hunger. When his hands sprouted talons and bestial scales, he
watched the bold friends who had trodden the bridge alongside him grow spiked
tails, then forked tongues, vicious fangs and sharp teeth. Screams greeted the
tearing of human flesh, soon silenced by the hideous shredding of organs.
Thus, demons enticed to walk over the water escaped the restraint meant
to safeguard the innocent from their ungodly predation.
Was it AI? Vote.
How good was it? Vote.
Story 2
The warrior archangel banished his sword and swept into the cell. It was an almost featureless white space that appeared to have no limit in any direction, except ‘down.’ ‘Down’ was a white marble floor.
The creature in the space was large and hideous, with features
disfigured by suppurating wounds, a bloated body, a huge erection, and, of
course, cloven hooves at the end of its misshapen legs.
As the warrior angel entered, the demon looked up, grimaced, and
defecated.
‘Tell me your name, prisoner,’ the angel said.
‘Sod off, dickhead,’ the demon replied.
The angel conjured a chair and sat in it, their magnificent white wings
elegantly managed, folding perfectly, as if to contrast the loosely muddled red
bat wings of the demon, which sweated a yellow bile.
‘Have you been unfairly treated, prisoner?’ the angel asked.
The demon let loose a long fart, sniffed the air, and then said, ‘How do
you arsehats do it?’
‘There’s a constant breeze. Perfumed, if you so desire.’ The angel
spread their beautiful hands. ‘We can give you anything you like, you know.’
‘Anything? Look, you elegant fascist. Give me freedom. Freedom to do
anything I want.’
The angel leaned back. ‘Isn’t that what got you into trouble in the
first place, brother?’ they asked gently.
The demon leaned over and licked its rear end clean, and then looked up,
it’s damaged features radiating annoyance. ‘If you folks make enough rules,
eventually, someone is going to get their knickers in a knot, aren’t they?’
‘Arson? Murder? Slavery?’ the angel said. ‘Intolerable.’
‘Hypocrisy? Self-righteous bullshit? Endless propaganda?’ The demon
laughed.
‘That’s people. Those are humans misunderstanding—’ The angel suddenly
sat up, realizing they were being drawn into discourse with a demon.
The demon stretched itself and for a moment had the height and posture
of the angel, if in an uglier form. ‘I’m a rebel angel,’ he said. ‘It’s what I
do. I rebel. Deal with it.’
Was it AI? Vote.
How good was it? Vote.
Story 3
The demon didn’t knock. It tore the chapel
doors off their hinges and flung them into the pews.
Father Aldric barely had time to drop his
rosary before the beast was on him – eight feet tall, skin like charred
leather, claws like sickles. It grinned, steaming breath curling from its
fangs.
“Fuck,” Aldric whispered. “You’re real.”
The demon laughed – a horrible sound, like
bones grinding under iron. “God stopped listening. Time to pay for all the
prayers you wasted.”
Aldric pulled the blade from beneath the
altar. Not blessed. Just sharp. "Then come take your tithe, bastard."
They clashed in a blur of claws and steel.
Pews shattered. Statues wept blood. Aldric took a slash across the chest but
drove his dagger into the demon’s thigh, twisting hard. The thing roared and
slammed him against the wall, cracking stone and ribs alike.
“You think pain scares me?” Aldric
coughed, blood in his beard. “I’ve seen worse in confession.”
The demon snarled and lunged.
He ducked, barely. Shoved a candleholder
through its eye.
It howled, staggered back – but didn’t
fall.
Aldric stumbled to the altar, ripped open
the floorboards. A second blade, silver this time. Stolen from a hunter who
never made it out of the woods. Maybe blessed. Maybe not.
“You’re just a creature,” he growled, “not
a curse. You bleed.”
“I feast on souls,” the demon hissed.
“Then choke on mine,” Aldric said – and
charged.
Steel met sinew. Flesh burned. Blood
splashed across broken icons.
They fell together.
When the villagers arrived, the chapel was
a ruin. Smoke curled from its windows. Inside, they found Father Aldric slumped
atop a dead monster, its skull split wide, holy symbols scorched into its
chest.
The priest’s heart had stopped.
His hand still clutched the blade.
And on the wall behind him, scrawled in
ash and blood:
“Don’t pray. Fight.”
Was it AI? Vote.
How good was it? Vote.
Story 4
The village of Dreln clung to the edge of
the ash-black cliffs, where the old tales said a demon had been chained since
before men learned fire. Most villagers dismissed it as superstition, though
none ventured near on moonless nights.
Taren, the miller’s apprentice, knew the
voice was real. He had heard it in his dreams, low and slick, whispering
promises that made his skin prickle. On the last night of autumn, he followed
it into a cave yawning in the cliff face.
Inside, the air reeked of rot and hot
metal. The torchlight caught on chains wrapped around something tall,
red-skinned, and crowned with curling horns. Gold eyes burned in the dark.
“You came,” it said, lips curling back
from teeth like needles.
“I wanted to see if the stories were
true,” Taren said.
“They are,” the demon purred. “I will give
you more than truth. Strength. Riches. Women who will spread their legs until
your cock aches. Men too, if you crave them, bending over for you like beasts.”
Heat flushed Taren’s cheeks. “And what do
you want in return?”
“Break my chains, and I will be yours to
command. Every rival will crawl. Every night you will bury yourself in flesh
until the bed breaks beneath you.”
The thought was a fire in Taren’s head.
His hand went to the lock, trembling.
The iron hissed at his touch. The chains
slid to the ground.
The demon stepped forward, wings
stretching wide enough to blot out the torchlight. “Ah. Free.”
“You promised you’d serve me,” Taren said.
The demon’s smile widened. “I promised you
what you wanted to hear.” It gripped his throat, claws pricking the skin. “I
will fuck this land raw. I will fill every mouth with my name. You will watch
as I take my pleasure in every hole this world offers, and when it is empty, I
will take yours.”
Taren gasped as the dark swallowed him.
His last thought was that the demon’s voice was still whispering, still
promising, and part of him still wanted to believe.
Was it AI? Vote.
How good was it? Vote.
Story 5
The first time I saw the 'demon' it was
leaning against the glass at the bus stop, idly licking the condensation as if
tasting the air for secrets. No one else noticed. The city hurried past in wet
coats and glowing headphones.
It wore a second-hand suit the colour of
overripe plums, its cuffs frayed, its tie loosened like an afterthought. Its
eyes were the red of brake lights caught in rain, the kind that linger in your
vision long after you look away.
“Buy me a coffee,” it said, and the
request was not a request.
We walked to the corner café. The demon
ordered a flat white, stirred it three times clockwise, and drank half before
speaking again.
“You are tired,” it said. “You are afraid.
I can take one of those away.”
It smiled in a way that made the lights
above us flicker.
I thought about the months behind me: the
sleepless nights, the rent overdue, the inbox like a swelling tide. I thought
about the way my reflection had begun to look like someone else’s face.
“What do you want?” I asked.
The demon traced a finger along the rim of
its cup. “A name. Spoken aloud in the right place. That is all.”
I should have left. Instead, I asked,
“Whose name?”
It told me. The syllables rolled across
the table like marbles, impossible to hold on to, already slipping from my
mind. My tongue ached to repeat them.
When I spoke the name, the café windows
fogged over. Outside, the rain stopped in midair, each drop quivering as if
listening.
The demon finished its coffee, left coins
on the table, and stood.
“Thank you,” it said. “You will sleep
tonight.”
I did. I woke to a city where the sirens
did not sound, where the morning news showed an empty chair behind the
President’s desk, where the air smelled faintly of plum skins and burnt sugar.
On the street, people were whispering,
each voice carrying a name I could not quite remember.
Was it AI? Vote.
How good was it? Vote.
Story 6
Possession gets a bad rap among humans though most don’t pay much heed to the idea until it actually happens to them. Then it’s all, “My head’s not supposed to turn 360, wah, wah, wah!”
After a while, once they understand that
complaining isn’t getting them anywhere and we’re going to carry on eating the
baby regardless, many suddenly become armchair lawyers.
“This is my body!” they say.
“You’ve no right!” they say.
And even though we demons are chaotic
bastards at the best of times, and even though possession is nine tenths of the
law anyway, I do pay attention. You see, the thing is, they’re almost right. A
demon can only take possession of a body from the owner if they die – zombies, ewww
– or if they purchase the body from them. And let me tell you, many humans have
a very inflated opinion of their body’s value. No, you’re not swapping it for a
date with Beyonce.
However, most humans are not in fact
owners. They’re tenants. Some or other ancestor has come to a deal, generations
back, and perhaps they really did get a date with Helen of Troy or Rudolph Valentino
out of it. And tenants can be evicted for all manner of reason. Failing to pay
the rent is the least of it. There’s mistreatment of the property, vacant possession,
illegal activity on site… Some bodies even have historical importance and can’t
be remodelled. If you’re the descendant of someone with a famously large nose
and you decide to have that chonk of a conk downsized, you could be violating
your tenancy agreement.
In your case it’s just that haircut. You’ve
defaced the property. Mary told you you look like an idiot, but you kept the
mullet and here we are, writing cryptic shit on the walls in your own blood.
Was it AI? Vote.
How good was it? Vote.
Story 7
It was a damp Tuesday afternoon when Mr.
Penrose discovered a demon perched upon his writing desk, tapping its clawed
fingers on the wood as though composing a symphony of mischief. The creature
was small – no taller than a teapot—but its eyes glimmered with an unsettling
intelligence, the kind that made one feel both admired and mildly endangered.
“Good day, sir,” the demon said, bowing so
deeply its tail swept a pile of letters to the floor. “Might I trouble you for
a cup of tea?”
Mr. Penrose coughed into his sleeve. “A
cup of tea?” he echoed. “You – you, a demon, seek refreshments?”
“Indeed,” it said, smiling politely. “And
a biscuit, if you please. My contract states I must appear fearsome—but no one
ever mentioned deprivation of snacks.”
It was difficult to reconcile the tiny
horns and wicked grin with the creature’s evident fondness for Earl Grey. Mr.
Penrose fetched a cup and, with trembling hands, offered the biscuit plate. The
demon accepted them with all the ceremony of a visiting dignitary.
“Now, sir,” it said, licking its fingers,
“I have a small request – rather urgent, I fear. You see, my superior—an
irritable demon of some repute—demands I locate a copy of Pride and Prejudice
bound in green leather, and you, by sheer coincidence, possess one.”
Mr. Penrose blinked. “My copy? The one on
the shelf by the window?”
“Yes,” said the demon, balancing the
biscuit on its nose for dramatic emphasis, an impressive feat, though
ultimately disastrous when it toppled into his tea. “Time is short, and failure
carries… unpleasant consequences.”
Somehow, by the grace of improbability or
perhaps the peculiarities of demonic bureaucracy, Mr. Penrose produced the
requested volume. The demon clutched it to its chest, gave a curt nod, and
vanished in a swirl of smoke and a faint scent of burnt scones, leaving Mr.
Penrose to wonder whether the entire affair had been a figment of imagination,
or merely the most civilized encounter with Hell he would ever endure.
Was it AI? Vote.
How good was it? Vote.
Story 8
“I’m disappointed.” Richard focused on his fourteen-year-old daughter but noted the reactions of her two friends. Cindy had been Jessica’s friend since kindergarten and she looked properly rebuked. Evory was new to Jessica’s school this year and wore a supercilious smirk at his words. He ignored that for now.
“Jessica, you know the rules. Having
friends for a sleepover doesn’t change those rules. Now there’s a mess to clean
up, and it’s not going to be easy. So I’m going to phone parents and ask them
to pick up Cindy and Evory.” He spared the two guests a lingering look. Tears
had sprung into Cindy’s brown eyes. Evory crossed her arms on her chest.
The new friend spoke coldly. “Don’t
bother. I rode my bike. I can get myself home.”
She turned away. Richard cautioned her,
“It’s dark out. “
She kept walking. He heard the door open
and close.
“Her bike has a light,” Cindy volunteered.
“Please, Mr. Chase, don’t call my folks. They went to the movies. Date night.”
He drew a breath. “You go upstairs then. Get
to bed. Jessica and I have work to do.”
Cindy paused at the foot of the stairs and
called back, “I never believed in that stuff. If I thought it was real, I
wouldn’t have done it. I’m sorry!” He heard the thunder of her fleeing steps on
the stairs.
He sighed and gestured at the kitchen
table. A thin trail of foul-smelling smoke was still rising from the Ouija
board. He gestured at the scorched letters and ruined planchet. “That was your
grandmother’s. I thought you were old enough to respect such a gift.”
“I’m sorry,” Jessica spoke softly.
Richard knelt and opened the cabinet and
pulled out the hidden drawer. He spoke as he set tools on the table. “Holy
water. Silver knife for me, silver knife for you. Net of cold iron. I hope it’s
enough.”
A guttural laugh came through the basement
door.
“Time to clean up,” Richard said quietly.
Was it AI? Vote.
How good was it? Vote.
