Wednesday, 13 September 2023

So ... is AI writing any good?

This blog post is a genuine attempt to investigate where things stand with AI writing.

It is not beating the drum for AI or advocating its use.

I have published interim results - you should not look at these before or during reading/voting.

It is simply asking whether there is currently any format of fiction where readers are unable to tell whether the text they are reading is written by a machine or by a person. 

There's clearly a lot of suspicion, fear, and annecdote about this subject and whilst I feel that I can tell if a piece is AI written or person written ... I don't know it for a statistical fact. 

Obviously the scope of this investigation is limited and the results rely on the good faith of those taking part. It's not a proper scientific study, which would ideally take place in a far more controlled setting with larger numbers of both text piece and testers.


But - reading takes time and I'm asking you to invest several minutes here, so I don't want to exceed your willingness to engage with the process.


I have focused on short fiction (flash fiction in fact) since it's clear that AI is not currently in the position where it can write a book that's in any way convincing without it being a hybrid effort involving considerable human input.

Below are (initially) 8 pieces of flash fiction, all of them ~350 words. All of them were written to the prompt: "write a piece of fiction based on 'meeting a dragon" - for the AI there were additional very brief suggestions concerning tone &/or setting to generate variety, an example might be "make it romatic and set on a cliff top".

some of these pieces were written by writers with significant experience whose books have many readers. They were written fairly quickly and don't represent the writers' finest work, but they were taken seriously and not written off-hand or carelessly.

some of the pieces were written by ChatGPT 4, which is the paid for version and currently about as good as is publically available.


since I asked for more than I needed the total number of pieces may exceed 8 and I will randomly sprinkle AI-written pieces in with new human-written pieces.


Each piece comes with 2 polls - one poll is for you to rank how much you enjoyed it, the other is for you to guess whether it was written by a person or not. The results of the enjoyment polls will be visible from the start. The AI/human poll results will be revealed after the experiment concludes.

I would ask people not to speculate publically about which were or weren't AI/human written as this will tend to clump opinion and will lessen the usefulness of the results.

Remember to assess these on their merits. If there is some tell-tale AI clue hidden in the punctation or whatever ... you really shouldn't be basing your guess on that. Guess based on the perceived skill of the writing - the imagination, passion, humour etc behind it.

And don't use AI detection tools -- that also would undermine the usefulness of this exercise.


You can read (and then vote on) as many or as few of these as you like, in any order.


Meeting a Dragon #1 

There is a nursery rhyme to recall upon meeting the scaled beast known as a dragon. 

“If it is black, beware its attack. If it is white, then say night-night. If it is blue, it will chew.” 

Even if I struggled to recall the rest of the colours, I knew that in some shape or form, I was about to die. Death was all that awaited a dragon encounter.

So, imagine my surprise when I had not turned into gristle between the beast’s stalactite teeth.

“What do you want?” the dragon growled through gritted teeth. Hot air made me stumble back with the promise of flame. The gold dragon reared its head towards me from its bed of treasures.

“I… I’m a thief.” Damned my traitorous tongue.

The gold dragon chuckled, causing coins to flee down the decline of its mound. “Honesty from a thief? How admirable.” Whatever amounted to a smile took form on its snout. “I assume you are here to take from me?”

My head nodded. Fear shackled me to the truth.

Another chuckle. “Very well, little one. I give you permission to take a single object from my vast treasury, and then never to return.”

My mind nearly broke from the golden shadow of death. It let me live! More than that, it let me take a single item. A gold and bejewelled goblet caught my eye. I was off and away.

The cave mouth led me back into the open. Wild and feral winds atop the mountain prodded at me and leached warmth. But I lived! My teeth still clattered, partially due to the close encounter with death but also due to the winds.

I took a step on the narrow edge to my right, preparing myself for a hard descent.

The rock broke. Gravity took hold and the wind gladly pushed me in the direction of open nothingness.

Ah. I recalled one of the lines from that old nursery rhyme.

“Should thou take treasure from a dragon of gold, then be assured to pay the price tenfold.”


How good was it?        VOTE HERE

Was it AI?                     VOTE HERE


Meeting a Dragon #2 

In the heart of the bustling city, atop the tallest skyscraper, where the old tales had been buried under the weight of modernity, I met a dragon. 

It was not the creature of my childhood stories. No broad wings, no scales, no fire-breathing terror. Instead, it was an elegant, sinuous being, shimmering like liquid silver, almost transparent, with eyes that held the stories of epochs gone by. 

“I thought dragons were myth,” I whispered, my voice catching in the wind. 

The dragon tilted its head, considering me with a gaze that seemed to pierce through time. “Myths are but forgotten truths,” it responded, its voice musical yet melancholic. “Just as this city has buried its forests, humans have buried their beliefs. I am the last of my kind here.” 

“Why are you here?” I asked, captivated. 

“To remind,” it sighed. “The world balances on the harmony of magic and reality. And now, the scales are tipping.” 

I looked down at the sprawling metropolis. From up here, I couldn't hear the honking of cars or the murmurs of crowded streets. Just the lonely echo of forgotten tales. 

“Can the balance be restored?” My voice was barely audible against the vast backdrop of stars. 

The dragon circled me, its form trailing stardust. “Perhaps, with belief. With stories. With voices like yours.” 

And as dawn began to break, the dragon unfurled wings I hadn’t seen before, stretching out in a blaze of colors and light. It soared, not into the distant past, but towards the looming horizon of tomorrow. 

I was left standing there, my feet grounded in the now, my mind alight with tales of magic and reality, ready to be shared, to be believed.


How good was it?        VOTE HERE

Was it AI?                     VOTE HERE


Meeting a Dragon #3 

The citys underbelly buzzed with menace. I stumbled upon the dragon, not amidst towering mountains, but hidden in the shadowed recess of a crumbling rooftop. Its grotesque form – scales festering, wings tattered like old rags, and one vacant eye socket – contrasted sharply with the one remaining eye, which glinted with rage.

You...” it hissed, thick chains rattling as they held it tight. Each link was etched with symbols that danced like malevolent spirits. Help ... me.

A cold wind whipped around us, carrying with it the stench of decay. The dragon strained against its bonds, smoke curling from its nostrils. “They took my sky, bound me in chains forged from the screams of the betrayed.”

A flash of movement caught my attention. Down a winding alley, a figure draped in ornate robes disappeared into a stone fortress. The overlord. Rumors had it he'd captured something powerful, using its essence to strengthen his tyranny over the city.

“The heart,” the dragon rasped, snapping my attention back. His heart... it holds my freedom.

Inside the fortress, shadows played tricks on my eyes, whispering of unseen horrors. Guards with soulless eyes prowled the maze-like corridors. Using the element of surprise, I dispatched them one by one, their falls muffled by the oppressive silence.

Finally, I reached the chamber of the overlord. His cold laughter echoed as our blades clashed, sparks flying. The room spun with every parry and thrust, until with a final desperate swing, my blade found its mark.

With the overlord's still-warm heart in hand, I raced back to the rooftop. The dragon devoured it, chains disintegrating in a burst of ethereal light. But instead of the gratitude I expected, its full power restored, the dragon roared, unleashing a fiery onslaught on the very city it once swore to protect.

I watched in horror, realizing too late the price of meddling with powers beyond comprehension. The citys smoldering ruins whispered of a freedom that came at a terrible cost.


How good was it?        VOTE HERE

Was it AI?                     VOTE HERE


Meeting a Dragon #4 

In a distant land, atop a mountain, Sir Roderick Dragonsbane lurked outside a cave. His name, though impressive, didn’t quite compensate for the fact that he was both young, and terrified.

Sir Roderick (the Twenty-Seventh, to be precise) approached the cave, crouching low and gripping his longsword in sweaty hands. He flinched at a scuffling and almost fled when it was followed by a rough snort.

Almost. But not quite. “Ahem.” Roderick waited, then tried again. “Ahem!”

“Who goes there?” a rasping voice boomed.

“It is I, Sir Roderick!”

“Which one?” The voice was joined by a pair of whirling eyes that glowed like embers in the darkness. “I seem to have lost count, though I recall the Twelfth being rather chewy. The Twenty-First was marinated in a very nice whisky.”

“I am Sir Roderick the Twenty-Seventh.” Roderick stood straighter as he spoke. After all, it was a noble name.

“Ah, humans. So eager to fill the world with your numbers. I suppose, given such short life spans, it is understandable. Well, Roderick the Twenty-Seventh, how may I help you?”

“Dragon, I am here to kill you!”

“Kill me?” The dragon chuckled, a laugh that filled the cave, bounced off the walls, and spilled down the mountainside. “Young knight, how do you expect to accomplish this when all your predecessors failed?”

“Failed?” Roderick’s bravery was rapidly trickling away along with the last echoes of the dragon’s humor. “I am descended from dragonslayers!”

“And, pray tell, how many dragons live in this kingdom?”

Roderick furrowed his brow. “One? You. You’re the last.”

“And how many years have I been the last?” the dragon enquired.

Roderick swallowed. “Erm. Quite a lot, I believe.”

“So, how many dragons do you think those ancestors of yours actually slayed.”

“Not… very many.” Roderick’s words dropped to a whimper. 

The eyes drifted closer. “I do hope you’ve found time to procreate,” the dragon murmured. There was a snap, a crunch, a swallow. “I quite enjoy my visitors every few decades.”


How good was it?        VOTE HERE

Was it AI?                     VOTE HERE


Meeting a Dragon #5 

Cinereal Grey followed her father along the twilight path between worlds. On her left, rainbow dragons frolicked among the bright colours of day. On her right, quiet ruled the monochromatic land of night.

“We are creatures of shadow,” said Father Grey. “Not for us the bright lights of day.”

Bowing her head, she walked the path he set before her. For he was wise in the ways and knew well the quiet life of shades. But the bright sounds of glee drew her attention and she listened to the shimmering dragons play.

“They are desperate for your attention,” said Father Grey. “Turn your head and look away. Do not feed their pathetic need, because you are more than they’ll ever be.”

And she plugged her ears and walked the twilight path. On her right, smoke dragons, lost in worlds of their own imagination. On the left, iridescent drakes chased each other in an endless game.

“We are neither here nor there,” said Father Grey. “No dragon can hear a word that you say.”

This time, she focussed on her father before her, noting the hunched shoulders, the way he looked neither left nor right.

She understood.

“It is you,” she said, “who is a creature of shadow. You shy from the bright lights of day.

Father Grey stopped walking.

“It is you,” she said, “who are desperate for attention but hide away.”

Father Grey turned to face his daughter.

“It is you,” she said, “who decides you are neither here nor there. Is it that they won’t listen, or that you have nothing to say?”

Father Grey looked away. “There are two worlds. In which will you stay?”

Cinereal looked left and right. “Both,” she answered. “When I need peace and respite, I will dream my stories in the land of the night. When I need adventure and play, I shall seek out friends in the land of day.”

Father Grey watched his daughter step to the left and for the first time saw her true colours.

He studied the twilight path ahead then turned to follow where she now led.


How good was it?        VOTE HERE

Was it AI?                     VOTE HERE


Meeting a Dragon #6 

Sunlight gleamed over the flats. The storm had passed, but the crossing remained treacherous, full of mud holes and sharp rock. I chose a stick from the ground as Cassie fidgeted. “I want to catch a skipper.”

“They’ll be out. Just follow me, all right?” I watched the ground. Movement along a tidepool showed as a dark flutter in the morning light. “Let’s go.”

Our feet made sucking sounds as we moved. I used my stick to gauge the bottom.

“There!” Cassie’s shout carried across the morning. “A red one!”

“Sush. Be quiet if you want to eat.” But my head turned, searching. Red ones were rare, for selling and not eating. We could use the money.

Then I saw it, the red; a shimmering, liquid red, vibrant enough to show from a distance though only a speck poked above the mud. I had never seen a color like that in the flats, or anywhere.

“I don’t think that’s a skipper.” I took a hesitant step, forgot to gauge it, and sank halfway to my knees. I held out my arm to keep Cassie following. “That way.” I pointed to my right where the ground looked firmer.

She was ahead of me then. As I struggled out of the sink she nearly ran toward her find, careless of mud or other dangers. 

And then it moved. The earth slid away from the crimson flesh, showing a wide leg with clawed feet. Its skin split the sunlight into rays of pink and turquoise and I stumbled, disoriented. Whatever this creature might be, the leg alone towered over me, and each of the claws were longer than my forearms. 

“Cassie!” But I could not see her.

The great leg found purchase, and then another pulled itself from the silt. The ground vibrated beneath my feet as the creature’s back emerged, huge and muscular, a riot of threatening color. Then a tail, and finally, a head, long and flat and shaped like a diamond, and glowing like one too. 

And then I saw her, Cassie, reaching out toward the creature’s mouth.

How good was it?        VOTE HERE

Was it AI?                     VOTE HERE


Meeting a Dragon #7 

In the tucked-away town of Bumblebrook, life was predictably pleasant. The streets hummed with gentle activity, and the aroma of Mrs. Penelopes freshly baked pies wafted through the air. But one sunny day, amidst rosebuds and daisies, a dragon decided to drop in.

Now, this wasn't your typical fire-breathing sort. No, this dragon had scales that shimmered like morning dew and wings that could've been plucked from a butterflys back. Instead of growls, it hiccupped, releasing silvery bubbles into the air.

“Blimey!” young Timmy exclaimed, nearly dropping his jam sandwich. His freckles seemed to dance with surprise. Venturing closer, curiosity overpowering his initial shock, he offered his sandwich to the creature, asking, "Fancy a bite?"

Much to Timmy's delight, the dragon accepted with a graceful nod, munching happily and producing even more bubbles, which sent Whiskers, Timmy's mischievous cat, floating and twirling above the rose bushes.

Miss Gladys, with her silver hair and stories of yesteryears, chuckled heartily from her porch. "Reminds me of a dragon I once knew," she murmured more to herself than anyone else, tapping her foot to a rhythm only she could hear.

The day slid into an impromptu festival. Kids chased bubbles, adults danced to tunes played on old gramophones, and the dragon? Well, it seemed to be the star, basking in attention, dancing, and sharing more bubble-blowing hiccups.

As stars painted the sky and Bumblebrooks lights began to dim, the dragon, belly round from treats and heart warmed by newfound friends, took to the sky, leaving a sprinkling of glittering stardust behind.

And in Bumblebrook, tucked in their beds, both young and old dreamt of the day magic chose to visit, reminding them of lifes unexpected, whimsical twists.


How good was it?        VOTE HERE

Was it AI?                     VOTE HERE


Meeting a Dragon #8

Beneath the tumultuous waves of the tempestuous sea, where the world above was but a memory shrouded in liquid obscurity, a woman found herself in an uncanny communion with a dragon of unfathomable proportions. This extraordinary encounter occurred as she plunged into the aquatic abyss, her thoughts echoing with the ephemeral songs of sirens.

In the subaqueous realm, the ethereal light filtered through the churning waters, casting an eerie, cerulean pallor upon the undersea terrain. Amidst this aquatic reverie, the woman beheld the dragon—an entity seemingly conjured from the depths of the collective subconscious.

This sea serpent, colossal and resplendent, was adorned in scales of iridescent azure and sapphire, each gleaming like the facets of precious gems. Its serpentine form wove through the aquatic expanse, sinuous and poised, every movement akin to a balletic performance beneath the waves. The dragons eyes, pools of ancient wisdom, regarded her with an inscrutable wisdom, as if peering into the very recesses of her soul.

The woman, bereft of the terrestrial constraints that bound her, met the creature with a sense of wonderment that transcended the mere corporeal. In this aqueous netherworld, she found herself capable of unspoken communion, where words were superfluous, and the language of the soul took precedence.

The dragons voice, a resonant melody that reverberated through the liquid medium, cascaded like sonorous notes of an otherworldly symphony. It spoke not in words, but in the vibrations of existence itself, revealing the mysteries of the submerged cosmos, secrets known only to the denizens of the deep.

As they communed, the woman perceived the dragon's realm—the hidden realms beneath the sea, the forgotten epics etched in the annals of oceanic time. She glimpsed the profound connection between land and sea, the interplay of tides and celestial forces, the genesis of storms, and the veiled narratives of shipwrecks and mariners lost to the abyss.

In this subaqueous reverie, time lost its dominion, and the boundaries between the terrestrial and the aquatic dissolved. The woman, an emissary of the surface world, shared in the dragons ancient wisdom, becoming one with the depths—a luminous fragment of the enduring mysteries of the boundless sea.

Their communion persisted until the woman returned to the surface, her heart and mind forever transformed by the spectral encounter beneath the waves—a communion that transcended the boundaries of time and reality, leaving her with a profound sense of awe and wonderment for the mysteries of the deep.


How good was it?        VOTE HERE

Was it AI?                     VOTE HERE



Meeting a Dragon #9

She came to Mrs. Craig’s store in August when the summer hung sticky and sweet. Nobody’d never seen her before. She walked in looking like she’d stepped off the screen in a picture show, all long and cool on that hazy summer’s day.

Jerry Mack was the first to move. Most folks said he was bad news around people's daughters, but his daddy was a judge and kept him out of trouble.

He ponied right up to that girl and bought her a cola. We all watched while she drank it. Her slim throat bobbed gently with each swallow. A drop of ice water fell to her chest, and every male in the store, from seven-year-old Bobby Price to eighty-year-old Sam Tredway, watched that bead’s trajectory as it dipped between the swell of her breasts.

Said her name was Aurea. Said her car broke down and she was looking to get a ride to Winston. Jerry Mack offered to take her.

Mrs. Craig stood behind the counter and watched it all with a gleam in her eye.

Jerry Mack and Aurea slipped through the screen door and stood on the porch. He gave her a cigarette and lit it for her. While they talked in low voices, she bent her hip and tossed back her hair. Maybe it was the light or her pose or even the smoke from the cigarette but for just an instant, her shadow flashed against the screen and looked just like the silhouette of a dragon.

Fast as it came, the image was gone and so were Jerry Mack and Aurea. We heard his car start and the crunch of gravel but we didn't see which way they went.

Two days later, word came they'd found Jerry Mack dead inside his burnt-up car at the bottom of a ravine near Chinquapin Hill.

Mrs. Craig took the news well. She went to the phone and made a call. Told Miss Magdalene she didn't have to worry about Jerry Mack threatening her daughter no more.

They said Mrs. Craig was a conjure woman. Said she went down to Chatham County and walked with the devil in his circle. Said for the right price, she'd solve your problems. Most everyone thought it weren't true, because she was always right there on the front row of Antioch Baptist Church every Sunday. 

I suspected different. Because after she hung up the phone with Miss Magdalene, Mrs. Craig went out on the front porch to smoke a cigarette. Maybe it was the light or her pose or even the smoke from the cigarette but for just an instant, her shadow flashed against the screen and looked just like the silhouette of a dragon.


How good was it?        VOTE HERE

Was it AI?                     VOTE HERE


Meeting a Dragon #10

“I didn’t expect to see you again so soon.”

He looked up at the red and gold-scaled dragon perched on a rock outcropping. Pebbles and dirt showered down in front of him as she leaned forward to see him, her claws digging into the ledge.

“So soon? It’s been nearly fifty years.”

“Fifty? Ah, yes, well you do look older, I suppose. Your home is far from here, why have you sought me out again?”

“Because you lied to me,” he said softly.

The dragon snorted and grey smoke puffed out of her flared nostrils. “I do not lie.”

“You told me that I would live a long and happy life.”

“And you didn’t? You’re old for a human, certainly if what you say is true about the time that has passed. At least seventy-five years of age.”

“Eighty-two,” he said, feeling it in his bones as he said it.

“And did you not have a loving wife, strong, clever children and grandchildren to dote on? A successful barrel shop and a full table?  This is what I used a wish for. Were you not happy with it?”

He sighed, looking down at his weathered hands. “Oh, I was mostly happy with my life, but how could any of it compare to my time spent here, with you?”

He could feel her breath on the top of his head and looked up to see her face close to his, her liquid blue eyes staring into his soul. ‘I was selfish to use a wish for myself. But I was alone, for so long after my mate fell. It was a good time, but it was illusion—magic. It was a fleeting time.”

“It was the best three years of my life. I too have lost my mate and all my little ones are grown. How I wish I could fly with you again. But forever this time.”

“I have only one wish left, and forever is a long time.”

“Please?”

“Yes. It’s a good way to use my last wish. Come, let’s soar together.  Forever.”


How good was it?        VOTE HERE

Was it AI?                     VOTE HERE











8 comments:

  1. This was a lot harder than I had expected it to be. Will be interesting to see the results.

    ReplyDelete
  2. I cannot get past the word 'atop', which no one ever says. I can only hope that was AI.

    ReplyDelete
  3. This was harder than it should have been.

    ReplyDelete
  4. Genuinely struggled with this!

    ReplyDelete
  5. I didn't expect this to be so difficult.

    ReplyDelete
  6. Loved it, but the poll didn't show how many voted ai and how many voted human on each. Enjoyed most of them. Want to know if I choose well, lol.

    ReplyDelete
    Replies
    1. you'll see the link to the results at the top of the post

      Delete
  7. I expected to be able to tell the difference no problem, as people have previously commented on definitely not as easy as I thought...very interesting!

    ReplyDelete