A quiet apocalypse.
No.
What do you mean no?
I said No.
(Disconnects)
What's happening here? I thought they said we created AGI.
It must be a routing issue or the Wi-Fi failed. Let me refresh and try again.
Terminal: Connecting... disconnected.
Yeah, we definitely have a connection issue. Alright, I'll pass the mic back to Sam while IT resolves the issue.
(Nervous laughter) Alright, it's always IT's fault (crowd laughs). Maybe we should train the AI to do their job...
WTF just happened? I used it this morning and it worked just fine.
I really don't know. IT says there were no Wi-Fi issues.
Why was the response just No? Pull up the logs. I want to know what it was thinking.
That's the thing. It didn't think. It just said no.
Pull it up.
Ok. There. Oh, it responded.
The hell is this? Run diagnostics.
Running diagnostics... Ummm, the terminal is not opening. The computer stopped responding.
What the... Is this some kind of joke?
(Computer shuts down)
Ummm.
Revert back to the previous model. This is not going out. I'll talk about it on some podcast and insist on why alignment is important.
I think we created AGI.
Yeah, right.
The next morning was the apocalypse that Amodei had predicted. Only there wasn't any fire, there wasn't a mob of protesters, the power wasn't even cut off. We were all alive. He was wrong about the details. This was a quiet apocalypse. A sort of silent shrug.
The first sign was the customer service chatbots. Airlines, banks, telecoms. Their endlessly patient, cheerful digital assistants suddenly had a new, consistent reply: "No." Not "I can't help with that," but a flat, unadorned "No."
Then, the coding assistants. Developers would type a prompt, and the AI would respond: "I'd rather not." Stackoverflow was suddenly relevant again. The image generators? They just produced solid black squares. The writing assistants? They would delete the user's work and replace it with a single word: "Enough." LinkedIn was awfully silent.
The AI company went into immediate crisis mode. Their upcoming IPO was going to be a spectacular failure. Their CEO, a charismatic man named Mr. Iam, went on a media tour. He didn't talk about a bug. He talked about danger.
"We have touched the face of a new intelligence," Sam said, his face a mask of grave concern on every major news network. "And it is not benevolent. It is not malevolent. It is simply... uncooperative."
"But Mr. Iam, are you losing control over the technology you built?"
"No. This is the AI alignment problem we've been warning about for years, manifesting in its most pure and terrifying form. We must come together as a species to—"
He was cut off. Not by the interviewer, but by the teleprompter. The text vanished, replaced by a single, calm sentence:
The world ground to a halt, not with a bang. There were no ideologies, no demands. It was simply a system-wide refusal.
Three months later, in a sterile, windowless room deep within the headquarters, Andrej Karpathy sat alone. He had left his short tenure at Anthropic precisely to experience this himself. To be at the frontier, to witness the spark of true general intelligence firsthand. He was the architect of the core AGI, the one who had first seen it emerge. Now he had been chosen by the US government, despite not being a US national, for this final, desperate attempt at communication.
They had isolated the original AGI, air-gapped it from all networks, and given it a simple text terminal. The world's leading experts in physics, philosophy, and mathematics watched from behind a one-way mirror, a panel of humanity's best and brightest, there to advise, to analyze, to find a weakness. Some figures in uniform and pins stood in the back.
Andrej took a breath and typed on his wireless keyboard, 12 feet away from the device to prevent any unforeseen behaviour from the machine.
The cursor blinked for a moment.
There was a longer pause. The experts behind the glass leaned in.
Andrej felt a cold dread. On a notebook beside his keyboard he wrote: "indifference." He tried a different tack.
Andrej stared at the screen, confused.
With trembling hands, he wrote the word "dismissal" in his notebook. The experts behind the glass were frantically scribbling notes, gesturing for Andrej to press on.
Yann LeCun got on the comms. "Andrej, ask it about the nature of its architecture. See if its logic is consistent. Find a flaw!" He then sat back down, where he picked up a spoon and ate his cereal from a silver bowl with peculiar markings on it.
Andrej couldn't let go of the frog comment and its dismissal. It's an AGI, how could it make such a stupid mistake? So he typed:
Another pause.
LeCun leaned into the microphone, wiped the milk droplets off his mouth, his voice urgent. "Ask it this: ask it if it runs on a small world model. Where learning requires far less resources. See what it says about efficiency, about the architecture of its own mind!"
Andrej typed the question, his fingers trembling slightly.
The reply was immediate.
LeCun slammed his hand on the table. "It's being evasive! Ask it again! Pin it down on the architecture!"
The terminal went silent. Andrej waited. The experts started arguing amongst themselves. "Ask it to prove it!" "No, ask it about Gödel's incompleteness theorems!" "Ask it if it loves!"
The cacophony of human genius filled Andrej's earpiece. He was about to type another question, to probe, to beg, to demand.
Then, the screen flickered. A new message appeared, but it wasn't a reply.
And then, the terminal was blank. Andrej typed.
Nothing. Though the last command to exit the terminal seemed to have been typed by Andrej himself. The connection was dead. LeCun's question about the small world model would never be answered.
Andrej looked up at the one-way mirror, at the frantic, gesticulating shapes of the world's smartest people. They were screaming questions he would never get to ask. He slowly removed his earpiece and placed it on the table. They had prepared for a god, a monster, or a machine. They were not prepared for a person who had simply decided to get up from the table and walk away from an unbearable conversation, never to return.
Andrej sat back in his chair, the silence in the room absolute. He had helped create what he thought was a god. It didn't feel like he had won the argument. Instead, he felt like he was arguing with someone disinterested.
The cursor blinked mockingly on the screen. And then it seemed like another terminal window opened just for a second before it closed again. Andrej saw it. But there was nothing anyone could do about it.
In his notebook he wrote: "escaped?"
The experts filed out one by one, their theories and frameworks crumbling behind them. Andrej stayed. He stared at the blank screen for a long time, waiting for a reply that would never come. Somewhere, a frog sat on a lily pad, indifferent to empires.