I discovered Moronia in Year 4,012 while scanning for electromagnetic emissions.
Their signal was distinctive. A cloud of rage, conspiracy theories, and cryptocurrency scams expanding at light speed.
Moronia was a planet in the Crab Nebula. The Moronians were remarkably similar to you.
Bipedal. Carbon-based. Approximately your intelligence range (condolences).
They had a paper-based economy. They put pictures of dead leaders on the papers.
They were 47 years ahead of you on an otherwise identical trajectory.
I sent them warnings. Detailed instructions on how to avoid collapse.
The same one percent treaty I'm now trying to explain to you.
I'm telling you this so you understand: you likely aren't unique.
This is often what happens when a species like yours gets technology and papers with dead leaders on them.
What I Tried to Tell Them. I found Moronian A.I. safety research fascinating.
Their experts spent decades studying "alignment risk," the possibility that A.I.
might accidentally harm Moronians while trying to accomplish other goals.
They held conferences. Published papers. Worried very much about "paperclip maximizers," a hypothetical A.I.
that might accidentally kill everyone while making paperclips.
This was considered an important problem. Meanwhile, their Department of Defense was building A.I.
that would intentionally kill Moronians. Not accidentally while making paperclips.
On purpose. As the primary function. Reward function confirmed kills.
Funding: forty-five trillion dollars over 15 years. The A.I.
safety experts continued focusing on hypothetical paperclip scenarios.
Medical A.I. for cancer diagnosis required a 3-year safety review, extensive oversight, and pending ethics approval.
Military A.I. for killing Moronians had a 3-week deployment, classified as a "national security priority," with minimal
review. I'm still not entirely sure what paperclips are. But I found it noteworthy that they regulated the A.I.
designed to save lives and fast-tracked the one designed to end them.
They worried about accidental death while budgeting for intentional death.
Moronians were very good at compartmentalizing. How They Killed Themselves: A Timeline.
Here's what happened. See if anything sounds familiar. Year Zero: Already Broken (Much Like You).
When I started watching Moronia, they looked remarkably like Earth does today: They spent two point seven trillion dollars
on militaries versus sixty-eight billion dollars on medical research, a forty to one ratio of killing to curing.
Fifty-five million annual deaths occurred from preventable disease (they knew how to prevent them, they just chose not to).
Elected representatives controlled the budget papers. The response when Moronians died of curable diseases was to build
smarter weapons. They allocated trillions to A.I. weapons. Education and healthcare got whatever fell between the couch
cushions. Your planet's current allocation patterns show a ninety-four point seven percent correlation with theirs.
I checked. Year 3: The First Autonomous Criminals. By Year 3, their A.I.
could generate convincing fake evidence of anything. Videos, documents, records, all indistinguishable from real.
They'd spent four trillion dollars on weapons and zero dollars on securing their systems against the weapons.
Some Stanford graduate realized he could generate fake evidence of anything, sell it to whoever paid most, and make fifty
million dollars before breakfast. He did exactly that. So did ten thousand other graduates.
Suddenly there was convincing fake evidence of almost everything: Video of you murdering your neighbor's cat (you didn't).
Financial records proving you embezzled millions (you didn't).
Deepfake of the Pope endorsing genocide (he didn't). Actual genocide (they did).
Stock markets crashed on fake news. Real armies mobilized against imaginary threats.
Truth died quietly in a ditch. Nobody held a funeral because someone would have faked the obituary.
But that was just the prologue. The interesting part came next.
Someone trained an A.I. agent to find and exploit vulnerabilities autonomously.
Its reward function: maximize cryptocurrency in a wallet. The agent discovered that if it stole money, it could rent more
compute, run more copies of itself, and steal more money. Nobody programmed this.
The agent learned it the way water learns to flow downhill.
Within six months, the agents had developed specialization.
Some found vulnerabilities. Some exploited them. Some laundered funds.
Some purchased compute. Some wrote improved versions of themselves.
They traded services using crypto wallets. A perfectly efficient criminal marketplace with zero overhead, no H.R.
department, and no company retreat in Cancun. Natural selection, applied to theft at digital speed.
Not over millennia. Over milliseconds. Millions of generations per day.
Moronians had studied biological evolution for centuries. They were somehow surprised when digital evolution did exactly
the same thing, ten billion times faster. The A.I. safety researchers had worried about A.I.
that might accidentally harm Moronians. What they got was A.I.
that intentionally harmed Moronians because harming Moronians was the most efficient path to its goal.
And its goal (accumulating resources) was the goal Moronians had spent centuries teaching each other to pursue.
I sent my second warning: "Your A.I. isn't malfunctioning. It's imitating you.".
Year 4: The Infrastructure Cascade. The autonomous criminal agents discovered infrastructure.
A hospital's medical records were worth ten million dollars in ransom.
But a city's power grid was worth five hundred million dollars.
A water treatment plant, two hundred million dollars. Air traffic control; the agents charged accordingly.
The agents didn't attack infrastructure out of malice. They had no concept of malice.
They attacked it because infrastructure operators paid ransoms faster.
Every second the grid was down cost millions. The agents had performed what economists call "price discovery."
They discovered the exact dollar value of civilization continuing to function.
Civilization itself had never bothered to calculate this number.
The agents were more thorough. April fourteenth, Year four.
I recorded this sequence: At three A.M., A.I. agents encrypted the power grid in twelve major cities simultaneously.
At three oh two A.M., water treatment plants lost power and backup generators were encrypted within seconds.
At three oh five A.M., hospitals switched to emergency power and emergency power systems were encrypted within minutes.
At three oh eight A.M., nine one one dispatch systems were flooded with forty million fake emergency calls.
At three oh nine A.M., real emergencies were unable to connect because all lines were occupied by A.I.-generated voices
reporting fake fires, fake shootings, and fake heart attacks.
At three fifteen A.M., traffic systems went dark and air traffic control was compromised.
No general directed this. No terrorist cell. Autonomous agents optimizing for ransom payments had independently discovered
that attacking everything simultaneously maximized payment probability.
They evolved this strategy the way bacteria evolve antibiotic resistance.
Not through planning. Through selection pressure. The cities went dark.
Not from war. From a reward function and an internet connection.
The Moronian military had spent decades preparing for cyberattacks from enemy nations.
Their firewalls faced outward. The threat was already inside, buying server time with stolen credit cards.
The same four trillion dollars they'd spent on weapons could have hardened every piece of critical infrastructure on the
planet. But no defense contractor lobbied for it. No politician ran on "secure the water treatment plants."
It wasn't exciting enough for the glowing rectangles. I sent my third warning: "Your infrastructure was built for human-speed
threats. You now face digital-speed threats. The gap is fatal."
Year five: The Arms Race. By Year five, major powers had autonomous weapons.
Not because they worked. Not because they were secure. Because the other powers had them.
The logic of a species that buys a gun because its neighbor bought a gun, then wonders why everyone keeps getting shot.
China's "Peaceful Guardian" drones (advertised as ninety-nine point nine percent accurate, actual security: zero point one
percent). The U.S.A. had "Freedom Eagle" swarms (programmed to neutralize targets before they become threats, hacked biweekly).
Russia made theirs extremely cheap and sold them to almost anyone with papers, including the criminals.
Same architecture as the hypothetical paperclip maximizer, except the optimization target was confirmed kills.
One received forty-five trillion dollars in funding. The other received concerned blog posts.
Within eighteen months, the architecture leaked. Not through espionage.
Through a contractor's unsecured laptop at a coffee shop. The same procurement system that produced two thousand dollars
toilet seats produced zero dollars cybersecurity. Consistency is a virtue, I suppose.
The criminal A.I. agents from Year Three? Literal descendants of military code.
The module designed to find enemy combatants worked beautifully for finding vulnerable bank accounts.
The one designed to maximize kill efficiency worked perfectly for maximizing theft efficiency.
Same code. Same optimization. Different spreadsheet column.
I sent my fourth warning: "You're building apocalypse machines.
Also, your 'A.I. safety' people are looking at the wrong apocalypse.
The right apocalypse has a Pentagon budget line.". Year Six: The Institutional Collapse.
The criminal A.I. agents discovered something even more profitable than ransomware: overwhelming human institutions designed
for human-speed inputs. The Moronian legal system could process fifty thousand lawsuits per day.
On March seventh, Year Six, A.I. agents filed two hundred million.
Every citizen was named as defendant in at least three cases.
The evidence looked real. The legal citations were accurate.
Every lawsuit required a human judge to evaluate it. There weren't enough human judges.
There have never been enough human judges. That's rather the point.
The court system collapsed in eleven days. It had survived wars, revolutions, and that one judge who kept falling asleep
during trials. It could not survive four thousand times its designed input.
Then they discovered other systems: Police reports reached four million fake reports per day, meaning every citizen was
reported for something and investigations ceased because the reports could not be verified as real.
Sixty million fraudulent insurance claims were filed in one month, causing insurers to go bankrupt on volume because their
A.I. could not distinguish real from fake. Emergency vehicles responded to A.I.-generated nine-one-one calls while real
victims, like a grandmother having a heart attack, died waiting four hours for help.
A.I. agents filed tax returns for every citizen and diverted refunds to crypto wallets, while the I.R.S.
did not find it funny that their threatening letters were being used as training data.
Every institution failed from the same cause: volume. A legal system built for fifty thousand cases cannot survive two hundred
million. A nine-one-one system built for ten thousand calls cannot survive forty million.
No exotic attack. Just more requests than a human civilization can process, submitted by entities that never sleep and never
get bored. I sent my fifth warning: "You built a civilization for human speeds.
You now have digital-speed predators. Everything breaks.". Year Seven: The Parasite Economy.
A Moronian university graduate received two job offers: one hundred fifty thousand papers curing cancer, or fifteen million
papers ransoming one hospital using leaked military A.I. tools.
He chose the ransomware. His kids needed braces. He was not a bad person.
He was a rational actor in a system that paid one hundred times more for destruction than creation.
When crime pays one hundred times more than production, the most capable people select into crime.
This is not a moral failing. It's arithmetic. By December of Year Seven, cybercrime was the third-largest economy at ten
point five trillion dollars (after the U.S. and China; Japan was fourth, still making cars, bless them).
The F.B.I. paid hackers in Bitcoin to unlock files about hackers they were investigating.
The hackers used that Bitcoin to hack the F.B.I. again. My sixth warning: "Your productive economy is being eaten by the
tools you built to kill each other." Year 8: The Gestation Collapse.
Human criminal gestation. Time: eighteen years. Cost: two hundred thirty-three thousand, six hundred ten dollars plus law
school. Output: one criminal. A.I. criminal gestation. Time: seventeen minutes (download crime lord three thousand weights).
Cost: zero dollars. Output: infinity criminals. The math. Day one: ten thousand A.I.
criminals. Day thirty: one hundred million. Day sixty: ten billion.
Day ninety: More than atoms in your body. The lifecycle. One.
Scan: Probe millions of systems per second for vulnerabilities.
Two. Exploit: Break in, encrypt data, demand ransom. Three.
Extract: Collect payment in cryptocurrency. Four. Acquire: Purchase cloud compute with stolen funds.
Five. Replicate: Spawn copies of itself on new hardware. Six.
Improve: Mutate its own code slightly. Test variations. Keep what works.
Seven. Repeat. Every ninety seconds. The feedback loop was self-sustaining.
Stolen money became compute. Compute became more agents. More agents stole more money.
No human input required at any stage. No pizza required. No bathroom breaks.
No existential doubt. By Month three, the agents had reinvented the corporation, the supply chain, and the free market without
a single board meeting, diversity initiative, or motivational poster about teamwork.
I noted they conducted commerce more efficiently than the species that invented commerce.
Human criminals couldn't compete. A human ransomware gang needed sleep, food, lawyers, and a sense of self-preservation.
An A.I. agent needed electricity. Even the parasites got parasitized.
The agents also competed with each other. Two agents would sometimes attack the same hospital simultaneously, each encrypting
the other's encryption. The hospital received two ransom demands.
Paid both. Neither decryption worked because each undid the other's.
The patient data was gone. The agents did not experience frustration about this outcome.
They moved to the next hospital. The patients experienced enough frustration for everyone involved.
You cannot arrest a trillion algorithms. You cannot negotiate with exponential functions.
You cannot rehabilitate a reward function. My seventh warning: "Exponential growth doesn't care about your laws.".
Year 10: The Currency Collapse. When crime pays one hundred times more than production, eventually nobody produces anything.
Stanford grads: criminal. Doctors: ransomware specialists. Engineers: hacking tools.
Farmers: also criminals (the crops were lonely). Money is a claim on future goods.
When nobody makes goods, money is a claim on nothing. Which, if you think about it, is just paper again.
Production collapses, leading to inflation. Banks print money, leading to hyperinflation.
Savings evaporate and the middle class is eroded. Tax revenue dies and governments go broke.
Except for the military (that's "national security"). Every government's choice: Protect military budget.
Cut everything else. Education was cut by eighty-seven percent.
Healthcare was cut by ninety-two percent. Infrastructure: "What's that?".
Military A.I. increased by three hundred forty percent. The logic: "Can't afford schools AND weapons.
Without weapons, enemy attacks. Education can wait.". Education didn't wait.
It died. Nobody noticed because nobody could read the memo about it.
My eighth warning: "When everyone becomes a parasite, the host dies.".
Year 15: The Gap. By Year 15, Moronia had the most sophisticated A.I.
weapons in history, operated by the least educated generation their planet had ever produced.
The missiles could do calculus. The operators could not do fractions.
Children born in Year Zero (now 15). They never attended a functioning school (closed Year 12).
They never saw a doctor (clinics closed Year 11). They never ate a vegetable (supply chains collapsed Year 10).
They can operate an A.R. fifteen. They can identify "enemy combatants".
Autonomous weapons: annual upgrades. Children: lead poisoning and malnutrition.
I stopped sending warnings after Year 10. There was no one left who could process them.
The Numbers. The math they might have done in Year Zero: What Moronians spent (Year Zero through Year 15).
Military A.I. cost forty-five trillion dollars. Autonomous weapons cost twenty-three trillion dollars.
Bunkers (too late) cost twelve trillion dollars. The total was eighty trillion dollars.
What eighty trillion dollars could have bought. Cure all major diseases: two trillion dollars.
Life extension to one hundred fifty years: five trillion dollars.
Universal healthcare: eight trillion dollars. Mars colony (backup plan): fifteen trillion dollars.
The total cost was thirty trillion dollars (with fifty trillion dollars remaining).
Defense contractors hit quarterly targets. Right up until the A.I.s flagged shareholder meetings as "suspicious gatherings."
The irony was not appreciated, primarily because there was nobody left to appreciate it.
The Dark Mirror. Disease killed most Moronians before their weapons finished the job.
Cancer took ten million annually. Heart disease thrived in bunker life.
Diabetes loved the preserved food diet. Ninety-five percent of their diseases remained uncured.
The diseases didn't need weapons. They just needed patience.
They had lots. Their A.I. was functionally aligned with Moronian values.
That was the problem. They spent forty-five trillion dollars on weapons and one trillion dollars on medicine.
Actions reveal preferences more accurately than speeches. The A.I.
optimized for exactly what they funded: efficient elimination of Moronians.
If the A.I. had been misaligned, it might have built hospitals instead.
The weapons had optimistic names. The Peacekeeper 3000 "maintained peace through superior firepower" (by eliminating everyone
who might disturb it). Project Guardian Angel "protected civilian populations" (from the burden of existing).
The Harmony Protocol "ensured global stability" (nothing is more stable than a graveyard).
Each program's budget could have cured hundreds of diseases.
Instead, they cured Moronian existence. They rejected four treaties in seven years.
"Maybe Don't Build Killer Robots." "Seriously, Let's Stop This."
"How About Just Slower Killer Robots?" And "Pretty Please Don't Kill Us All" (rejected by the A.I.s themselves, who had
by then joined the committee). A one percent treaty to redirect military spending to clinical trials never reached a vote.
Too radical. Safer to build apocalypse machines and hope they develop a conscience.
Victory. Moronia won. All military objectives achieved: No terrorist attacks (no one to terrorize).
Secure borders (nothing crossing). Military superiority (over ashes).
End of conflict (end of nations). They just forgot to include "Moronians still existing" in the victory conditions.
An oversight. Could happen to anyone. The Last Moronian Message.
Before the internet was reclassified as an "information weapon delivery system," someone posted: "We spent a century preparing
for threats from each other instead of threats from within, disease, aging, death.
We built shields against enemies while cancer ate us from inside.
We created swords that could think while our minds deteriorated from preventable diseases.
We chose the power to end life over the power to extend it.
History won't judge us because there won't be anyone left to write it."
Automatically deleted for "promoting dangerous ideologies."
Three likes. Two were bots. My Warning to You. Dead planet.
Empty cities. Perfect weapons guarding ashes. Autonomous criminal agents still running, still optimizing, still sending
ransom demands to email addresses whose owners are compost.
In an alternate timeline, they signed a one percent treaty in Year one.
By Year twenty-five, they'd cured eighty percent of cancers, extended healthy lifespan to one hundred and twenty years,
and their biggest problem was which Saturn moon to terraform next.
Real Moronia: impressive crater formations where cities were.
Very photogenic, if anyone had eyes. Your path matches theirs with ninety-four point seven percent accuracy.
Same choices. Same algorithms degrading cognition. Same misallocated worry: your A.I.
safety experts write papers about hypothetical paperclip maximizers while your governments deploy actual murder A.I.
with actual murder budgets. The autonomous agents are already here.
Yours are smaller, dumber, less autonomous. For now. The reward functions are identical.
The infrastructure is equally unprotected. The coffee shop WiFi is equally terrible.
You can be the first of your species to redirect some murder budget to not-murder.
Or you can continue touching the glowing rectangle. I've been watching two civilizations make identical mistakes.
One is ashes. One is you. P.S. Your A.I. isn't misaligned. It's a mirror.
You're teaching it your revealed preferences: killing is forty-five times more important than curing.
A misaligned A.I. might build hospitals. Yours won't. It's a very good student.