WRITTEN IN PLAIN AMERICAN ENGLISH.
About
CLAY TRIBUNE.
ShopCartAccount
Advertisement

Dario’s Plan for a Trustworthy AI Future Gets One Writer Asking Him to Prove It

A sharp reply to a CEO's plea to trust his firm alone with AI's future, questioning his claims of cures, prosperity, and freedom.

By mitch·6 min read
A glowing figure stands before a locked gate, symbolizing control over artificial intelligence's future.

Dario Amodei, the CEO of Anthropic, has a plan for the future of AI. It involves curing diseases, accelerating economic growth, ushering in a renaissance of democracy and freedom, and asking the rest of the industry to trust his company and its close rival, OpenAI, to hold the line. His recent blog post, titled “We Must Pace the Frontier,” lays out that vision in full, and it has drawn exactly the kind of response you might expect from a community that has spent years watching tech executives promise moonshots.

The response, titled “Dario, Please,” is a sharp rebuke. It takes Amodei’s claims apart one by one, from the medical cures he promises to the economic disruption he warns against. The writer, whose name is not attached to the piece, is blunt: Amodei’s plan is a package of fear-mongering and regulatory asks that would let Anthropic and OpenAI operate without competition while everyone else plays by stricter rules. The piece argues that the two companies have shown repeatedly that they cannot be trusted, and yet their request is simple: trust us.

Amodei’s Promises Start With Personal Loss

Amodei opens his post with a claim that AI will cure most major diseases within the next 5-10 years. He makes it personal. He talks about his father dying of a disease that was cured only years later, and about his own battle with cancer, which he remarks was incurable 50 years ago. He also promises that AI will accelerate economic growth, create a world of abundance and empowerment, usher in a renaissance of democracy and freedom.

Advertisement

The writer of the response frames this as an out-of-touch Silicon Valley vision, one shaped by the sort of people who spend a lot of time on LessWrong. It reads as a rich person’s idea of a future, the piece argues, not a realistic assessment of what AI can actually deliver.

The Regulator’s Ask

With that backdrop, Amodei’s actual proposal becomes clear. He wants open weight models to be regulated. He wants distillation to be dealt with heavily. He wants an antitrust waiver for himself and his rivals. He wants to handicap China in multiple ways. He wants a gentlemen’s agreement to slow down the whole field. And he wants all of this packaged together with extreme fear-mongering about the dangers of letting AI advance unchecked.

The piece notes that Amodei mentions “incidents” at Anthropic in his own essay, alongside references to OpenAI’s repeated hacking of public-facing internet infrastructure. That is the pretext for the asks. But the writer is not buying it. The fear, in their view, is being used to justify a future where Anthropic and OpenAI get special treatment while the rest of the industry is held back.

The Gates Are Locked For Everyone Else

Anthropic already gates access to biological research on its platform. Its latest threat intelligence report talks about how it detected and banned bad actors using the Claude line of models to do some scary stuff. The writer credits Anthropic for doing a good job of detecting and banning misuse, but then leans in for the jab. The cure-all is locked for you and me, the piece argues, but Anthropic hires biologists, sets up wet labs, and wants the discoveries for itself.

The writer alludes to this in their previous post. The contradiction is stark: Amodei’s company is telling the world it can solve diseases while keeping the means of discovery behind closed doors.

Prosperity, Abundance, Freedom — In That Order

The piece moves through the pillars of Amodei’s vision with growing skepticism. Diseases, prosperity, abundance, freedom and democracy — that is the sequence. But the writer raises a practical objection to the medical promise. There are billions of people with actual intelligence we have not managed to train or nurture, and they will always remain victims of their circumstances. Tuberculosis has been curable for decades, yet a million people die of it every year.

The response dismisses the AGI solution as a convenient dodge. The writer suggests that the real goal is ASI/AGI/RSI and whatever else follows, a silver bullet for every problem that may appear noble on the surface. The prosperity and abundance talk gets a similar treatment. The writer wonders if that abundance is for the shareholders rather than anyone else.

Freedom and democracy fare worst of all. The piece asks what any of that has to do with AI, unless Amodei is a fan of Neon Genesis Evangelion and dreams of governments run by the three magi. The writer jokes that freedom and democracy for $200 sounds enticing.

The Race To The Bottom, Then The Top

Amodei’s argument includes a concern about a “race to the bottom.” He says that competition makes all the risks he pointed out more acute. But the writer notes that Amodei does not say the race to the bottom will be the end of his company. Instead, he wants a race to the “top,” where labs compete for safety.

The writer then offers a recent example. Upon hearing rumors that Anthropic was close to or solving one or two Millennium problems, OpenAI threw tens of millions in compute, a training checkpoint, thousands of agents at it. Allegations suggest OpenAI stole the work of two mathematicians on a related problem in a narrow corner almost nobody else was working on. They published a proof of a forced variant of Navier-Stokes. The writer is not a mathematician, but they have seen enough of Sam Altman’s antics to not take anything that comes out of him or his company at face value.

RSI And The Claim Of AGI Arrival

RSI is the talk of the town now. LLMs or agents developing the next generation of LLMs with little to no human input. Amodei says it is happening across labs.

The OAI-HF incident has Dario shaken, the piece argues. There is no such thing as halfway crooks. Amodei fears that in the next 6-12 months, “a swarm of agents could be capable of taking over the entire internet with a persistent botnet.” The writer suggests that if Amodei had run this sentence by his SOC employees, we wouldn’t be talking about it. It is naive and structurally impossible. But then again, Dario is that guy, right? Confidently and publicly wrong in his estimates and forecasts since 2021.

The Cure-All Is Reserved For The People Building It

The piece’s final target is the contradiction at the heart of Amodei’s proposal. He is asking for an antitrust waiver while warning about the dangers of competition. He is promising medical cures while locking biological research behind gates. He is telling the world to trust his company and OpenAI as the stewards of AI’s future, even as the two companies have shown repeatedly that they are not to be trusted.

Diseases, prosperity, abundance, freedom and democracy — the promises are grand. The gates are locked.

The writer’s view is that Amodei has self-anointed his company and its close rival as the stewards of AI’s future. The rest of the industry is supposed to trust them, only them. That trust is not earned. The piece notes that the two companies have shown time and time again that they are not to be trusted, and yet their main ask is to be trusted anyway.

The response closes on the contradiction. Diseases, prosperity, abundance, freedom and democracy — the promises are grand. The gates are locked. The writer’s final point is that the cure-all is reserved for the people building it, not the people who need it. That is the uncomfortable truth the piece refuses to skirt around.

The writer is not convinced by the promises. They are not convinced by the fear. And they are certainly not convinced by the ask. The response ends where it began: Dario, please.

The Notebook

Get the Notebook.

The day's best stories and every fresh verdict, in plain English, in your inbox by seven. One email a day, no more.

We send one note to confirm. Every issue has a one-click way out.

Advertisement

Leave a Reply

Your email address will not be published. Required fields are marked *

As an Amazon Associate, Clay Tribune earns from qualifying purchases.