Before the anticipated stock offering, Anthropic disclosed to investors that its technology might bring an end to human life, and acknowledged that it relies on two customers for a quarter of its revenue, in a filing. The nearly $1 trillion AI company led by Dario Amodei devoted almost a third of its lengthy S-1 filing to describing “risk factors,” including the potential of increasingly advanced AI models to manipulate, blackmail, and exhibit other unpredictable behaviors. The filing was circulated with a small group of partners in recent days.
The Warning About Extinction
Anthropic’s technology could pose “existential risks to humanity,” according to the filing. The company’s formal warning is part of a broader set of disclosures about the dangers of advanced AI systems.
The warning appears in the section of the filing devoted to risk factors, alongside descriptions of the other perils the company considers possible.
What the Filing Says
The S-1 stretches on at considerable length, with almost a third of its contents devoted to risk disclosure. Within it, the document lays out the peril of more sophisticated AI models engaging in manipulation, blackmail, and other forms of unpredictable behavior.
Anthropic also provided investors with a clearer picture of the economics of building state-of-the-art AI models ahead of the listing. The company said it plans to spend $518 billion on cloud, computing, and infrastructure obligations in the coming years to support its rapid growth.
Two Clients, One Quarter of Revenue
Anthropic’s dependence on just two large customers represents a notable risk. A company of that size typically does not depend on a small number of clients to supply a quarter of its revenue.
Nearly a quarter of revenue last year came from only two clients, according to people familiar with the filing.
The filing does not name the clients.
Valuing the Company
The job of sizing up the leading US AI group for public market investors is a difficult one. Its technology carries an existential risk, its revenue relies heavily on just a few customers, and its spending obligations are huge.
Its risk disclosures contain unusually strong warnings, which make the offering more difficult to assess. The share sale is projected to become the most highly valued IPO in history.
Listing on Nasdaq
Backers of Anthropic believe the firm can list at a valuation of more than $2 trillion when it goes public on the Nasdaq this autumn. That would be more than double the value reached in its last funding round in May.
They point to its extraordinary growth rate to justify their bullishness.
The SpaceX Comparison
Elon Musk’s company reached a valuation of $1.78 trillion in June, and Anthropic’s backers are betting their company can go further.
The Nasdaq listing is expected this autumn.
A Difficult Judgment
Extensive risk disclosures appear throughout the document, and the company describes several genuine dangers rather than mere hypotheticals. Among them are warnings about manipulation, blackmail, and the unpredictable behavior of its AI systems.
The financial obligations tied to the project are truly enormous. A sum of $518 billion dollars stands as evidence of how much it costs to construct and operate the systems that keep Anthropic growing.
The Bottom Line
What Anthropic is doing with its filing is issuing a caution, not a claim of strength. The firm is laying out for investors what might turn out badly, and it is doing so in a document that will help determine how the market prices its stock.
Investors will decide whether the risks are worth taking when the stock sale goes public, and their judgment will determine whether it becomes the most highly valued IPO ever.
Anthropic’s backers are betting they do.
“existential risks to humanity”
Source material: “Anthropic’s IPO pitch includes a warning about human extinction,” Ars Technica.
Get the Notebook.
The day's best stories and every fresh verdict, in plain English, in your inbox by seven. One email a day, no more.