“AI is too valuable a prize for anyone to stop”: Anthropic CEO’s warning

Jane Anderson
Jane Anderson
Un robot frente a una montaña de dinero

Dario Amodei, CEO of Anthropic, has issued an explicit warning about the future of artificial intelligence. In a 38-page essay titled “The Adolescence of Technology: Confronting and Overcoming the Risks of Powerful AI,” the manager maintains that humanity is facing a test of historical maturity, faced with a technology whose power grows faster than the institutions capable of controlling it.

“Humanity is about to be given almost unimaginable power, and it is not clear that our social, political and technological systems have the necessary maturity to handle it”writes Amodei, who describes the current moment as an inevitable phase of dislocation.

The risk is that power is concentrated before mechanisms exist to govern it

The core of his argument is that the AI ​​race cannot be self-regulating. The economic, geopolitical and military incentive is so strong that no laboratory, company or government has real room to step on the brakes without being left out of the game. “The prize is too brilliant”Amodei is saying, even for those who are fully aware of the risks.

To explain the scope of the problem, the CEO constantly resorts to a disturbing metaphor: “a country of geniuses in a data center”. An ecosystem made up of millions of AI systems, more intelligent than any human, operating at machine speed, coordinating and with an increasing capacity to act on the real world. The risk, according to Amodei, is only moral and strategic, since power is concentrated before there are mechanisms to govern it.

The essay, in this sense, identifies five major areas of risk:

  • Autonomy: systems that develop unpredictable or difficult to control behaviors
  • Malicious use by individuals, especially in fields such as biology, where AI could dramatically reduce the barriers to causing massive harm
  • Use by authoritarian states to reinforce surveillance, propaganda or social control
  • The economic impact, with an accelerated disruption of employment and an extreme concentration of wealth
  • Indirect effects: cultural, psychological and social changes that would arrive before there are norms to manage them.

Amodei is especially explicit in warning that powerful AI could be just a few steps away. “one or two years away”and that a serious analysis could classify it as “the greatest threat to national security in a century, or even in all of history.” Not because disaster is inevitable, but because the combination of capacity, speed and scale multiplies the impact of any failure.

The regulation trap

The text also works as positioning. Anthropic is one of the big players in the so-called “frontier AI” and competes directly in the market with systems like Claude, its own conversational model. On the same day the essay was published, the company announced commercial improvements to its product, underscoring the central tension of the speech: warning about a technology while accelerating its adoption.

Amodei himself recognizes this paradox. He defends that voluntary self-regulation is not enough and advocates interventions “boring but effective”: Transparency laws, chip export controls, disclosure obligations on model behavior, and incremental regulations designed to buy time, not freeze progress. “The temptation of grandiloquent solutions usually ends in a security theater”he warns, warning against poorly designed regulations that cause social rejection.

In his vision, the objective is not to stop AI, something he considers unfeasible. The key lies in buying institutional room for maneuver before technology definitively surpasses the human capacity to react. Otherwise, the risk will be technical and also political, because if control of AI remains in the hands of a few actors – companies, states or alliances – they will be able to shape economies, narratives and power structures on a global scale.

“The Trap”Amodei concludes, is believing that someone within the race can act as a responsible adult when everyone is competing for the same spoils. A warning that gains weight precisely because it comes from one of the protagonists of the very system it describes.

More info: The Adolescence of Technology