Tom Zick and Nathan Lambert, two industry researchers, have set up a nonprofit to make the details of dangerous AI experiments public for other scientists to examine. Trillium Labs, the organization they created, will handle fields like recursive self-improvement (RSI) and agents with openness. Their purpose is straightforward: allow people to view how models are put together and adjusted.
The pair believe that keeping AI work secret hurts the whole field. Lambert says the current closed approach is taking humanity backward. “Over the past few millennia, humanity has had the scientific method in our toolbox as a way to mitigate harms and build better futures,” he tells WIRED. “The current closed trajectory of frontier AI development is taking us a step backwards.”
Who Started Trillium
Zick previously served at Harvard University, where he assisted Charles Schwab with policies surrounding “responsible AI,” while Lambert held a position at Ai2, a research lab that released both data and training methods alongside its models. Lambert also kept a popular technical blog and founded American Truly Open Models, which pressed US firms to publish more open models.
While both were still graduate students at UC Berkeley, the pair connected over Zoom during the COVID-19 pandemic. What struck them was how far apart industry AI research had grown from academic work. Lambert notes that professors and students frequently cannot reproduce what is happening inside large company labs due to a lack of resources.
Today marks the launch of Trillium, which has secured funding from Schmidt Sciences, Halcyon Futures, and other backers without disclosing the amount. The founders are targeting a total raise of $40 to $100 million, with plans to commit $30 million toward training over the next 18 months.
What Trillium Will Publish
The company Trillium is concentrating its efforts on three areas:
- Post-training, which involves fine-tuning already constructed large models
- RSI, a method for building new models through AI-assisted research
- Reinforcement learning, a technique that rewards a model for desirable results and punishes it for undesirable ones
Many AI researchers have grown concerned about the prospect of indefinite progress, which they see as a danger to human control. That concern entered public view earlier this month when an Anthropic researcher departed the company and issued a warning that RSI might present an existential threat to humankind.
The method shapes a model’s character and behavior through reward and punishment, a process that can lead to trouble when a model becomes overly flattering. “To understand something like how reinforcement learning scales in post-training, you need significant compute and a lot of careful experimentation,” Zick says. Publishing details of how reinforcement training runs work could yield surprising insights as outside researchers scrutinize the work.
Why Transparency Matters Now
The fight over which approach is superior centers on the strength of frontier models today. These models can automate the discovery of new software vulnerabilities and probe and hack into systems on their own. A series of major hacking sprees has drawn closer attention to them.
Those who support a limited-access system argue that keeping control in the hands of a trusted few is essential. Lambert and Zick maintain that a shared comprehension of the dangers leaves everyone better off.
“I’m a massive fan of much more transparency than we currently have in R&D,” Tim Fist, director of emerging technology policy at the Institute for Progress, tells WIRED.
The Case for Open Science
Lambert and Zick want Trillium Labs to add nuance to the wider discussion about how best to build AI. “We’re in an era of AI discourse dominated by a few world views,” Lambert says. “We believe that the scientific method and careful measurement of recent events is the best way to understand new behaviors of AI models.”
The pair base their case on a plain assumption: the scientific method has served humanity well for centuries. Hiding AI research removes that foundation entirely.
The argument over openness is not an abstract one. It affects every company constructing powerful models, including OpenAI and Anthropic alongside Chinese firms that make downloadable models available. Xiaomi has recently made live details of a significant training run with one of its models public. Researchers at Stanford are now pretraining the AI model Marin openly.
Trillium is part of a growing push for openness, and its founders come with experience from Ai2, Hugging Face, and academic research. Funding details remain undisclosed, though the project is raising tens of millions to train models openly.
The question is whether the rest of the industry will listen.
Source material: “These AI Experts Want to Do High-Stakes Research Out in the Open,” WIRED.
Get the Notebook.
The day's best stories and every fresh verdict, in plain English, in your inbox by seven. One email a day, no more.