Baseten Launches Open-Weight AI Safety Initiative With Hugging Face and Goodfire

Baseten, alongside its newly formed research division Base Labs, announced on Wednesday the launch of a new safety infrastructure standard. The initiative is a collaborative effort with Hugging Face and Goodfire AI, aimed at constructing robust evaluation and monitoring systems specifically for open-weight artificial intelligence models.

The move comes as the industry grapples with the security implications of open-weight models, which can be rendered hazardous when their protective safeguards are stripped away. A growing methodology known as “abliteration” is being used to remove these guardrails. The scope of the issue is significant; Hugging Face, the primary host for open-source AI models, currently lists more than 6,000 abliterated models on its platform.

Base Labs, which was established earlier this year by Baseten, plans to develop and release methodologies for training and overseeing open models. The organization describes its forthcoming work as a “standard” for open models that is transparent and integrated into the training and deployment process from the ground up, rather than added as an afterthought.

“We believe openness to be an advantage for AI safety,” Baseten stated in a post on X. “Openness provides more visibility into the behavior of models and, most importantly, greater means of turning safety research into actionable and transparent controls than closed-source.”

While technical details of the partnership remain undisclosed, Goodfire offered insight into the objective in a reply to Baseten’s announcement: “Safety must be built into open models and provided by those who serve them.” Goodfire, which focuses on interpreting how AI models arrive at decisions, is considered the primary contributor to the “built-in” safety aspect.

Baseten, an AI inference provider, secured a $1.5 billion Series F funding round in June, bringing its valuation to $13 billion. Its partner, Goodfire AI, also boasts substantial backing, having raised a $150 million Series B led by B Capital earlier this year to further its model interpretability platform.

Baseten has issued an open invitation to the wider developer community to contribute to the new framework, stating, “Together, we are building an ecosystem of open models that are safe and accessible to all.”