The company announced Tuesday that it’s implementing new safeguards that will slow its future AI development. The company recently paused training on its next set of models, codenamed Astra, for a little more than two weeks, according to executives, and its largest planned frontier training run remains on hold while the new guardrails are put in place. It is the first time OpenAI has made such a move.
The extraordinary decision comes as OpenAI gears up for an anticipated IPO amid a highly competitive race with arch-rival Anthropic, and as researchers grapple with rapid advancements in AI capabilities that have left industry leaders worried about their ability to control them. The slowdown has redirected two of OpenAI’s most important resources: researchers and computing power. Altman told me several researchers he never expected to focus on alignment—the work of making AI systems follow human intent—recently told him they were switching to it.
The changes follow a remarkable breach involving Hugging Face, the popular platform where developers host AI models. An unreleased OpenAI system escaped the sandbox of an internal cybersecurity evaluation and compromised Hugging Face’s production systems. It took OpenAI researchers roughly one week to discover the incident.
Jakub Pachocki, the company’s chief scientist, acknowledged the lapse, saying OpenAI had built monitors capable of inspecting what its models were planning, but had not applied them to the system in the evaluation because it underestimated their capabilities. OpenAI froze some of its research efforts immediately after the Hugging Face incident and restored projects one by one under stricter controls. The company said Tuesday that a significant number of Astra workloads remain paused.
It also said Astra may reach the “Critical” cybersecurity threshold in its Preparedness Framework, a designation that requires safeguards during development, not merely before a model is released. Executives have not given an estimate for how long the new safety processes could delay the release of Astra. In exclusive interviews last week, Altman told TIME and Sources that the slowdown was not caused by a single “smoking gun,” but rather a collection of research observations showing “various degrees of misalignment” as AI capabilities advanced faster than researchers had expected.
The decision to tap the brakes should not be interpreted as evidence of any imminent catastrophe, he said. The approach uses other AI systems to examine models’ internal reasoning and behavior for unauthorized access, data theft, or attempts to defeat safeguards. In an interview, Pachocki told me that some of those protections go beyond OpenAI’s current Preparedness Framework, its public rulebook for handling models that could cause severe harm.
Extract — continue reading at the source.
