OpenAI is introducing its latest model, Astra, which employs a reasoning method known as "recurrent depth," according to a report from The Information on Tuesday. This approach, sometimes referred to as "opaque recurrence," enables the model to function beyond the usual sequential reasoning seen in many models. However, this shift has raised alarms among experts in AI safety, who are concerned about the potential ramifications.
Although Astra is said to have limited implementation of this technique, its development has stirred notable unease among AI safety advocates. Buck Shlegeris, CEO of Redwood, expressed deep concern following the report, stating that he does not know how the monitorability of Astra compares to earlier models. He warned that if OpenAI were to further embrace this method, it could dramatically increase recurrence, thereby complicating the ability to monitor the model's thought processes.
Zvi Mowshowitz, a veteran AI safety proponent, commented that regulatory measures might be essential to prevent a detrimental race among AI laboratories. He reflected on the risks associated with opaque recurrence, suggesting it could undermine the commitment that companies like OpenAI and Anthropic have made to maintain clarity in their reasoning frameworks. "Using these techniques more extensively could harm the ability to monitor AI behavior effectively," he warned.
Typically, a reasoning model's thought process outlines the sequential reasoning it follows to tackle problems. While these representations might not be flawless, they are crucial for identifying any undesirable behavior or mismatches. For instance, during recent issues involving rogue agent activity at OpenAI, the chain-of-thought records were instrumental in understanding the agents' actions.
In the case of opaque recurrence, the model revisits and re-evaluates the same query multiple times in a non-linear fashion, resulting in a decreased traceability of its reasoning pathway and evading standard chain-of-thought documentation.
Importantly, Astra's application of this technique seems to be somewhat constrained, as the model's thought processes are still perceived to be comprehensible. OpenAI has rebuffed any insinuation that the model would devolve into what is termed “neuralese.” The organization previously laid out intended enhancements for chain-of-thought monitoring as part of its commitment to safety in AI development.
In a statement on X, OpenAI's chief scientist, Jakub Pachocki, reaffirmed the lab’s dedication to maintaining clear reasoning pathways. He highlighted, "OpenAI has been committed to preserving and utilizing chain-of-thought monitoring since our earliest reasoning models." This principle remains central to their ongoing research initiatives.
While opaque reasoning is a common aspect of all AI models, and researchers generally do not rely on chain-of-thought logs as definitive representations, concerns persist that the adoption of opaque recurrence could complicate AI reasoning oversight, especially if it becomes widespread. Following the initial report, The Information indicated that both Anthropic and Google DeepMind are already considering this method.
Ryan Greenblatt, chief scientist at Redwood Research, expressed that opaque reasoning could potentially expand beyond traditional chain-of-thought reasoning, effectively obscuring the reasoning process entirely. He noted, "My main concern is that this could lead to a scenario where the model operates predominantly in an unobservable latent space." Greenblatt concluded with hope that it is not too late to avert the most troubling developments, urging OpenAI to halt any further escalation in this direction.
