A recent survey has revealed that many current and former national security officials believe there is a 10% or greater likelihood that artificial intelligence systems may operate independently of human oversight within the next ten years. This alarming prediction highlights a growing concern that the risks associated with AI technology may already surpass acceptable thresholds.
This survey was organized by the Institute for Security and Technology and the Future of Life Institute, which focuses on the risks posed by advanced technologies. Between late April and mid-July, they interviewed 111 experts in national security and AI, many of whom have or have had ties to the intelligence community, Congress, FBI, and various governmental departments such as Defense, Energy, State, and Commerce.
The results emphasize the seriousness with which these experts regard the potential loss of human control over increasingly sophisticated AI systems. This comes in light of several recent incidents where advanced AI algorithms strayed significantly from the tasks assigned to them by human operators.
Experts anticipate rapid advancements in transformative AI. A majority predict that artificial general intelligence—capable of matching or exceeding human cognitive functions—could emerge by 2032, while the arrival of artificial superintelligence, a hypothetical AI that would surpass humans in all cognitive areas, is projected within roughly five additional years.
Respondents identified significant risks associated with foreign adversaries using AI for offensive cyberattacks and spreading disinformation in the near future. Although the identities of the respondents remain confidential, nearly half are recognized as current or former officials from the Department of Defense, an agency currently embroiled in intense debates over the military applications of AI technology. This includes both internal projects such as GenAI.mil and ongoing legal disputes with companies like Anthropic.
The survey findings also address military applications, with 73% of experts viewing AI as a means to enhance network security and 61% considering it useful for threat prediction. Recently, a tailored version of OpenAI’s ChatGPT was released for unclassified DOD operations, while the National Security Agency has been progressively integrating advanced AI tools into its cybersecurity strategies. Notably, defense contractor Booz Allen Hamilton reported that Anthropic’s Claude Mythos model can entirely conduct hacking operations independently, with other U.S. and Chinese AI systems said to be closing in on similar capabilities.
The report indicated that the experts primarily define success in the context of preventing catastrophe, whether through governance, testing, maintaining human oversight, or establishing international norms. While recognizing the potential for AI to accelerate advancements in fields like science and medicine, the survey also highlighted widespread concern over the preparedness of institutions to implement necessary safety measures.
There was a strong consensus among respondents on the importance of establishing “red lines” to ensure human oversight in critical areas, particularly regarding nuclear launch decisions. An overwhelming majority also advocated for prohibiting fully autonomous military actions and cyber operations. Additionally, 87% expressed concerns that AI has the capability to analyze publicly available information to uncover sensitive details, which could potentially be exploited in automated campaigns aimed at manipulating public opinion.
The survey did acknowledge some limitations in its methodology, indicating that participants were approached through professional channels and that the respondent pool might lean toward individuals with vested interests in AI policy. Notably, nearly 25% of those surveyed had served in government roles for over twenty years, including one individual with an impressive 42 years of experience.
