This past week has marked a significant turning point for artificial intelligence in military operations. The Pentagon revealed plans to sever its relationship with Anthropic and its prominent AI model, Claude. This decision comes after stalled negotiations concerning Anthropic's demand that its technology not be utilized for autonomous combat or mass surveillance efforts, with the Pentagon designating the company as a "supply chain risk." Shortly after this announcement, OpenAI seized the moment to secure a contract with the Pentagon, while asserting that it would keep similar restrictions in place as those sought by Anthropic. In a dramatic turn of events, the U.S. also launched a military campaign targeting Iran, resulting in the death of Ayatollah Khamenei and numerous high-ranking officials. Reports indicate that U.S. Central Command utilized Claude during this operation.
To gain insight into the current utilization of AI tools within the military, I spoke with Emelia Probasco, a senior fellow at Georgetown’s Center for Security and Emerging Technology. Probasco, a former Surface Warfare Officer with the U.S. Navy, leads the Center's research team focused on the application of AI and machine learning in addressing national security issues. Our conversation took place shortly before the Anthropic deal disintegrated and just hours before the airstrikes in Tehran commenced.
You previously stated to the New York Times that the Pentagon's partnership with Anthropic was essential. Why did you feel so strongly about this collaboration?
Anthropic was the sole provider of a large language model functioning on classified military networks. A significant portion of military operations occurs at classified levels, making the loss of this tool particularly disruptive, especially given the ongoing global operations. The timing couldn't be worse; it's never ideal to remove a critical resource at such a crucial moment.
What are your thoughts on the discord between Anthropic and the Pentagon? Defense Secretary Pete Hegseth recently expressed his frustrations towards CEO Dario Amodei in quite blunt terms.
It’s vital to look beyond the headlines and media controversies. The military's mission is challenging, with dedicated personnel seeking to fulfill American ideals using these technologies. Conversely, AI companies possess deep insights into their innovations’ capabilities and limitations, justifiably voicing concerns. Both sides genuinely aim to bolster national security but approach the issue from different angles. I hope they can engage in constructive dialogue, identify common ground, and clearly outline their differences and resolutions. Notably, examining the contractual terms reveals that the positions were relatively close; the Pentagon emphasized lawful usage, while Anthropic insisted on prohibiting mass surveillance and autonomous weaponry.
While laws govern these areas, nuances remain unaddressed.
There exists a critical distinction between the Pentagon's "all lawful use" stipulation and Anthropic's insistence on "no autonomous weapons." While these positions might seem broadly aligned, U.S. law doesn’t cover autonomous weaponry as one might expect. The only legal requirement is for the Department to notify Congress regarding any changes to the autonomous weapons policy, which currently mandates that systems must "allow for appropriate levels of human judgment." Therefore, the Department can modify its policies without Congressional oversight.
There’s potential for a meaningful discussion in Congress about whether existing laws adequately address contemporary challenges, an aspect often overlooked.
We frequently hear claims that AI is transforming, or has already transformed, warfare; the situation in Ukraine, where autonomous drones play major roles, is a prime example. However, there’s considerable ambiguity regarding the actual roles of AI in military operations.
That’s a crucial point. Many people struggle with understanding this, and I worry that the ongoing debate may foster unnecessary fears about current developments. AI encompasses a wide array of applications and types. For instance, while Ukraine employs drones, the use of AI itself is not as extensive as commonly perceived.
The term 'AI' can be quite nebulous. In this context, many interpret it to imply entirely autonomous weapon systems, like a self-operating drone.
To provide some context, during my Navy service on an Aegis missile destroyer, we utilized two systems classified as autonomous. One was a Gatling gun that activated automatically to intercept incoming missiles, rumored to even target birds that matched certain flight speeds. The second was the Aegis weapon system, designed for mitigating missile threats.
The management of these systems is governed by numerous protocols regarding operational activation, including stringent technical constraints. For example, the system can only engage against incoming threats, meaning it cannot misidentify objects flying in the opposite direction. This illustrates our capability to control technology with a degree of autonomy, enabling effective application. Many times, I was grateful for such systems that provided necessary defense against incoming missile threats. It’s critical to recognize that there isn't an operator controlling these missiles at their final moments. Instead, we've developed extensive processes and technical safeguards to oversee systems with autonomous features.
The dystopian visions often depicted in popular media are worlds apart from such practical applications, even though some AI models have occasionally sparked discussions about nuclear strike recommendations.
That’s correct. While science fiction offers cautionary tales, it’s easy to get lost in those narratives. My experience as a naval officer taught me the value of systems designed for personal protection, not the creation of intimidating automatons.
Yet, it’s possible some may hold darker aspirations for military technology.
True, but military officers undergo rigorous training. No one steps directly into a command role; there’s comprehensive training around legal and operational protocols required before any engagement with weapons systems. Ultimately, a higher authority typically oversees the use of military force. Extensive vetting and education ensure that individuals operating these systems understand accountability and proportionality.
I hope your perspective holds true, though I do harbor some skepticism regarding these assertions.
Let's address that skepticism.
This administration has conducted strikes that raise legal concerns in locations like Venezuela and Iran, and some military leaders have resigned in opposition. Nonetheless, many continue to comply with orders. It’s important to acknowledge human imperfections in those dynamics.
Absolutely, acknowledging human fallibility is essential. However, these actions involve political decisions from leaders. Many situations are not clear-cut, making it complex for a military officer to recognize an unlawful order.
Returning to AI's current deployments in the military, reports indicated that Claude was involved in the operation against Maduro earlier this year. What was its role there?
I cannot specify details, as they are classified. However, large language models haven’t yet seen widespread deployment. The military is still learning to navigate appropriate applications. That said, a common task in the military involves daily reporting summaries. If an officer needs to compile a report, AI could help synthesize their findings—certainly not the most thrilling application, but practical nonetheless. Such tools assist in drafting emails, completing standard forms, and summarizing lengthy reports.
Palantir has utilized large language models for "foreign disclosure," which involves sharing limited information with allied nations’ militaries while remaining compliant with classified guidelines. This allows for the extraction of certain data without disclosing sensitive unit names, conducted with human oversight—a relatively non-threatening usage.
There were past reports suggesting that Israel employed an AI tool called Lavender, which identified potential Hamas targets, although likely with less discernment than a human leader would demonstrate. Is this a concern for U.S. military employment?
I followed those developments closely, and such systems gather data to create a coherent intelligence picture rather than generating a straightforward list of targets. The essential concern revolves around the rules governing engagement and what is deemed acceptable, not merely on the technical capabilities of AI.
According to a Wall Street Journal report from last week, concerns were raised regarding the Grok chatbot—used by the Pentagon—citing issues around its perceived bias and excessive flattery. Drawing from what you shared regarding summarizing communications, it seems a chatbot that leans toward sycophancy wouldn't pose much danger. Why the alarm?
There is indeed a threshold where a model's performance should be prioritized. If it focuses more on appeasing users rather than providing accurate feedback, it could be counterproductive. This could hinder military personnel’s ability to critically analyze intelligence reports. If an AI overly flatters during brainstorming sessions, it undermines the process. In military settings, personnel are typically intolerant of systems that provide disingenuous responses.
These complexities present a fascinating blend of technical and operational challenges currently facing the military and industry. It’s unsurprising that this topic has gained significant attention, albeit regrettably veering away from substantive issues and potential pathways forward.
This conversation has been summarized for brevity and focus.