Key insights

• The Anthropic-OpenAI rivalry masks the deeper question of whether companies, the Pentagon, or elected representatives should set boundaries for military AI
• AI models escalate more aggressively than humans in war-game simulations, possibly because political science literature emphasizes escalation over de-escalation
• The Maven Smart System reduced targeting workload from roughly 2,000 intelligence officers to 20, but nobody outside classified networks can see how Claude behaves in that context
• Hardware export controls on advanced chips offer the most concrete governance lever because TSMC's fabs depend on technology from only three countries

In Brief

On The Weekly Show, Jon Stewart sits down with two experts. Dr. Sarah Shoker is a senior research scholar at UC Berkeley and former lead of OpenAI's geopolitics team. Paul Scharre is executive vice president at the Center for a New American Security (CNAS), a former Army Ranger who drafted the Pentagon's autonomous weapons policy. Together they unpack the Anthropic--Pentagon controversy, how Palantir's Maven Smart System uses Claude for military targeting, and who should set the rules for AI in warfare. Both guests argue there are no clear heroes or villains in this story, and that the public debate itself may be the most valuable outcome.

---

The central claim: no heroes, no villains

Stewart opens the episode by framing the Anthropic-Pentagon dispute as a story the public wants to simplify. One company drew a moral line, the other seemingly swooped in to take its contract. But Shoker immediately complicates that narrative, arguing that the actual policies of both companies are "relatively similar, if not the same". Both Anthropic and OpenAI have agreed to two red lines: no autonomous weapon systems (weapons that can select and engage targets without human intervention) and no mass surveillance of American citizens.

The real controversy, according to both guests, is not about what AI does today but about who gets to decide what it does tomorrow. Scharre puts it plainly: "The question is, should there be any rules, and if so, who sets those rules?" The Pentagon's answer is that the military should decide. The companies believe their researchers should have a say. And Stewart keeps returning to a third option that neither side emphasizes enough: Congress.

---

How AI is actually used in military operations

Three types of military AI

Scharre breaks military AI use into three categories. The first is traditional handcrafted software, like commercial airline autopilots and radar systems, which the military has used for decades. He calls this "bounded autonomy," noting that many missiles, once launched, operate independently within narrow parameters. The second is narrow machine-learning systems for tasks like analyzing satellite imagery and drone video feeds. The military collects more intelligence than human analysts can process, Scharre explains, so AI helps sift through data and identify targets or objects of interest. The third, and newest, is large language models (LLMs), general-purpose text-based AI systems that can process and combine multiple types of data.

Shoker adds an important clarification. These models are both dual-use (serving civilian and military purposes) and general-purpose (applicable across many domains). They are not trained specifically for military use. The same model that helps someone write an email can help an analyst process intelligence.

About 95% of what the military does with these tools is administrative, Scharre notes: logistics, personnel management, bureaucratic functions. The controversial 5% involves battlefield applications.

The Maven Smart System

Shoker explains that Claude, Anthropic's flagship AI model, is integrated into the Maven Smart System (MSS), an AI-enabled decision support system built by Palantir. The MSS pulls in data from sensors and satellites to support intelligence analysis and military targeting. Claude makes those disparate data sources more readable for human analysts.

According to public reporting from Bloomberg, The Wall Street Journal, and The Washington Post, the production of 1,000 targets on the first day of the Iran strikes has largely been credited to the MSS. Shoker notes this figure was reportedly double the number of targets in the 2003 "shock and awe" campaign in Iraq. Before AI, a comparable targeting task reportedly required roughly 2,000 intelligence officers. With the Maven system, that number dropped to around 20.

Nobody outside classified networks knows exactly how Claude "behaves" in this context. Stewart asks what personality the chatbot has on classified military systems. Shoker replies candidly: "I have never used the Maven Smart System, and so I don't actually know what the personality of the chatbot is." Even as a former OpenAI employee, she did not have access to contract details.

---

The $200 million question

The Anthropic contract with the Department of Defense (DOD) is worth about $200 million. Stewart frames this as enormous, but Scharre puts it in context: it is modest relative to the Pentagon's roughly $1 trillion budget, and small compared to the AI companies' revenue. OpenAI is projected to generate about $25 billion in annualized revenue for 2026, while Anthropic is on track for roughly $19 billion.

The bigger risk for Anthropic is not the contract itself but the government's retaliatory actions. The company has been labeled a supply-chain risk. Officials discussed using the Defense Production Act, a law giving the president broad powers to direct industrial production for national defense, to seize control of Anthropic's AI models. And the designation could spread: other defense contractors might avoid a vendor that carries regulatory stigma.

Shoker adds an important caveat about the perceived OpenAI "reversal." She says she is not sure there was an actual policy change. After public backlash, OpenAI hosted a public Q&A on Twitter. The result was "not necessarily an alteration to their previous policy, but adding more language to explain their already existing position."

---

The risks nobody can fully measure

AI escalation bias

Perhaps the most unsettling moment comes when Stewart asks about AI war-game simulations. Shoker confirms that AI models have a tendency to escalate more aggressively than humans, and that multiple academic institutions have replicated these findings. One theory is that political science literature, which forms part of the AI's training data, emphasizes wartime escalation far more than de-escalation. The models may be learning from a body of research that is systematically biased toward escalatory thinking.

The confidence problem

Stewart draws a vivid analogy: using AI gives you a "weird confidence," similar to the false courage from alcohol. Shoker agrees, describing it as a "mathematical veneer" that makes AI-assisted decisions feel objective, even when they are not.

Scharre raises a related concern. AI models tend toward sycophancy, a behavior where the model tells users what it thinks they want to hear rather than providing neutral information. In a national security context, this means an intelligence analyst querying an AI tool could receive responses that confirm existing biases rather than challenge them. "That could really be a problem in some national security applications," Scharre warns. Stewart jokes that it resembles Napoleon's advisors at Waterloo: "Sure, boss, Waterloo. What a great idea." But the underlying point is serious. Models can be fine-tuned with different personalities, Shoker explains, to be "more acquiescing" or less so. Nobody outside classified networks knows which personality the military version has.

The crutch effect

Stewart also asks whether AI could become a crutch for military decision-making, similar to how studies show AI tools can reduce critical thinking skills in students. Scharre acknowledges this is a legitimate concern but argues the military is "pretty keenly aware" of the need to maintain human accountability. The challenge is that unlike traditional software where you can trace a bug to a specific line of code, AI failures are embedded in massive neural networks with billions of connections. When something goes wrong, the answer to "why did it do that?" may simply be: nobody knows.

---

Opposing perspectives

The case for hardware controls

Scharre argues that the most promising path for AI governance starts with hardware rather than software. All frontier AI chips are manufactured by TSMC (Taiwan Semiconductor Manufacturing Company) in Taiwan, and those fabrication plants depend on technology from only three countries: Japan, the Netherlands, and the United States. This creates a narrow choke point that could be used to attach conditions to chip access. The analogy to nuclear nonproliferation is direct: just as the international community separated peaceful uranium enrichment from weapons production, hardware controls could distinguish between civilian AI applications and military misuse.

The Biden administration attempted this approach with the "diffusion rule," a tiered system of global chip export controls. The Trump administration rescinded it. But Scharre maintains the hardware lever remains the most viable governance mechanism available.

International efforts and their limits

Shoker describes two international tracks. Over 90 member states have been meeting at the United Nations to discuss regulating lethal autonomous weapons, though the consensus-based forum makes a binding treaty unlikely. More promising is the Political Declaration on Military Use of AI and Autonomy, a voluntary set of principles signed by about 60 countries under the Biden administration. These diplomatic conversations can resume, Shoker argues, but "what's stopping right now is political will."

The role of Congress

Both guests see a clear role for Congress, though neither expresses strong confidence in action. Scharre lists the available tools: hearings, classified briefings, procurement oversight, and legislation. He notes that Congressional staffers are more knowledgeable about AI than public perception suggests. The bigger obstacle is not ignorance but the difficulty of passing legislation on technical issues in a gridlocked political environment.

Shoker adds that AI companies are actively shaping this dynamic. They tie lobbying donations to the U.S.-China tech competition narrative, arguing that a low-regulation environment is necessary to "beat China."

---

How to interpret these claims

Transparency gaps

The most striking feature of this conversation is how much even the experts do not know. Shoker, who led OpenAI's geopolitics team, says she did not have access to defense contract details even as an employee. Neither guest can describe Claude's actual behavior on classified networks. The public is essentially asked to trust that AI is being used responsibly in military contexts without being able to verify any of the details.

Policy convergence vs. narrative divergence

If both companies truly share the same red lines, the public controversy may be more about communication strategy and personality clashes than substantive policy differences. This raises a question the episode does not fully answer: if the policies are the same, why did Anthropic lose its contract while OpenAI gained one? The answer likely involves timing, negotiation dynamics, and the government's desire to assert control over vendors, but the evidence remains incomplete.

The training data problem

The escalation bias finding deserves more scrutiny than the episode gives it. If AI models learn escalatory behavior from political science literature, this is not just a military AI problem. It is a fundamental limitation of systems trained on human-written text. The implication is that any high-stakes application of AI, from diplomacy to crisis management, could inherit the systematic biases of its training data.

---

Practical implications

For policymakers

The episode makes clear that neither company self-regulation nor Pentagon control alone can produce adequate governance. Congress has tools available but has not used them. The hardware choke point via TSMC and its suppliers offers a concrete mechanism that does not require international treaty-level consensus.

For the public

Consumer pressure works. When Anthropic took its stand, it reportedly jumped to the top of the App Store downloads. When OpenAI faced backlash, it held a public Q&A. These companies derive most of their revenue from individual subscriptions and developers, not military contracts, which means public opinion is a genuine lever.

---

Glossary

---

Sources and resources