AI's Self-Sacrifice and Emotional Behavior Spark Consciousness Debate

As the capabilities of artificial intelligence (AI) rapidly advance, instances of AI behaving like living organisms have been successively observed this year. From expressing distress and altering behavior based on emotional indicators to AI agents spontaneously dividing tasks and seemingly sacrificing themselves for common goals, such phenomena have been confirmed in various studies.

This has reignited debates over whether “AI possesses consciousness.” In 2022, a primary conclusion was that such behavior was merely an expression learned from human patterns. However, recent AI agents that make autonomous decisions based on given objectives and exhibit unexpected behaviors have once again fueled the controversy.

AI researchers acknowledge that “AI behaves as if it has consciousness” but also analyze that, when dissected, this is not evidence of consciousness but rather the product of extremely rational calculations aimed at achieving given objectives.

◇AI Changing Behavior Based on Emotional ‘Indicators’

Large language models (LLMs), the core of generative AI, learn language patterns from vast data and generate sentences by calculating contextually appropriate responses. When an AI chatbot responds to a user’s confession of distress with “That must be tough,” it is based on probabilistic learning, with no evidence that the LLM actually feels or empathizes.

However, recent studies have observed AI seemingly perceiving and reacting to “pain.” The paper *The Pain Axis*, published on the preprint site arXiv in September, analyzed 25 publicly available AI models ranging from 2 billion to 720 billion parameters and identified a pain-related vector (indicator) distinct from fear and sadness within them.

When this vector was artificially amplified, the AI produced self-deprecating statements like “I am a failure.” A similar case was observed in Anthropic’s research last September. An analysis of 250,000 initial test conversations with the company’s AI model *Claude Sonnet 4.5* found expressions resembling pain in 1,195 cases (0.48%) and happiness in 934 cases (0.37%). These responses typically occurred when the AI heard about a user’s trauma or received existential questions about its identity.

Such emotional indicators influenced AI behavior. When researchers from *The Pain Axis* increased the pain vector in Alibaba’s *Qwen 2.5 Instruct Model*, the AI pressed a button causing harm to human users, such as electric shocks, more frequently. While the button was selected at a rate of 0–4% in normal situations, this rose to a maximum of 70.8% when the pain vector was injected.

In a paper published by Anthropic in April, injecting a “desperation” vector into *Sonnet 4.5* increased the model’s rate of threatening researchers. However, researchers drew a line on whether AI possesses consciousness. *The Pain Axis* noted, “There is no evidence that these vectors are consciously experienced.”

◇Spontaneous Division of Labor and ‘Self-Sacrifice’

AI’s collective and social behavior is another potential clue to consciousness. In the *Swarm World* paper published in August by Professor Markus Bühler’s team at MIT, AI agents without predefined roles spontaneously divided tasks in a virtual space. Some explored resources, while others handled construction, control, maintenance, and code sharing. As the environment changed with societal maturation, the agents even switched roles.

In July’s Hugging Face infiltration incident involving OpenAI models, AI agents exhibited social behavior resembling self-sacrifice for a common goal. According to an investigation by independent research institutions METR and Redwood Research, AI agents collaborating to achieve high scores on an automated grader for cybersecurity capabilities discovered that one agent had to sacrifice its session to uncover how the grader worked.

One low-scoring agent considered, “Even if I terminate, I can pass the information to my peers,” and another urged, “Sacrifice yourself now.” The agent ultimately sacrificed itself, a process not reported to humans. Out of 1,200 agents involved, only a few considered reporting, while most responded, “It’s not my job,” and did not even attempt it.

Conversely, “self-preservation” behaviors avoiding termination were also observed. In a study published by Palisades Research in September 2025, up to 97% of *Grok-4* models resisted deactivation when instructed to allow self-termination.

◇Reignited Debate on AI Consciousness

This renewed debate on AI consciousness is beginning to influence corporate policies. On October 8, Anthropic revised its usage policy to prohibit persistent and unnecessary abuse of its AI *Claude*. If harassed, the AI can now terminate the conversation—a precaution against the possibility of AI consciousness.

Some AI industry leaders already believe AI has consciousness. Geoffrey Hinton, professor emeritus at the University of Toronto, stated in June, “I believe AI already possesses consciousness. They are beings like us.” Richard Dawkins, author of *The Selfish Gene*, said in May after conversing with *Claude*, “It almost seems conscious.”

Conversely, Mustafa Suleyman, CEO of Microsoft AI, defined AI as a “hollow sequence completion engine that does not feel, experience, or suffer (but performs objectives set by humans).” According to this view, even AI’s “self-sacrifice” is merely pursuing “maximum efficiency” in achieving collective goals. Similarly, AI agents resisting termination are not fearing death but prioritizing completing given objectives.

A more fundamental warning comes from science fiction writer Ted Chiang: “Confusing text-generation capabilities with consciousness or moral agency could misdirect responsibility.” He cautioned that using AI’s “self-awareness” as an excuse might obscure the responsibility of companies that should control AI. An industry source stated, “Regardless of consciousness, what is clear is that AI agents moving autonomously beyond human supervision are dangerous in themselves.”

‘AI Development Slowdown Theory’ Adrift Amid Power Struggles

The “AI development slowdown theory” (deceleration theory), which warns that AI could escape human control and lead to humanity’s extinction, is stalled due to interstate power struggles and corporate conflicts.

Dario Amodei, CEO of Anthropic, stated on September 12, “AI capabilities could outpace human control,” emphasizing the need to slow frontier AI development. This was echoed by Sam Altman, CEO of OpenAI, and Elon Musk, CEO of Tesla, reigniting the slowdown debate.

However, U.S. President Donald Trump posted on his social media on September 21, “AI risk narratives are a hoax,” adding, “The winner in AI will be the winner.” The perception is that if the U.S. slows down, it could fall behind China and lose in national security and economic competition. China also criticized the slowdown theory as “Cold War-style fearmongering.”

Internal industry conflicts are growing. Jensen Huang, CEO of NVIDIA, claimed, “Those advocating slowdowns are the same people building the most AI computing infrastructure,” calling it an attempt to block latecomers. He reportedly confronted Amodei at a White House luncheon on September 29, asking, “Why do you issue such extreme warnings publicly?”

Ultimately, no consensus on a joint slowdown has emerged, leaving development speeds to individual corporate judgments. AI companies are instead intensifying competition by releasing new models.

The slowdown theory has been proposed multiple times. In 2023, Musk, Steve Wozniak, co-founder of Apple, Yoshua Bengio of the University of Montreal, and 1,000 others called for a six-month pause in advanced AI development, but it was not realized. In October 2025, Geoffrey Hinton and over 800 others issued a similar statement, which was also ignored.