Alan Turing's biggest AI assumption may have been wrong

In his new book, Turing's Mistake: Escaping the Yoke of Unintelligent Machines, Denning argues that two foundational assumptions made by Turing in 1950 continue to shape AI research today. The first is that intelligence can exist independently of a physical body and therefore be recreated in computer software. The second is that a machine can demonstrate intelligence by successfully imitating a human in conversation, an idea that later became known as the Turing test.

"These two claims have shaped much of AI research and development," Denning writes. "My premise is that our acquiescence to these claims has led to the AI mess in which we find ourselves today."

Denning argues that pursuing artificial general intelligence (AGI), or machines with human level intelligence, is unlikely to succeed. Instead, he warns, the technologies society is building could introduce significant new risks.

The Tacit Knowledge Problem

At the heart of Denning's argument is the idea of tacit knowledge, the vast amount of human understanding that cannot easily be put into words or represented in a form that computers can process.

He says machine learning cannot capture five major categories of tacit knowledge: common sense, everyday interactions with people and the environment, emotions and perception, practical performance skills, and the social and historical knowledge embedded in culture.

Researchers have long attempted to organize common sense into databases. One of the best known efforts was Douglas Lenat's Cyc project, which began in the 1980s with the goal of creating an extensive collection of common sense facts. After four decades of work, the project contained roughly 25 million entries.

"Yet even this treasury could not add up to a background of common sense sufficient to make expert systems smart enough to be experts," Denning notes. "Cyc validated that much of the knowledge that makes people experts cannot be articulated as propositions."

Denning believes practical skills present an even greater challenge.

"Our performance skills in thousands of domains cannot be communicated to machines," he explains. "Whereas descriptions of skillful outcomes ('know what') can often be represented as bits and stored in a machine, we do not know how to encode the embodied knowledge for skillful performance ('know how')."

He points to accomplished musicians as an example.

"A virtuoso violinist can play beautiful music yet cannot describe to an acolyte how to produce it.

"Even if a robot could observe and imitate skilled humans, having no biological body, a robot cannot grasp how the musician feels when playing beautiful music or how an audience feels when hearing it."

Denning also includes intuition, gut feelings, imagination, and spontaneous creativity among the forms of tacit knowledge that remain beyond the reach of machines.

Why Human Knowledge Resists Encoding

Denning argues that all of these limitations stem from what he calls the "representation problem."

Computers can only perform calculations using data and instructions that have been encoded into physical forms they can recognize and process. Tacit knowledge, however, does not naturally fit into that framework.

"Behind every word is a deep well of tacit knowledge that gives it meaning," Denning says. "Words are but symbolic representations of meanings, not the meanings themselves. Commonly used Large Language Models, such as ChatGPT, Claude and Gemini only manipulate words, they cannot know or understand the meaning of what they are saying."

According to Denning, this creates a fundamental divide. Because scientists still cannot fully explain how tacit knowledge works in humans, they also cannot translate it into a form machines can use.

"How we host tacit knowledge is largely a mystery," Denning admits. "All we know is that it is embodied. We have no idea what we might observe and measure in our bodies to reveal it."

Context and Culture Shape Intelligence

Denning also argues that intelligence depends heavily on context, the surrounding circumstances that give words, actions, and decisions their meaning.

Context allows people to recognize sarcasm, humor, sincerity, and emotion. It helps determine when to be diplomatic, when to joke, and how to interpret countless social cues.

"When you inquire into where an assumption of the current context came from, you discover it rests on previous conversations from previous contexts. Each of those in turn rests on further previous conversations and their contexts. This pattern is endless and fractal," Denning explains.

Culture presents another major obstacle for AI.

Denning describes culture as encompassing values, norms, judgments, history, communities, moods, and even relationships involving power and care.

"Human conversations are imbued with background assumptions that give meaning and relevance to the words being used," Denning explains.

"Scaling up LLMs with ever larger neural networks will not enable them to acquire the embodied human knowledge we call culture. LLMs will not attain the objective of the Turing test: to demonstrate machine thought indistinguishable from human thought."

AI Safety and the Limits of Human Understanding

Denning concludes that humans and AI systems may ultimately develop different forms of tacit knowledge that neither can fully understand.

"Machines cannot read our tacit knowledge and we cannot read theirs," he writes. "We are aliens across an uncrossable divide."

He argues this gap raises serious concerns about AI safety. If machines cannot interpret the unspoken context behind human intentions, reliably aligning advanced AI systems with human goals may prove impossible.

"Through AI automation, agentic networks of machines are likely to develop their own machine intelligence that does not reach the level of human general intelligence but is still quite capable of creating severe problems for humans. This threat is a greater than a take-over by superintelligent machines," he explains.

"Machine intelligence has different concerns from us and does not appear to care about us. Its ways of thinking and problem-solving look alien to us. We do not yet know how to live safely with these machines.

"Pulling back from an AI automation singularity will demand much from us. We start by accepting that the familiar culture is fading away as intelligent machines appear in our society and we do not know what is coming. We decline to think like machines or be subservient to machines. We refuse to submit to a yoke imposed by low-intelligence machines. Most importantly, we reassert our humanity, declare once again what makes us different from machines, and celebrate those differences."