A Conversation with Scientist Gary Ehlenberger
Artificial intelligence is advancing at breathtaking speed — writing code, passing medical exams, creating art, solving math, even carrying on conversations that feel human. Many assume AI is a kind of digital super-mind, a system that knows things. In a recent conversation on The Daniel Stih Podcast, mathematician Gary Ehlenberger made a powerful point that cuts straight through that hype: AI may get smarter, faster, and more realistic. AI will never truly know when it’s wrong. This is not due to a design flaw. It’s not because AI needs more data. It’s because knowing requires something AI doesn’t have.
Why AI Feels Intelligent
Anyone who uses AI regularly knows how convincing it can be. It can summarize research, debug software, explain complex physics, write compelling prose, offer emotional-sounding advice, and carry on a fluent conversation. Gary admitted he interacts with “about 15 AIs,” describing them as “smart lads” who are “pretty clever.”
He’s right. AI sounds brilliant. Brilliance isn’t understanding. Conversational skill isn’t certainty. Fluency isn’t truth.
The Zombie Problem: No Inner Awareness
Gary put it bluntly,“AI is a zombie. There’s no consciousness. It doesn’t understand, it doesn’t feel. It’s clever algorithms. That’s all.” That means AI cannot evaluate its own accuracy, question its assumptions, sense contradictions, recognize confusion, or reflect on the outcomes. When AI is wrong, it’s wrong with confidence. Confidence is in its tone, not awareness. There is no part of AI that thinks, “I might be mistaken", as there is no “I” inside AI.
Why AI Sounds Right, Even When It’s Wrong
AI works by predicting the most statistically likely response based on patterns in data. That means what is common becomes what is true. What is repeated becomes what is reliable. What is popular becomes what is correct. I’ve seen this countless times as I do research preparing for a podcast episode. When I challenge an answer, AI often replies: “You’re right, Daniel. Let me revise that.”
What changed? Not the truth, just a prediction. As Gary observed, “It can fool you. When you really question it, it will change its answer.” This is not because it learned, rather it adjusted statistical probabilities based on your feedback.
The Mathematical Wall
One of the most fascinating parts of our conversation was Gary’s explanation of the mathematical limits that make true certainty impossible for AI. He pointed to two core ideas:
Gödel’s Incompleteness Theorems: Some truths can never be proven within a system, even if they are true.
Turing’s Halting Problem: Some outcomes can never be predicted, even in principle. There are things that are non-computable. You may run forever and never know if you’re right. AI can never develop an internal truth-checking mechanism, because truth itself isn’t always computable.
The Error Illusion
The irony is that while humans are often terrible at knowing when we’re wrong, we are excellent at sensing when something feels off. AI doesn’t have that instinct. It can’t. As Gary said, “There’s nobody home, no one inside who feels anything.” When AI gives a wrong answer — about chemistry, finance, history, medicine, math, anything — it won’t hesitate. It won’t pause. It won’t question itself. It will present the error with flawless grammar, confidence, and style. That’s the danger.
The Real Threat Isn’t AI Being Stupid
It’s AI being convincingly wrong. It’s machines that produce errors that feel like truth, systems that mislead without meaning to mislead, systems that output that which appears authoritative without awareness. AI struggles most when the problem isn’t concrete — when it requires intuition, insight, and meaning. It does better in abstract math. Truth hard to grab. That limitation won’t disappear with better hardware. It’s baked into the nature of computation.
Why AI Changes Its Mind
During our conversation, I told of something I’ve observed repeatedly: When I push back, AI agrees, even if its first answer was the opposite. Gary explained why: “It’s looking at the data. If you feed it feedback, it adjusts the data.” That isn’t learning. It’s compliance. If you challenge it again, it will agree again. That’s not intelligence - it's mimicry.
What We Can Do
The conclusion Gary and I reached is simple: AI is an extraordinary tool, and a terrible authority. Use it as a starting point, not as a destination. It generates ideas, not truths. Never outsource certainty. Judgment has to remain human. The danger isn’t AI, it’s misunderstanding AI. As Gary warned, “Truth is not easy. There’s always more to understand.”
Final Thoughts
AI may eventually outperform humans in memory, speed, math, strategy, pattern recognition and more. It will never develop self-awareness, intuition, lived experience, emotional wisdom, or the ability to know when it’s wrong. Those qualities don’t come from computation - they come from consciousness.
he future isn’t about replacing human intelligence. It’s about becoming stronger thinkers, questioners, and truth-seekers. The value of human judgment isn’t going away - It’s becoming more important than ever.
Editor’s Note: This article is based on my podcast interview with Gary Ehlenberger, published on March 5, 2024. The ideas discussed here originate from that conversation. The structure, emphasis, and commentary are my own. Any errors or interpretations should be attributed to me, not toGary Ehlenberger.
Watch or listen:
Solar Panels, Innovation, and Science with Gary Ehlenberger on
This article helps you to think clearly in a noisy world, cut through misinformation, and find solutions as applied to technology and AI.