Roman Yampolskiy: AI Can’t Be Controlled — and We’re Building It Anyway
6/15/20261 hr 22 min
Roman Yampolskiy has spent two decades trying to prove that superintelligent AI can be controlled. He couldn’t. I invited him on to make his case. Subscribe if you want science with evidence, not speculation. Roman is a professor of computer science at the University of Louisville and one of the earliest researchers in AI safety. His book AI: Unexplainable, Unpredictable, Uncontrollable started as an attempt to solve the alignment problem. After decades of work, it became a proof that the problem cannot be solved. Not difficult. Mathematically impossible. I push back hard. We go after the Einstein test: can a large language model trained only on pre-1911 physics reproduce what Einstein did with the same data? We ran that experiment. It failed. Roman and I disagree about what that means. We also get into the halting problem and what it actually tells us about predicting smarter-than-human behavior, whether value alignment is a real problem or a well-funded category error, the case for a government moratorium on frontier model development, and why Roman thinks giving an AI agent access to your computer is the dumbest thing a smart person can do. What you’ll hear: Whether AI control is mathematically impossible or just unsolved Why Roman thinks all current AI safety work is security theater What the halting problem actually means for superintelligence The alignment problem: real issue or well-funded category error Why Roman wants a moratorium on frontier model development What to tell your kids about careers in a world where Roman might be right If you listen to other people, the best you can become is average. CHAPTERS 00:00 Creating a mind without an off switch 01:34 Solving problems beyond our own intelligence 04:08 Einstein’s epiphany and the limit of AI intuition 08:18 Assessing the Einstein test: Why the experiment failed 12:22 Path dependency: Are LLMs and GPUs our QWERTY? 16:10 The barriers preventing AI from solving physics 21:54 Safety vs. Capability: Why toddlers are safe but teens are not 23:06 The halting problem: Predicting agents smarter than us 25:58 The impossibility of a system proving its own integrity 28:18 Regulation: Genuine safety or a gift to oligarchs? 33:28 Is human cognition non-computable? Penrose vs. the field 39:00 Ethical duties: Must we treat AI with humanity? 43:00 From internet memes to monsters: Decoding the book cover 46:22 Customized realities: Can everyone have their perfect world? 49:50 Von Neumann probes and the panspermia hypothesis 55:02 Categorizing AI: The one version that should terrify you 58:22 Pause AI: The movement for a development moratorium 59:58 Career advice for kids in a post-professional world 01:07:58 Cross-examining Sam Altman 01:15:48 Roman’s dream debate 01:19:50 Lessons for a younger self Substack: https://briankeating.substack.com Get the transcript, fascinating bonus content, and my Monday M.A.G.I.C. Message: https://briankeating.com/yt Have a .edu email and live in the USA? You automatically win a meteorite: https://BrianKeating.com/edu Subscribe: https://www.youtube.com/DrBrianKeating?sub_confirmation=1 Support Into the Impossible on Patreon, get my weekly M.A.G.I.C. Message, unfiltered bonus content, and live monthly Office Hours with me: https://www.patreon.com/drbriankeating Join this channel for perks, monthly Office Hours, and your name in the Member Roster at the end of every episode: https://www.youtube.com/channel/UCmXH_moPhfkqCk6S3b9RWuw/join Featured Guest: Roman Yampolskiy on Twitter/X: https://x.com/romanyam?lang=en AI: Unexplainable, Unpredictable, Uncontrollable: https://www.romanyampolskiy.com/books/ My books: Losing the Nobel Prize (memoir): http://amzn.to/2sa5UpA Think Like a Nobel Prize Winner: https://a.co/d/03ezQFu Focus Like a Nobel Prize Winner: https://a.co/d/hi50U9U Galileo’s Dialogue (first-ever audiobook): https://a.co/d/iZPi9Un Twitter/X: https://x.com/BrianKeating Substack: https://briankeating.substack.com Blog: https://briankeating.com/blog Audio-only: https://briankeating.com/podcast #intotheimpossible #briankeating #AIrisk #artificialintelligence #aisafety #podcast #superintelligence #RomanYampolskiy Learn more about your ad choices. Visit megaphone.fm/adchoices
Clips
Transcript preview
First 90 secondsSpeaker 00:00
A computer scientist who helped found the field of AI safety just told me we're building something we can never switch off, and that the smartest people in the room agree with him.
Roman Yampolskiy· Guest0:09
We're gonna have systems ten, hundred, thousand, million times smarter than us. What does that mean? They see patterns we don't see. You can have squirrels, monkeys, whatever you want. They are very intelligent beings, but they're not competitive with us. We're just on a different level, and it's exactly what we're gonna see here. Having an agent and giving it full access to your computer, your bank accounts, your email, sounds like the dumbest thing you can possibly do, and watching smart people do that really blows my mind. Well, we're asking for a pause in frontier model development contingent on someone solving control. If I'm right and control is unsolvable, that moratorium becomes a permanent ban. If I'm wrong, in ten years I'll get utopia, free stuff, and be very happy to be wrong.
Speaker 00:51
That's Roman Yampolskiy, a computer scientist who helped found AI safety and who now argues that can't ever be done. Not hard, provably impossible. Now, I'm used to going into the impossible, but today he's gonna get into why the math says so for the first time at this level of detail, and the one button he'd actually press if he had the opportunity.
Brian Keating· Host1:13
We can't bound the limits of knowledge, and part of controlling super AI... Again, I'm, I'm being devil's advocate here. I'm not, I'm not saying this is my position. I'm just saying this is what I think David Pasques, David Deutsch would say, is that because to control something you need to have knowledge of its future prediction, of