Are We About to Lose Control of AI? | AI Reality Check

Cal Newport· Host0:00

Last week, Anthropic released a report with a scary-sounding title, When AI Builds Itself, and it came accompanied by a scary animation that shows machines replicating themselves exponentially, like cells in a Petri dish. Now, the body of the report itself keeps these dark vibes going. I wanna read you some actual quotes here from the intro to the report. They say, "For most of AI's history, humans drove every step in its development cycle. But at Anthropic, we are delegating a growing share of AI development to AI systems themselves, which is speeding up our work. Taken far enough and given enough compute, this trend points to an AI system capable of fully autonomously designing and developing its own successor. This is called recursive self-improvement. We are not there yet, and recursive self-improvement is not inevitable, but it could come sooner than most institutions are prepared for." A little bit later, they then add, "AI that can build itself would be a major development in the history of technology, one that could bring enormous good for the world in science, healthcare, and beyond. But full recursive self-improvement also might increase the risks of human- humans losing control over AI systems." Now, if you look at the, the headlines generated in response to this report, most of them focused on a section of the report that, uh, seemed to call for a worldwide pause