The Global Story · · Transcript & summary by WhipScribe
Summary, chapters and key takeaways for this episode are on their way; the transcript below is complete.
Our own transcript of the public episode, with a timestamp about every 90 seconds. Tap a timestamp or the link button on a paragraph to copy a link that opens this page there.
0:00This BBC podcast is supported by ads outside the UK. to score big from seasonal finds to favorites you'll use all year long. Plus with Wayfair, enjoy fast and easy shipping with delivery in time for the holidays. Save up to 70% off early Black Friday deals. Shop now at Wayfair.com. Wayfair, every style, every home. California, vacations are supposed to be the fun part, but stuff can happen. Travel Insured International has travel insurance that can help protect the money you've put into flights and hotels. And help when luggage is delayed, lost, or damaged. Or if you need to see a doctor on your trip. While planning a trip to the airport, check out TravelInsured.com. Don't wait until things go sideways.
0:58Protect yourself from travel surprises today with Travel Insured International. Terms and conditions apply. Shop plans at Travelinsured.com. On the Global Story, every day we look at the forces changing the world. And right now, it feels like AI is taking over our lives and taking over the news when very few of us were even talking about it five years ago. So we're introducing the global backstory, the new strand of our show where we'll explore the history behind the news today. You hear stories about AI all day, every day, but most of them are about where AI is going. This week, we're looking at where it came from. And to kick things off, today we are exploring a comparison and cautionary tale that's come up a lot.
1:47AI poses an existential risk. So too did nuclear weapons. AI should be treated as an existential threat on par with nuclear weapons and pandemics. It's as serious as that. It's like when nuclear came along, this is a thousand times bigger. Mark my words, AI is far more dangerous than nukes. Far. From the BBC, I'm Asma Khalid in Washington, D.C. And I'm Tristan Redman in Paris. And today on The Global Back run us all through what we have been cooking up this week? With pleasure, Asma. Yes, so we're going to find out about the Luddites, people who felt so threatened by technology in the 19th century that they took hammers to it. We're going to discover the writing that shaped how we think of AI from a famous British novelist who believes we have to embrace AI, not fear it. We're going to discover the writing that shaped how we think of AI from a famous British
2:45novelist who believes we have to embrace AI, not fear it. And we're going to debate how humans have assessed through the ages what it means for a machine to be intelligent. But we're starting with your conversation with Dan Carlin. We are indeed. And you know, when we first spoke to Dan in prepping for this interview, we were talking about how people these days worry that AI could be used for both good and bad purposes. There are fears, for example, that terrorists could use it to design a biological weapon. And Dan told us that back in the 1930s, scientists studying the atom were in a similar moment. In fact, Albert Einstein was one of the first to realize that nuclear science could be used both as a source of energy
3:25and as a weapon. And in 1939, when the world was on the brink of war, Einstein and other leading scientists sent a letter to the American president FDR telling him about it and warning that the Germans were onto it too. So these were all people intimately aware of where the science was and starting to watch world events coalesce in a way where they saw the science and the world events sort of linking up in a sense that they didn't want to see the side that they were hoping would win the war fall too far behind. So the letter was a way of telling the president of the United States, look, you might be able to use this technology to make super bombs. The other side is pretty far advanced in this.
4:07OK, let's just read a little bit of this, just so listeners have a sense of what they were saying. The letter begins, some recent work that has been communicated to them leads them to expect that the element uranium may be turned into a new and important source of energy in the immediate future. It goes on to say that this new phenomenon would lead to the construction of bombs, and it is conceivable, though much less certain, that extremely powerful bombs of this type may thus be constructed. It goes on to say that you may think it is desirable to have some permanent contact maintained between the administration and the group of physicists working on chain reactions in America. They advise you might want to speed up this experimental work, which is currently being carried out within the limits of the budgets of university labs.
5:02I mean, it sounds like it's a precursor to the Manhattan Project and government involvement of this all. And it was really interesting to see the ways in which this letter is advising the president, I think, both sort of about the reality of what is going on, but also calling for government investment at the time. To me, what's more important here is what was Einstein doing with these other scientists that led them to understand that maybe we need to write a letter to President Roosevelt? What we have going here shows you how much of a governmental and secretive effort this is, even using back channels from physicists to presidents. That, to me, is what's different when you talk about the analogs with AI.
5:44We have people in private rooms that are having conversations about AI that have stockholder and shareholder information they have to take into account. Can you imagine somebody suggesting that the United States has to invest in heavy water facilities? And oh, yes, let's put that on the stock market and get some capitalistic investors. Was it a sort of a factor of its time? I guess I'm wondering, right? I mean, this is all happening in the context of the world at war and the world felt a lot more uncertain. The difference is the profit motive. I understand at no point in this letter was thinking about how can we make money off this? At no point was Roosevelt reading the letter thinking, how can we make money off this?
6:18At no point were a bunch of people thinking, God, if we do this right, we'll get rich. I mean, this was purely a question of if they get it before we get it, we lose the war. The analog to look here would be to imagine AI developing as it has without any consumer benefits seen at this point at all. Meaning no chat GPT, no Claude. No, imagine it purely as a weapon. And then afterwards, something you see, oh, well, you know, there's a dual use aspect to this thing. But if it developed as a pure weapon or mainly as a weapon first, well, then you imagine that governments are going to classify everything. You know, I mean, we executed people in this country, Julius and Ethel Rosenberg, for giving nuclear secrets to another country. And they were executed for sharing that technology
7:06with the Soviet Union during the era of the Cold War, you're saying. Yeah. I mean, all you'd have to do today to give AI secrets to another country is just subscribe to the latest technical manuals in the industry, right? I mean, this is a secret weapon that Einstein is talking to FDR about. AI is a weapon, but it's not a secret weapon. And it was not developed principally as a weapon. So let's talk about some of the bigger thematic similarities and differences when nuclear technology was first being developed. Was the general public then really not aware of the power of nuclear technology until the atomic bomb was essentially dropped. I'm not sure they knew about it even afterwards. Really?
7:52If you look at the speech that Truman gives afterwards. A short time ago, an American airplane dropped one bomb on Hiroshima and destroyed its usefulness to the enemy. That bomb has more power than 20,000 tons of TNT. He has to try to explain to the general public what he's even talking about. With this bomb, we have now added a new and revolutionary increase in destruction to supplement the growing power of our armed forces. People didn't see any images. People didn't see any. They didn't hear news stories about it. No, it was still. Well, remember, this is not only was it secret before they were used in the Second World War, but it was instantly the highest level of classification afterwards because everybody
8:38understood after the war, OK, this is the new alpha weapon system. In their present form, these bombs are now in production and even more powerful forms are in development. It is an atomic bomb. It is a harnessing of the basic power of the universe. But even the effects of it, I mean, I remember reading in history books about the effects of Hiroshima and Nagasaki and just the horrific things it did to people's bodies, right, in terms of like burning skin. I mean, these were things we learned in our sort of many years after history books. You're saying the consequences of what that weapon could do on the ground was not known to people at the time? Well, it was, to be honest, it wasn't even known to the experts, because if you look
9:27at after the war, the first thing they start doing is sending experts to the sites where the weapons were used and running tests and taking surveys. I mean, they're trying to figure out what the weapons did to the effectiveness difference between an air burst and a ground burst. In other words, this sounds awful, but Hiroshima and Nagasaki are the only real world laboratory tests ever conducted on this stuff. And the people that dropped those bombs wanted to know what those bombs did. They're trying to figure out what they have here. So if the experts don't know, then the individuals, the people on the street don't know. So it sounds like you're saying the entire technology was so secretive that the average American had really no sense of what nuclear technology was capable of.
10:09No, not at all. OK, so the idea that it could be used in some fashion for good, right? Nuclear power, nuclear medicine uses like PET scans. That potential was not even discussed in public. It was never a part of public conversations in universities or in the press or anything like that. It was later. So, for example, there was a program called Atoms for Peace. And this arose after the Second World War, when the industries out there could start to see things, the dual use side of the technology. But we should emphasize that in 1939, when FDR gets this note from Einstein, nobody's thinking about PET scans and dual use. I mean, they have this overriding dynamic that we're talking about, which is the other side can get the bigger gun.
10:52And if they do, we lose the war. But from the scientist's standpoint, Einstein's letter is an attempt to rush the president of the United States. This was all stuff that would have happened organically over time. We were going to discover all this stuff. What they're basically saying is if the other side discovers this first, you're never going to have a chance to discover it. The side that gets it first will use it first. And then, you know, you won't be around to make these decisions. I mean, there does seem to be a bit of a parallel in that realm, right? I mean, we certainly hear perhaps more of an emphasis on the pros and cons of AI in the public sphere these days. America is the country that started the AI race.
11:32But you have also U.S. government officials, lawmakers discussing the idea that this is a race and that we need to get there first. And as president of the United States, I'm here today to declare that America is going to win it. That is sort of central, I feel, to what I've heard from a number of folks here in Washington, that they believe the United States needs to win the AI race. Otherwise, nothing else will matter. This is why history is useful, right? Because you can see this same dynamic over and over. But now they are no longer certain that America's lead will continue in the future. When Richard Nixon was running against JFK in 1960, Nixon was hammered by JFK over what was called the missile gap, that the administration, which Nixon was a part of, had allowed
12:21the Soviet Union to gain an edge on the United States and the number of missiles that it had available. When they see the missile gap widen and once our atomic monopoly begins to cease. And the missile gap was seen as something which could ensure that the country could be wiped out from a first strike from the other side before you could respond. They are uneasy about a military strategy that relies so heavily upon massive retaliation because they're not interested in seeing their house preserved only to see it blown up. That can't be allowed to happen because that's an existential threat. And that's what Trump is talking about. To fall behind another country in AI is the equivalent of having a missile gap.
13:05We must have an administration that will rebuild our military strength until America is once again first across the board. So this idea that if the other side gets too far ahead, militarily speaking, it will be something we can't make up. And then it's too late. So you can't lose your pace. I'm not sure this dynamic has not been ever present. You think it's always been present. Yeah. But here's the thing that taps into an almost DNA, though, survival, fight or flight instinct on a mass scale that takes away our agency. In other words, you can't sit back calmly and coolly and think, should we develop nuclear weapons or should we develop AI or not? Because you're not in a situation where it's an option.
13:53If the other side gets it, you're in a unique situation where you can't recover. And that creates that dynamic where we're not thinking about what's best for humanity. We just don't want to be lapped in a race where if you lose, you lose everything. I mean, the sort of existential nature of this race is something that we hear emphasized from the most powerful person in American politics. Our children will not live on a planet controlled by the algorithms of the adversaries advancing values and interests contrary to our own. I think that does echo down, right, to a variety of people that this feels existential. And the president has told us many times that in his view, it is existential. If we look at this from the side of the people who wrote that letter, right, if we look at this from the Einsteinian, for lack of a better word, perspective. Those people, philosophers from the time period and
14:45whatnot, were arguing that what nuclear weapons meant is that nation states can no longer operate the way that they have always operated. In other words, in a competitive sense, right, a militarily opposed sense, because all human beings have to do to destroy the planet is act the way we always have. And we always act that way because of competition between peoples, kingdoms, nation states, you know, however you want to frame it. The problem is, is that the AI follows the same tendency. If you wanted to see how a guy like Einstein, if he wanted to treat AI the way he wanted to treat atomic weapons, you would share it. And you would start to create international circumstances where war would go away. Because if you can't have war because the technology is too devastating,
15:33how do you prevent nation states from having war with each other? And this is where you get into things like the Russell Einstein memo, where they called for an end to this kind of thing, the sharing of technology. Give your enemies the technology. This is all stuff that sounds crazy to us today. But when you realize that what we're talking about here is an existential threat to humanity and that it's tough to imagine us not going there if the nation state competition that we've always had continues, well, then you can start to see why they were talking about how you just can't play the game. We've always played it with weapons and AI as a weapon that are this powerful. Big news! Wayfair's Black Friday sale is here, early.
16:20Right now at Wayfair, get up to 70% off everything you need for the season ahead. Save on kitchen and dining, bedroom furniture and mattresses, holiday decor, and way more. Now is the time to score big, from seasonal finds to favorites you'll use all year long. Plus, with Wayfair, enjoy fast and easy shipping with delivery in time for the holidays. Save up to 70% off early Black Friday deals. Shop now at Wayfair.com. Wayfair, every style, every home. California. Vacations are supposed to be the fun part, but stuff can happen. Travel Insured International has travel insurance that can help protect the money you've put into flights and hotels and help when luggage is delayed, lost, or damaged, or if you need to see a doctor on
17:02your trip. While planning a trip to the airport, check out TravelInsured.com. Don't wait until things go sideways. Protect yourself from travel surprises today with Travel Insured International. Terms and conditions apply. Shop plans at TravelInsured.com. This idea of the technology being so powerful and an existential threat to humanity, that does remind me of some of the language we've heard from some leaders in the tech world recently about AI. I mean, I'm thinking of someone like Dario Amadei, the co-founder of Anthropic, talking about the need for caution and to sort of slow down the development of the technology. I believe that this is the most important global security issue facing the world today.
17:49We saw, you know, Bill Gates recently, founder of Microsoft, say that this is the most dangerous thing that humans have ever gone near. We always said when we cross these thresholds, we will engage all of society because we will have created the most dangerous thing ever. This makes nuclear weapons look like nothing. We said we would engage and we crossed those thresholds and there was complete silence. And so we have heard this emphasis on the dangers of the technology from founders. And I think, I mean, we've talked about this a lot on her show, actually. Is that a sense of trying to boost attention as you're heading into, say, launching your company on the New York Stock Exchange? Right. I mean, launching your company out for
18:32public. Is it about garnering attention or is it about really trying to raise the alarm bells and call for government regulation? Listen, the people that know the most about this in the 30s when we're talking about nuclear weapons are physicists, right? Guys who have no stock in this. Whereas the people that are talking to you now about either how dangerous or how not dangerous this is have a huge financial interest and they may not even be able to say what they want to say because they might screw up the stock of their company. And I'm not making any value judgment on them. I'm just saying we don't usually do that with potentially existentially dangerous weapon systems. Unless the potentially existential dangerous technology
19:14was created at a private company first, right? And then how does the government retroactively come in and take it over? The Manhattan Project was an all-hands-on-deck government program, right? The nukes that were made, it was the government making them from the outset. Great point, because you see it today. If you allow private entities to go very far down the road to creating weapons systems, and we've got some of the best. The United States has fantastic, I mean, every company from Raytheon, I mean, we have a million of them, right? They're all publicly traded. And I think about Musk's stuff right now. I mean, when the United States government starts using-? They're all publicly traded. And I think about Musk's stuff right now.
19:46I mean, when the United States government starts using- You think about Starlink, are you talking about? Yes, when you start doing things that used to be stuff that NASA did, eventually there may come a time when the government says, this is something that's a national security important. We'll give you a bunch of money to walk away, but we can't let you dictate to us how something like this can be used. So it sounds like one really key difference is that AI is happening largely through private companies and nuclear technology was a government program. And that's a really big difference, you're saying, both just in terms of like the structure, but also in terms of the secrecy and what the public knows of how these programs are being
20:25built. Well, and there's another aspect that just occurred to me, which is the inability also to keep the cat from getting out of the bag on all kinds of basic information. I mean, like, if you were a small country in an out-of-the-way place in the 1950s, you couldn't build nuclear weapons. You couldn't use nuclear technology. You would have required help from the most technologically advanced countries in the world. Small entities can take advantage of artificial intelligence capabilities, and in part because private companies have made this available to not just nation states, but non-governmental groups, organizations that are involved in rebellions. I mean, when you look at who's using AI for what,
21:05well, already you see that governments aren't controlling it with the tight-fisted hands that they control nuclear weapons with. And there's a lot of reasons why that is. Why is that? Well, in one sense, because it's more difficult to do. I mean, if you had to, I mean, look at what Iran's trying to do if they're trying to enrich uranium and how hard of a process that is. Very different than dealing with, you know, zeros and ones and things that can be replicated through copy and pasting code. And you see what I'm saying? It's just a very different system in terms of the industrial capability required to be a player. You've mentioned the Rosenbergs a couple of times. This is the couple that was executed for treason
21:40in the 1950s. They were executed for sharing secrets about the United States nuclear program with the Soviet Union. Spies is what they were called. Yes, they were called spies. The reason I bring that up is that there have been a couple of stories in recent weeks about potential spying on various AI companies, chips companies. I mean, here, a Saudi national who was working at OpenAI was arrested for spying. And Dan, you and I are having this conversation just shortly after news came out about the CEO of NVIDIA, Jensen Wang. This is this major chips company. There was news that he had an affair with an alleged Chinese spy. And so it sounds like these national security concerns are akin to what you described were the concerns also around some other technologies like nuclear weapons.
22:32How do you think this environment is different, though, in how they navigate it? I feel that here in Washington, I keep hearing concerns from lawmakers about national security concerns with the technology that these private companies are making. Ethel and Julius Rosenberg were only two people. There was a spy ring that had infiltrated the highest levels of the Manhattan Project. Guys like Klaus Fuchs, all these people were actually feeding information back to the Soviet Union. And in fact, when the American president told the leader of the Soviet Union, Joseph Stalin, about the atomic bomb, he was kind of blasé about it. And it turns out because he probably already knew about it. But the difference between that is that there was an understanding at the highest levels of the Manhattan Project that this would be something that other countries would want, that this would be the most important information that you could keep from other countries.
23:23And that's why there's this huge cordon of secrecy around it. The problem with something like AI is try, I mean, you can see how it's developed organically in multiple locations all at once. In other words, this is a little like what Einstein's letter to Roosevelt was warning about. But instead of lots of countries developing this, you have not just countries, but partnerships between countries and private entities, private entities alone. This is a situation where I don't know how you try to throw a cordon of secrecy around something like this because of how it's developed organically. I mean, in the Manhattan Project, they had to live there, right? So the difference between trying to keep a lid on the most sensitive military secrets of that time and trying to keep a lid on the most sensitive military secrets of that time and trying to keep
24:05a lid on the most sensitive military secrets of our time, which is not just AI, but not to scare you, but imagine we combine those two elements. You combine artificial intelligence with nuclear weapons and start to see, you know, where we're looking at in the future. The interesting thing is, is this is like it's always been, but not. You know, there are elements which follow predictable human dynamics in terms of superpower competition and all this stuff, right? We can look back in history and see all kinds of analogs. But the element that represents X in this equation is nothing like we've ever seen. Because the technology you're saying is so different because it cannot be controlled. All the things we've talked about, right?
24:42The fact that it is like something like nuclear weapons, but it is not like something like nuclear weapons. So all these things come together to make us able to maybe learn a little from the past, but not really. I mean, it's frustrating, right? You want to be able to say, hey, we learned from the 1950s what to do and what not to do, but it's just different enough so that the prescriptions that we might have learned aren't really applicable in this case. You know, nuclear weapons have not been used in 81 years, not since 1945. And despite the arms race, despite the Cold War, it seems that as a global society, we have figured out frameworks for what nonproliferation agreements, weapons inspections, nuclear deterrence.
25:30There is this assumption that countries that have nuclear weapons don't go to war with one another. We sort of, for the moment, at least seem to have figured that dynamic out. And I bring that up because nuclear technology over the last eight decades has also expanded and provided so many scientific breakthroughs that have benefited many of us. Is that where AI could be heading? It feels like we keep hearing these sort of doomsday scenarios about the existential threat to humanity, but there's a version of this AI story in which we as a global society figure this out and are able to live peacefully and benefit from this new technology. You know, when you said that, something occurred to me that was a little bit
26:12different. I apologize for taking it. But when you said we haven't used nuclear weapons in 81 years, well, we have. We've used hundreds and hundreds, maybe thousands of them since the Second World War. We've tested them all over the world. We've shot them into space. We've done airbursts. We've done underground bursts. We've tested. You can go to the Nevada test site now and the sand looks like glass and you have to be escorted in there and you have to wear gloves and the Geiger counters are everywhere. And this is decades and decades and decades afterwards. And so when you talk about AI, I keep thinking, where are the AI equivalents of the nuclear tests? I mean, I think you have a nuclear test, so you don't have a nuclear reality.
26:52Where are the learning experiences we're going to have with AI? Because a learning experience is coming close to the edge without falling off. You know, there was a wonderful question when the Manhattan Project was first tested about what would happen when a nuclear, an atomic weapon exploded. Would the sky catch on fire? What's the old line about the Hadron Super Collider? There was a small chance, you know, everything could be destroyed. Well, the question then when you get each new technology like AI is when you have that moment, what happens? And I feel like some people think we've already had that moment, but you hear it all the time when they talk about the super intelligent breakthrough, when AI can start to make itself better and then look at itself
27:38again with its new better self and make itself better yet again. And all that can happen in the blink of an eye. When that happens, that's our moment of test here, right? That's does the sky catch on fire when the bomb drops moment. And I think that's what's in our future. That's one of those data points that when you say, you know, when the history books write about it, that's going to be the moment where it's a pass or fail test for humanity, maybe. To me, that's the analogy with nuclear weapons. Well, on that note, Dan, I just want to say thank you for joining us. I really appreciate it. I appreciate you having me. Thank you so much. That was Dan Carlin, host of the podcast Hardcore History and author of the book The End Is Always Near, Apocalyptic Moments from the Bronze Age Collapse to Nuclear Near Misses.
28:33This episode kicked off our special series called The Global Backstory. All this week, we've got episodes that explain how we got here to this moment in AI. If you all have any thoughts or questions, we'd love to hear from you. You can get in touch by sending us a note or even a voice memo. Our email address is theglobalstoryatbbc.com. And before we wrap, a credit to the team who made today's show. Our episode was produced by Lucy Paul. It was edited by James Shield and mixed by Travis Evans. Our digital producer was Charlie Firth. Our senior news editor is Chyna Collins. I'm Asma Khalid and my co-host is Tristan Redman. Thanks as always for spending some time with us and we'll talk to you again tomorrow.
29:45Heroes don't always wear capes. We'll be right back. class. At Cal State East Bay, change your future and what's possible for everyone who comes after you. Cal State East Bay. Becoming a pioneer starts here. Apply or learn more at csueastbay.edu slash start.