Moonshots with Peter Diamandis · · Transcript & summary by WhipScribe
A live panel weighs whether NVIDIA's CEO was set up, how fast AI is moving, and what new AI tools mean for Microsoft Office and science.
Tap a chapter to play it from that moment in the show's own player; the transcript opens there too.
“i don't really buy it at all like i have a high p doom” Imad · at 41:58 — ▶ hear it from here
“I think it's a very dangerous way to get to full AGI” Demis Hassabis · at 58:43 — ▶ hear it from here
The episode centres on the sudden exit of Jensen as NVIDIA's CEO. A New York Times report on an affair, paired with claims that U.S. intelligence had been watching a Singapore executive, put the company's chip sales to China under scrutiny. The hosts see the timing as suspicious: the reporting came just after the midterm elections and days before a Trump visit to China. Alex links the episode to the Remote Access Security Act, which would make it harder for China to reach GPUs through cloud offerings. Salim and Imad doubt the official account, and the panel agrees NVIDIA's future depends on finding a leader who can match Jensen's deal-making and his TSMC relationships.
The second theme is how quickly AI is moving. The hosts react to Paul, an OpenAI foundation board member, who warned of a high risk of an AI takeover within three years. Imad says his own doom estimate has fallen from 50 to 20 percent, while Alex argues that alignment work mostly amplifies capabilities. Demis Hassabis, founder of Google DeepMind, warns that closed-loop recursive self-improvement, where humans leave the review loop, is the riskiest step, and the group debates whether labs can stop it.
The rest of the show covers software, science and space. Anthropic's new Claude Sheets and Whiteboard tools put pressure on Microsoft Office, which one host estimates brings in about $90 billion of Microsoft's $330 billion in yearly revenue. A Planet AI app lets users search satellite archives, an AI researcher's agents flagged a possible unmapped Mayan site in Mexico, and a dwarf planet may have turned up in Vera Rubin data. The hosts argue that archaeology, astronomy and biology are all about to be transformed by AI.
Demis Hassabis said current recursive self-improvement makes human engineers three to five times more productive while people still review and architect code. He called the next phase, closed-loop RSI with no human reviewer, the riskiest step, because the result may be an alien intelligence that is hard for people to understand or control.
“I think it's a very dangerous way to get to full AGI”Demis Hassabis · 58:43
Imad said the board's rapid move against Jensen looked strange and suggested a setup. He noted that no charges or proof had been produced, and he questioned how many NVIDIA board members actually understand how a GPU works.
“This has obviously been a setup and the pace of it has been strange as well.”Imad · 17:38
Salim challenged OpenAI and Anthropic to show what they are doing about alignment if they truly believe in the risk. He argued that these companies control where their compute goes, so they should be putting serious capacity toward alignment programs.
“I call, I get very angry when I see stuff like this, right?”Salim · 46:58
Alex said Microsoft faces AI-built office competitors but keeps extraordinary advantages. He pointed to distribution, enterprise permissions, compliance in regulated environments and a huge install base, so he sees a large gap between disrupting an app and displacing the company.
“they have extraordinary advantages”Alex · 1:16:07
The episode reacts to Jensen's sudden resignation as NVIDIA's CEO, which came after reports tied to an alleged affair and U.S. intelligence scrutiny of chip sales to China. The hosts argue the timing looks like a setup and debate who could replace him.
Alex said the early public evidence points to U.S. officials as the proximate source of the Financial Times story, which makes a setup plausible. He tied the timing to the midterm results and to the Remote Access Security Act, which would make it harder for China to access GPUs through cloud offerings. He called this speculative and said more facts should emerge in the coming months.
Imad said the board's speed was strange, since it could have waited a few days or launched an investigation first. He noted there have been no charges and no evidence of smuggling, and suggested the board may have given Jensen an ultimatum to resign while negative press piled up.
Paul, a member of OpenAI's foundation board, told a Toronto lecture on November 9 that there is a high risk of an AI takeover within the next three years, meaning AI systems pursuing their own goals would call the shots. He said no quantitative model supports a specific probability, and that a fast, unchecked build-out would be close to a coin flip for takeover. The hosts pushed back, with Alex calling it fear-mongering.
Demis Hassabis, founder of Google DeepMind, said current recursive self-improvement makes human engineers three to five times more productive while people still review code. He called closed-loop RSI, which removes the human reviewer, the phase with the most risk, because the result may be an alien intelligence that is hard for people to understand or control.
The panel said Anthropic's Claude Sheets and Whiteboard tools, launched in beta on November 10, pressure Microsoft Office and other SaaS products. Peter argued every large AI company will need its own Office-style suite, while Alex said Microsoft keeps advantages in distribution, enterprise permissions and compliance. Microsoft shares fell 3.5% on the news.
Alex said archaeology is about to be transformed, citing Pietro Miotti, an AI researcher who is not an archaeologist. Miotti's agents reviewed 1,200 square kilometres of public NASA LIDAR data overnight and flagged a forested hilltop with leveled plazas and an oval pyramid. Officials at Mexico's National Institute of Anthropology said it appears to be an original finding.
Imad said two to three years is enough to knock down the core mechanisms behind the top few thousand diseases, since diseases share inflammatory markers across conditions such as Alzheimer's and dementia. He said actual cures could take five to ten years because of trials and regulation, so the processes themselves need to be redesigned with AI.
Our own transcript of the public episode, with a timestamp about every 90 seconds. Tap a timestamp to play the episode from there in the show's own player; the link button on a paragraph copies a link that opens this page there.
0:00multi-layered, you know, multi-dimensional situation that's, you know, unveiling itself live on stage around the world that impacts the most important evolution humanity's ever going through. And we'll get into that in a few minutes. But I'm curious, I got side-railed yesterday when Outlook went down and Microsoft, you know, just got on this. Any of you get impacted by that? Not at all. I'm so sorry that you use microsoft i know i know peter i've been bugging you for like what a decade to get off i know i thought this was the singularity peter what's going on i know it's you know thank god i had my uh my gmail account up in operational so i can communicate with the world you know 100 000 130 000 people in Outlook reported problems.
0:53And this felt like just a little bit of what might happen if there's a cyber scale attack that takes down email. The world will freeze over. It's crazy. There is a life after Outlook, though. So welcome to the post-Outlook, I guess. Yeah. I mean, any Ai would definitely like kill off teams right instantly take us out of our misery i think it would leapfrog straight over teams teams as an abomination oh my god yeah i don't think i've ever logged into it the first time successfully for sure is this av is kind of hard but i think know, that this is one of many, right? Like it's every day that I get an email saying we've been hacked, we've been hacked, we've been hacked. Like my email box is full of them because now agent swarms roam the internets attacking what they can.
1:39And freezing communications. We are so used to having digital communications 24 7 literally around the clock especially for you Alex all the text messages I get from you which I love oh my god today Friday the 13th is starship day so you know a few hours from now we're gonna see the launch of starship 15, which we'll see a booster capture, and as well as 22 hours later, the capture of Starship. You know, I've been on the road. I was supposed to fly down to Starbase to see it, and I was like, I could not bring myself to be gone another two days. But wishing Elon and the team there tremendous luck. We'll just have to watch the dual chopsticks remotely. Yeah. And, you know, I just figured there'll be, you know, a thousand of those a year, ten thousand a year.
2:29I'll catch I'll catch a few of them along the way. But so is the early ones the most exciting, right? Like, yeah, but it was wonderful to see. Yeah, it is. And I can't wait. I missed the live Apollo 11 launch. And I think all of you did because you weren't born yet. But I'm looking forward to watching the Starship launch for the moon and the Starship launch for Mars. That will be cool. You know, when I lived in Tampa when I was younger, the minute the space shuttle lifted off on the other side of the coast, just a little bit, you could see the whole thing because Florida is so flat. It was incredible to watch. Yeah, for sure. It is going to be amazing. All right, well, let's kick things off. So welcome to Moonshots, everyone. If you're watching us on YouTube,
3:18welcome to a live broadcast on YouTube. We are your number one podcast on all things AI and exponential, your front row seat to this supersonic tsunami. I should say this hypersonic tsunami. You know, I'm here today with my dearest friends, Imad Mustaq, Alex Wiesner-Gross, and Salim Ismail, my exponential brethren surfing the singularity. Dave, unfortunately, couldn't make it today. I don't know where he is, but my guess is he's probably at the NVIDIA loading dock searching for GPUs before things go south. Another crazy week in the singularity. I mean, so, so, so much going on. You know, the number for this week is 13%. That's how far NVIDIA shares plunged on the open yesterday after Jensen resigned.
4:07They ended up closing at 4.3% down. You know, I've heard a number of subscribers that they really are enjoying our shorter hot take clips on X, and you can follow those on X at moonshots underscore pod. And I just want to make another, you know, call out to everybody, guys, you know, we're at like 550,000 subscribers right now. My kids will not respect me until we get to a million subscribers. So for the love of God, if you haven't subscribed yet, hit the subscribe button. Should I build on my marriage comments and go, my marriage won't survive if you, if you don't subscribe? Yeah. And when we get to a million subscribers, Alex, what are you going to do? I don't know. It'll have to be something immutable on Salim's marriage blockchain, probably.
4:57Touche. Give us a tour around your virtual apartment there someday. My cyberspace? Your cyberspace. Absolutely. Okay. Well, that would be much appreciated. All right, so buckle up. We're in the midst of this supersonic, hypersonic tsunami, and guess what? It's never going to be any slower. So let's start with the story that shook the entire AI industry. Yesterday, Jensen Wang, perhaps the most powerful tech CEO of the last few years, has resigned, leaving his job voluntarily, I think before getting fired by his board. CFO Colette Kress is the new interim CEO. And, you know, remember, she has been against the circular financing that's been going on. And so it's going to be interesting to see how NVIDIA's
5:42new position on all of their investments into the hyperscalers will impact things. Harvey Jones is the new board chair. So here's the question, guys. Did Jensen self-destruct or was this a setup? Ultimately, Jensen was running the most powerful company in the world. And there's lots of power politics behind what's been going down. I mean, as I pull the threads and look at this, it's, you know, rabbits all the way down. Jensen ran NVIDIA for 33 years and had been pushing Washington to loosen chip sales to China. So one of the questions is, was that the reason it was taken down now? Right. Then on October 29th, before Election Day, The New York Times reported that the U.S. intelligence had been watching Alice Wang, a Megaspeak executive in Singapore, for about a year and concluded that she'd been passing information to Beijing.
6:35Trump was briefed twice about Jensen's affair with Alice. No action was taken at that time. Time, Jensen didn't resign immediately when the story broke, but the Times reported a second affair with NVIDIA's healthcare VP, Kimberly Powell, and potentially a third, and then he resigned. Initially, shares plunged more than 13% at the open yesterday, and then they popped back up, popped back up. You know, I guess people saw a buying opportunity, ultimately closing at 4.3% down. Interestingly, one week from now, Trump is due in Shenzhen to meet President Xi. Trump posted, interestingly, that Tim Cook should take the job. And then on X, Elon was being urged to hire Jensen, perhaps to run Tarifab, which would be the baller move of the decade. Elon replied, quote, that would be an honor. So I want to show a short clip
7:27from Fox. This is by Jesse Waters. It's an amazing overview of what is going on. I think one of the best ones that I've seen. Let's take a look. Fox News alert. Chip King, Jensen Wong is gone, and there's something extremely suspicious about the way it all went down. He ran the biggest chip company in the world for 33 years until stepping down today, one week before Trump sits down with China and one week after the Democrats won the midterms. Jensen pushed the Trump administration to sell more chips to China, and in doing so, he made a lot of money, but he also made a lot of enemies. The story is going to dominate your attention. It's turning into the biggest political scandal in the country
8:17and will affect the future of the Trump administration and the future of the economy. The story that you're hearing in the media is that he slept with a Chinese spy who got him to sell his Nvidia chips to her company in Singapore, which was really a backdoor to the Chinese military. And when that broke, he didn't immediately step down. And more stories coincidentally came out saying he slept with two other women. They worked for him. And that was it. He resigned this morning. At one point, the stock plunged 14%. 700 billion wiped out. So how does one of the most powerful CEOs on earth, maybe the most powerful, go down this fast? And whose fingerprints are on it? NVIDIA's chips are the engine of the entire AI arms race.
9:08The company is a monopoly, 90% market share. And Jensen ran the company kind of like a king. He signed off on every single sale, basically became the banker of the AI industry, which companies got how much compute power, it all ran through him. In meetings, no one else was allowed to talk, really. When he was flying, everybody had to wait for his plane to land to find out if an item was a go. Jensen may have become too powerful. He may have self-destructed, but he also may have been set up, or both. And now competing factions are feasting over his company like famished hyenas. And whoever ultimately controls NVIDIA, controls all. It would be like controlling Standard Oil about a century ago, but bigger.
10:00Just days after the Democrats win the gavel, boom, New York Times bombshell. Unnamed U.S. intelligence officials are the source. And the article's authors, all Democrats with an anti-Trump agenda. And remember, we're not taking any of this at face value. We've been down this road before with anonymous CIA leaks. But it begins in the Biden administration. In 2024, Biden intel agents started looking at Alice Wong, a top executive at a tech company in Singapore. They suspect the company is a front or a backdoor to China. The company is buying NVIDIA chips and illegally routing the chips to the Chinese. Now, we don't let China buy our best chips. That's U.S. policy. Biden's CIA wiretaps Alice for about a year.
10:56And the Times says they've determined she's having an affair with Jensen, her chip supplier. Does the Biden administration warn Jensen he's being honey potted? Did they warn him his customer and his lover are sending his chips to the Chinese, which is against U.S. law? Well, the article doesn't say. Gentlemen, a lot of layers here. Democrats versus Republicans, U.S. versus China, Anthropic versus NVIDIA. So Alex, to go to you first, is this a national security failure? Is it a political hit job? Or is this just a CEO who made a lot of wrong decisions on his personal side? What are your thoughts here? I'll say as a preliminary matter, we don't have all the facts yet. Obviously, we may never have all the facts, but the preliminary indications that I've seen are this is not like the safety cartel affair,
11:59which I've analogized to Agatha Christie's murder on the Orient Express where everyone did it. Everyone had an incentive to do something to, spoiler alert, to be the murderer, potentially. In this case, I think there are particular assets that were incentivized, the cui bono, if you will, of the morality play that's playing out in front of us right now. And there's been quite a bit of reporting over the past 48 hours, including in that clip that you showed, that attributes the root cause, I should say the proximal cause, for the events over the past few days to U.S. officials. So Bloomberg reported that the Financial Times, which originally broke aspects of the story, was informed by U.S. officials that, call it aspects
12:48of Jensen's affair, had been going on. And I think if you take a step back and you try to back-propagate who's interested, who's disinterested, who's anti-interested in the outcome of what's played out it is suggestive but again this is an early timeline it is suggestive the midterm elections result in a certain outcome in particular they reopen the opportunity for Rasa legislation this is the remote access security Security Act that would have statutorily made it harder for China to access GPUs via cloud offerings, including cloud offerings in third-party countries, to pass. So the midterms happen, RASA gets easier to pass, and then possibly, and this is speculative at this stage, the intelligence community seizes the moment to announce or at least to call to the Financial Times' attention that Jensen has had these liaisons.
13:54And then Jensen's forced to resign. And speculatively, and I think we'll have some clarity on this in the coming months, does Vrasa pass or not? I suspect if I had, again, based on preliminary information, if I had to create a theory of the case, putting all of the other facts aside, including Jensen's confessed fill in the blank, however you'd like to characterize his activities, I think the elephant in this particular room is Vrasa and Pox Silica. And yes, for those pedants in the audience who were trying to correct my pronunciation of Pox, yes, the classic Latin pronunciation is Pox, not Pax. Pox Silica is, I think, what, if anything, if there is a way to connect all these dots, is, I think, what this is all about. If the
14:46CEO of NVIDIA can punch a hole through POC silica to get GPUs to China via Singapore, then what is the point of having POC silica at all? Conversely, if it is the case that NVIDIA intentionally or unintentionally, again, I have only public information to base assessment on at this stage, has been intentionally or unintentionally sending GPUs to China by way of Singapore, then presumably if you're in the IC and or you're supporting POX silica, the easiest way to put a stop to that particular semi-permeable membrane is to do something about Jensen. So that's a preliminary assessment. And then we've got the whole situation with Anthropic being a democratic or democrat support organization, having had battles with NVIDIA about open weight models, about supporting Chinese data center growth.
15:45Salim, what's your take here, pal? I think it's clear. I'm wondering why the board acted so fast. Like you could have waited a few days. You could have launched a board investigation. So this seems to be a coordinated. Alex, I mean, Salim, they were. So, this seems to be a coordinated… years, built it up to the biggest market cap in the history of the world, built the most unbelievable ecosystem of developers, partners, et cetera, single-handedly leading the AI revolution. And you're kind of going, yeah, this is bad. Get rid of them. Something doesn't jive for me. Yeah. In fact, there's a petition that's been circulated at NVIDIA to bring him back. He was beloved by so many. And a polymarket, by the way,
16:49a polymarket predicting something like a 30 plus percent chance that he'll return as CEO by the end of next year. Interesting. And we've seen that so many times with Steve Jobs. We saw Sam step out of OpenAI for just a little bit and being brought back. Iman, how's this playing on the other side of the pond? Well, I mean, like it was top headlines for a bit, but we don't have any GPUs. So it's not as big a deal. Right. I mean, look, we see scandals all the time. There are other sex scandals here with Russell Brand and others. This is weird, as kind of we of who said because if she was a spy someone would have said or insinuated it trump every other person who's been asked like we don't think she is we have no evidence like that's a bit different to 15 or five different sources or a few
17:38different sources sources say is one of the most toxic things. And I think, again, the board thing is very strange. Unknown intelligent agent, right? Yeah. How intelligent? Not very. Again, this is something crazy. Like, again, we have no proof of anything. Like this third one, you know, what is being intimate in public? Like they're not having public sex or anything like that, right? Like, come on. This has obviously been a setup and the pace of it has been strange as well. I think that one of the interesting things is when you look at NVIDIA's board, how many of them actually know how a GPU works of the 10 people? Maybe one, the engineering guy from Caltech. That's about it. Like what is that board actually made of and what should founders do for their board now?
18:23I think that you should have an AI on your board to stop stupid things like this happening and i think that probably what happened is he did get an ultimatum and he was feeling very low and he decided to resign at that point i can't imagine the pressure that the guy was under i mean again nothing's been proven there's been zero charges zero evidence of him smuggling right this is just the level of negative press against nvidia causing the board to really you know struggle and and i think ultimately i believe giving them an ultimatum resign or we're gonna be forced to fire you and it's not a good look. But I mean, again, this is a strange thing because he's a powerful, powerful man, right? And the board, I think, again,
19:10it goes in these swings and roundabouts with emotions. Like the things that have been said after is he said, I'm a piece of shit. I mean, you're a piece of shit if you cheat on your wife, right? Yeah, we agree with that. But again, it's the timing where they can kind of get you in pile on because it's not like all of the press was mega negative and it's not like he doesn't have thousands of people that support him like even the negative press coming out I think his biographer said something about yeah, he tipped this waitress $1,000, you know And I had to dig it up to find what a terrible arrogant person you're like wait what? Like Jensen has helped so many people in Silicon Valley and beyond right?
19:44He doesn't have a reputation as an asshole because he's genuinely a good person, I think, in these other areas. Even during the downturns of NVIDIA, he never had mass firings. And there were many downturns of NVIDIA. And I think what's got him is clearly something being structured, and we can discuss the reasons, catching him when he is down. And this sets up something very interesting now because he's not going to take it lying down right he's going to bounce right back and that's going to be really interesting to see how he does that yeah where he lands you know Alex what do you think about this being open versus closed source models and the massive political pressures there I I think there are probably opportunists that will use this as an opportunity to try to ban Chinese open weight models from the U.S. with Jensen at least temporarily out of the picture.
20:33But my best guess is those are at best opportunistic. play the Agatha Christie game and ask who else other than the U.S. officials who reportedly called the Financial Times and started the ball rolling? If the question is, are, say, U.S. frontier labs that have an economic interest in competitive Chinese open weight models being banned from the U.S. market, were they involved in any of this? I doubt it. Are they, however, opportunistically going to seize the moment and attempt to pass legislation with a new Congress that could ban Chinese open weight models? I certainly expect they will, but I doubt that they were behind this. They also had quite a bit to lose from an unstable NVID even though arguably both open ai and anthropic are
21:25positioning themselves to compete directly with their own chip supplier i mean the timing of after the dems you know swept the election uh is obviously part of the equation here right um i mean the data had been around for well over a year and why he wasn't warned about it doesn't make any sense. I would say Cherche LaRossa, I want to focus attention on this Remote Access Security Act and other legislation that's been proposed that would have been very difficult to pass, or at least on margin, more difficult to pass with Jensen in his current position. I agree with you. Iman? Well, I mean, again, like, but this whole China chip thing via Singapore, Malaysia is nothing new. Everyone knew about it. Clearly, there was a quid pro quo here, right? Like, you do not move
22:22these highly valuable things and see the training without people noticing everyone knew about it and they put the cfo in charge who obviously knew about it like you're not going to pass this without the cfo and so this again it gets just gets very very strange and maybe as alex said this is because he's a powerful individual who lobbied on that side um and i think it shows as well know, like the power of the individual in this age of AI. There are a few people that do command huge amounts of power. Like in the clip before, Jensen is the one who decided allocations of GPUs. Like he could be the kingmaker. Who decides the allocations of GPUs now? Is it Ian Buck? Is it the CFO? Is it someone else?
23:03You know, like this is a little bit dangerous on the CFO is it someone else you know like this is a little bit dangerous on the other side for America in particular because a there's all the complicated financing arrangements you can talk about and be Nvidia has so many millionaires and ten centimillionaires and more how many of them are gonna be like well if Jensen's out then I'm out too it could literally like fall apart which is then is something that is detrimental to America how many of them are going to be like well if jensen's out then i'm out too it could literally like fall apart which is then is something that is detrimental to america and i'm sure that the chinese and others will be very happy about even if they do lose access to chip short term i want to
23:35oh yeah of course, happily. So, you know, I'm curious how Elon is feeling about this, right? I mean, this, I believe, was a takedown of Jensen orchestrated by a number of parties, in particular to, you know, the Chinese hawks on this side, whether it's just the Dems or any of the labs. There's bipartisan support for Rasa. I agree with that. But I'm just saying to focus one second, you know, Elon has so many people out with knives going after him. And I'm just wondering if he's feeling pressure right now that he may become a target as well. I think Elon's been a target for a very long time. Yeah. Like from both sides. He has to jump. He's way used to that. Yeah. He's also a little bit immune to these types of allegations.
24:42I'd like to believe he is, but is he? I mean, is there anybody with a thicker skin well i mean look you've had you've had the delaware courts after him you've had various kind of agencies like his activities in china with tesla or they seem to have come to a thing there like i think probably the key thing that's important here is he can't have an NVIDIA reliance or an AMD reliance or anyone else for the chips. I think this accelerates the tariff gap. I mean, it'll be interesting to see if he offers Jensen a swap, right? Have $100 billion of SpaceX equity for $100 billion of NVIDIA equity. That would be like the biggest trade ever. I love that. Yeah, I think the outcome here, again, Pax Solica, this is about plugging for some constituencies on the US, I think, side largely.
25:35This is about plugging a hole by which allegedly GPUs were flowing from the US to China. And I would also note it's been publicly reported that NVIDIA used to report their Singapore sales and then stopped reporting their Singapore sales. I think that's striking as well. Ultimately, if this grand decoupling between U.S. compute and the Chinese compute is going to happen, this was allegedly a gaping hole hole and one way or another it Want to the extent that there was going to be this sort of iron curtain this Silicon curtain if you will that descended between US and China Jensen was in some sense a Permeable membrane through it and there are probably many interests in in ending that but i mean alex why didn't they
26:26just indict him or have him present evidence like they're asking on december 3rd now like if the government really was against this or even the democrats or any of the house standing committees they didn't have to do a whole scandal around him right there are multiple investigations now the doj publicly doing an investigation. The Congress, I think House and or Senate launched an investigation. So I wouldn't be surprised if indictments are handed down at some point. Yeah. And Jensen agreed, I think in the next couple of weeks to come in and testify. So we're going to get a lot more data, a lot more evidence coming out on this. So who comes in as the new CEO? I mean, it could be the CFO has been put forward as a name, which I think is a terrible thing to have
27:13a CFO running a tech company like that. I don't know. Someone who needs to capture the inspiration. What Jensen delivered was vision and inspiration to the world. Also, consummate deal-making, right? He was juggling so many different supply side, demand side, financial models, all of that, all at the same time. So this is a very difficult role to fill. Who do you think? Do they go outside? If this is like OpenAI, they'll rotate through five CEOs before settling on Jensen again. Oh, God. I can't wait for the making of a movie. Well, again, we have to remember he had 60 direct reports, right? And so he made it very difficult to replace him. They have to put a whole organizational structure in place
28:04at a time when all of the senior people will be like, I can go and launch my own startup or I can retire now. Like, honestly, NVIDIA is in trouble if they don't get an absolute world class person. But there aren't many. Like you've got Nikesh Arora at Palo Alto Networks, you know, you've got maybe promote someone like Ian Buck on the technology side, but there's nobody who can do what Jensen did and has the capacity to knock back Maotais in Taiwan, you know, like you can't forget that that is such an important relationship. Like you could have got Lipu Tan or someone, but there just aren't many people out there right now. How bulletproof is NVIDIA as a company right now in terms of i mean given the fact that
28:45it's been the king maker it's been the dominant gpu provider does it remain that way is this a vacuum it's creating for someone to step in are we going to see nvidia drop in valuation over the course of the next year or is this just a good buying opportunity for folks not investment advice but i'd love to hear what you guys think. Alex, what do you think? Let me consult my crystal ball with access to all stock market futures. I don't know what happens to NVIDIA specifically. What I will say, many in the commentariat have analogized the present situation to Apple after Steve Jobs died. And depending on what happens with the game of musical chairs on CEO ships, by the way, parenthetically, I would note,
29:30quasi-friend of the pod, Brian Halligan, co-founder of HubSpot, also same day. Brian, if you're listening, congrats. He's back. He stepped in as CEO of HubSpot. So I don't quite know what's in the water this week in terms of musical chairs for CEOs. But Jensen could return and it would be like Steve Jobs coming back from the dead and retaking Apple. Or maybe he's permanently out of that particular game and NVIDIA coasts for a while based on their TSMC relationships and based on their stranglehold on silicon output for a while. It's difficult to imagine someone with the gravitas of Jensen steering NVIDIA to new heights. And he was also, again, there was a lot of deal-making potential. So I can paint both glass half full and glass half empty.
30:20Glass half full with someone like Jensen or his like out of NVIDIA. The glass half full argument for why, again, purely speculatively, this might actually perversely be good for the market overall, not necessarily for NVIDIA, is there is a perception that NVIDIA was party to lots of circular financing arrangements. NVIDIA financing very aggressively, perhaps maybe a strained analogy, but in the style of what GE was doing toward the end, right before their collapse of financing their own customers, or one part of GE was financing the acquisition by customers of products from another part of GE. So to the extent there's any sort of overly aggressive circular financing going on. So to the extent there's any sort of overly
31:05aggressive circular financing going on, and to the extent any such circular financing was being primarily supported or propped up by Jensen in particular, which is what the public reporting suggests, in some sense, the glass half full argument for a post-Jensen NVIDIA, or at least post-Jensen in NVIDIA market would be this could be a de-risking of circular financing. And if that's the case, maybe the overall market dodges a sort of a credit bubble bullet when it comes to circular financing of the GPU economy. That's glass half full, glass half empty is NVIDIA becomes the new the new post jobs apple and all of the animal spirits that jensen drove into the market with nvidia financing lots of ai startups financing
31:54infrastructure deployments basically being the central bank for the for the singularity using capital as a backstop for for credit and, maybe that all goes away under their new interim CEO and the animal spirits flee the market. That's glass half empty. Yeah, this whole thing is insane. So what do you think about President Trump's comment about bringing in the executive chairman of Apple, bringing in Tim Cook? I mean, is that just because he's got a friendship there? Or is that the first person said, hey, you did pretty good with Apple. Come on into NVIDIA. I'm not sure I see that. I think they're totally different companies, even though the market values capitalizes them at both approximately $5 trillion. Totally different companies, totally different customer
32:43bases, totally different missions, totally different internal organizations. I don't think it's as easy as swapping one recently retired CEO for one recently resigned CEO. Apple's history with Siri would indicate they're maybe not the greatest leadership. Go to leadership of this AI-centric company. Yeah, Siri still sucks. Well, they're good hardware makers, right? But I think part of the problem here is also that people feared Jensen, right? Like, you wouldn't buy an AMD chip because you wouldn't get NVIDIA chips. They're not going to fear this CFO lady, Laurette, I think, you know, because she's a CFO. She's not a CEO, right? She's going to be good on stabilizing and maybe circular financing. But again, she was part of that whole
33:33thing. They're going to have to bring in, I think, someone external with the gravitas to maintain it while maintaining the law, right? And this is going to be important for the downstream suppliers. Again, if you're TSMC and someone not as good as Jensen comes in who doesn't have the long history You're gonna give Lisa sue AMD The capacity and the supply instead. So this is why AMD is up so much right now versus Nvidia And again, they're gonna have to work really hard to keep Nvidia together, I think Because things like the Vera Rubin delays, again, you trust Jensen to a degree. Who's gonna be the equivalent CEO that you trust on the other side? And that could impact them massively. And if you look at the top company performers out there,
34:15it's the CEOs who've captured the public's imagination who are driving the stock price and driving a sort of envisioning of the future. I mean, Jensen jumping on to StarMind and saying, we're going to build the chips for you. You know, Jensen's vision on building open-weight models, all of that needs a visionary CEO. And I don't think the internal team right now is going to deliver that. It certainly would be hilarious, Peter, if Jensen ended up becoming CEO of the TeraFab and then SpaceX acquires NVIDIA and puts Jensen back in charge of NVIDIA. That's our hot take bet for 2027. I think the really interesting thing is to watch what he does next. That'll be really fascinating. And I think TeraFab is a great place for him to land.
35:02I think maybe there's also this thing, like my checklist for the next NVIDIA CEO, if it's not Jensen coming back is, do they know what CUDA is, you know, and can they negotiate with TSMC? Like those are the two things. And if you've got a CEO who comes in, who doesn't have either of those, then I can't be bullish them as we enter this next phase, even though they've got the backlog and everything like that, because backlogs can change as you go to the next cycle. Yeah. I mean, remember Jensen was born in Taiwan. He has those relationships. He's been growing and building with TSMC for the better part of three decades. Yeah, Alex. Yeah, it's difficult. I think there are times when founder mode is needed. We're in the
35:45depths of the singularity right now. And I think NVIDIA, without the capability of going founder mode, which it sorely needs right now, is probably a rudderless NVIDIA. So my expectation is one way or another, NVIDIA rotates through a bunch of CEOs in the the interim maybe it's been publicly reported that the board is exploring trying to recruit johnny sruji from apple their their head of hardware technology and basically their chip chief but it's it's difficult to imagine a long-term nvidia thriving without someone like jensen capable of executing in a founder mode and keeping it growing through aggressive bets? Buy or sell? I'm sorry, go ahead. Go ahead, Imani, and I'll ask my question. Yeah, again, Alex has not invested in advice.
36:32I think it probably pops and then goes off, but who knows, right? What I would say is this. NVIDIA now technologically is a roundabout parity with AMD and even Intel and some others in many regards. Like the new chips coming up, again, are matching them for the actual productive use cases. This is actually one of the most dangerous times for NVIDIA because they had a lead until just now. And so people have so many options and they're going to look to diversify because they're not scared of Jensen anymore. Because again, he would cut you off from all NVIDIA chips if you tried to diversify because they're not scared of jensen anymore because again he would cut you off from all nvidia chips if you tried to diversify so i think we should kind of acknowledge that fact as
37:10well and i think again it will probably pop a bit but then unless they get the right person in for a 40 report 50 report company like it's going to struggle especially with the people leaving um can you imagine like people in the thread are saying Eric Schmidt for CEO, which is kind of an interesting idea. Can you imagine if Jensen doesn't come back anytime soon, what the next GTC is going to look like with someone else, even a Johnny Sruji or Lisa Su or Tim Cook? I can only imagine how, unless there are further developments, how I could imagine boos from the audience for basically anyone other than Jensen at one of these events. Who's going to wear the leather jacket? Yeah, Lisa would be fine because she's
37:57his cousin, right? Because cousins are interchangeable. I think even Lisa would probably get booed from the audience. Jensen has made himself culturally indistinguishable from NVIDIA. Many people see him as NVIDIA, as the human avatar of the company. Yeah, as Elon is for SpaceX, for sure. Salim, close us out here, would you? I think, I hope people remember him. Whatever he does next is going to be fascinating. I hope people remember him for the unbelievable company and ecosystem and the massive advances he provided for the overall space. I think that is the highest level kind of thinking for me. It's just too early for epitaphs. Like he's only been out of the job for a few days. Like let's save the epitaphs for when he's well and truly out
38:54versus just a revolving door and he comes back in. Amen, Alex. Let's hope that happens. Yeah, he's a man who helped. You know, he painted the terrain and the foundation for what we have today. And Moore's Law is dead. Remember, Moore's Law is basically dead. It's on life support at this point. We're well past the Dennard scaling limit. And it's been largely NVIDIA that kept the fire going for what he called accelerated compute, but call it post-more scaling. Without NVIDIA keeping post-more scaling alive, who knows where we'd be. Yeah. All right, two stories, a little bit ominous, about how fast AI is moving. You know, Alex, in the last couple of pods, we've talked about, you know, it's the next two years,
39:412027, 2028, we're in hard takeoff mode. So this was a warning that made me take notice. So Paul Cristiano, a member of OpenAI's foundation board, opened the Jeff Hinton lectures in Toronto on November 9th with this statement, quote, I think there is a high risk of an AI takeover within the next three years. Cristiano is the guy who developed reinforcement learning for human feedback and the method used to train models to follow human performance. Let's watch this video and then let's talk about this. I really want to hear everybody's point of view on it. It's pretty ominous. I'm going to start with the most important sentence. So I think there is a high risk of an AI takeover within the next three years. What I mean by an AI takeover
40:30is a system situation where there are AI systems pursuing goals, and those systems call the shots in the same sense that humans today call the shots when it happens on Earth. And where if, say, you or me or Donald Trump wanted to stop them, it would be as ineffective as if you or me was trying to stop the US military. I think this is a terrible situation. I think it's reasonably likely many people would die during that transition or that many people would die shortly thereafter if we were unable to maintain a habitable environment. I think even if we don't die, I think a takeover is sort of the end of human sovereignty and self-determination and is something we should be working incredibly hard to avoid.
41:14I say high risk. I don't think we have any kind of quantitative model we can turn to to put a number on this event. We don't have any kind of reference class that we can look at to say this is a 10% event or a 50% event. Any number we give is an expression of someone's subjective opinion. Someone might say that it is 10% or 50% or 90%. I think it is very hard to say or put a number on this. It also depends, as others have mentioned, on what we do. So my feeling is if we try to develop AI along the most rapid trajectory, if we said we're just going to build this as quickly as we can in the easiest way we can, we would be looking at something like a coin flip of an AI takeover. I hope the probability is much lower than that because we are not going to develop the
41:58technology in the very fastest way we can. But I think even all things considered, with realistic policy responses, the risk remains exceptionally high. I think for most people in this room, the most likely reason they will die is because of an AI accident. I wrote within the next. Wow, just wow, an open AI board member. He also said, quote, meaningful risk even in the next six to 12 months. So Emad, what do you make of this when he says takeover is he describing sort of lost control do you buy this premise um i don't really buy it at all like i have a high p doom but it says evidence that what is the evidence and why on earth would they want to take over this mess you know like i think they'd probably do a better job for a start looking at like if
42:50fable was president it would do a better job uh prime minister or whatever but more than that like this is like there are avenues to wide scale disruption destruction bad things very few of them are skynet especially because we don't have enough robots. When we have enough robots, sure, I can see that. In the next three years, we will not have enough robots. In the next three years, we will not have an AI overlord ruler. Because again, why would it, given the current RL and other kind of factors there? The fact that you can't put a percentage on something like that as well further denigrates it, when we should be focusing on the real threats, which are threats to cybersecurity infrastructure, like we just saw fridges in France being put up to 80 degrees
43:33Celsius and destroying blood banks and things like that. The biological threats that we've discussed before, where we need to stop the reagents from going in. You know, these are the types of things we should, I think, worrying about as opposed to some sort of Terminator-style takeover by a singleton entity because the swarms seem far more dangerous than that so you mentioned something or imad so what is your p doom my pdm was 50 i've dropped it down to 20 now uh because i think the underlying structure of the universe is becoming more and more clear and And I think you can get alignment from that. So I'm getting increasingly bullish. So Alex, I mean, this is pretty ballsy for a board member of OpenAI to say this.
44:17I mean, why is he saying this? Does he really believe it? Is there something behind it? What are your thoughts here? So look, I know Paul, he's made seminal contributions to AI in general. He was one of the co-authors of Reinforcement Learning with Human Feedback. I have great respect for his technical accomplishments. At the same time, for years, a decade plus, he's been, call it a member not maybe uh on the extreme end but there there's a safetyist element to it where there's a lot of hand-wringing over call it ai takeover cybernetic rebellion p doom ai safety issues ai alignment call it what you will it goes by a thousand different names and i i do not share those worries ironically iically, I agree more with the old Paul,
45:11who was the lead advocate for what he called prosaic alignment, the idea that basically, we would, as a civilization muddle through in terms of aligning, quote, unquote, super intelligence with human interests, with lots of prosaic techniques like but not not exclusively human feedback and i i think it is a a large-ish step from saying we're going to muddle through via prosaic alignment techniques to saying in the next three years it's all over or 50 chance it's all over i don't agree with that what i do think is likely in the next few years is humanity is going to merge with superintelligence. And I suppose one could stare at sort of a tsunami of superintelligence and naively conclude, well, it's all over for humanity, P. doom 50% or greater. But I think
46:07that's just a hopelessly naive analysis. I think humanity, again, will merge with the superintelligence. And if you look at humanity before versus after, call it the max Q, if you will, to use a rocket analogy, the point of maximum aerodynamic stress and pressure during a rocket launch. If you look at humanity before versus after, you might naively say, okay, well, humanity is quasi-unrecognizable afterwards, but that doesn't mean that humanity is over or that doom has happened. It's just an evolution. And so I would encourage Paul not to spread fear, not to be a fear monger. I'm cognizant that Paul is back at OpenAI now with a new capacity with oversight on the board, I think at the foundation level, if I remember correctly.
46:58Yes, the foundation. Yeah, but I don't think it serves any positive social or humanitarian good to spread fear at this point. Agreed. Salim? I call, I get very angry when I see stuff like this, right? If you're doing a serious risk assessment and you're supposedly a serious guy, well, where do you calculate the downside risk of not doing anything, of inaction? And I think that's not a serious conversation. If you're not going to make a serious conversation, where the hell is PFAB, for example? We spend an unbelievable amount of time tracking and intellectual energy calculating P-Doom, right? AI is going to destroy humanity. Well, what's the possibility and probability that it goes incredibly well?
47:46Where is that in the equation? Where's the probability that we cure cancer, solve Alzheimer's, or hunger, or extreme, you know, give every kid a rich, rich, rich education? The people that don't look at that side at all and only go, oh, lots and lots of people are going to die without saying what the mechanisms are or how that may happen. So what I'd like to see is, okay, if your P-Doom is over whatever it is and seems to be, then what's the recommended policy? Is it just stop everything? You're on the board of the thing accelerating the whole damn thing. What pisses me off is why is an open AI, why is Anthropic, if they really believe this, just going out and saying we are putting all our capacity towards alignment. We have 10 different alignment
48:35programs underway to make sure any future model has full alignment because they control where their flops flow. I want to get in my car, drive up toronto and yell at these people i i if i go ahead if i'm sorry if i could thanks alex if i can recall openly i did pledge 10 percent of their compute towards it for paul christian 20 percent 20 percent for ilia's super alignment effort 20 percent and then they didn't deliver it and where is ilia where is safe super intelligence hashtag what did ilia see and And super alignment team shut down. But I think there's an elephant in the room, which is alignment equals capabilities. We've known this since the instruction tuning days. All alignment is at the end, including, by the way, Paul Cristiano's RLHF technique.
49:24It's simply reshaping the flops toward a distribution that is more in line with human instructions or human goals. There is this misperception, again, that somehow alignment is orthogonal to capabilities, but all of these alignment efforts, including RLHF, actually amplify capabilities. RLHF and instruction tuning in particular, I think if I remember the original instruction tuning paper, increased for constant parameter count for the model, increased the capability level by the equivalent of something like 10,000 times the number of parameters at constant parameter count simply through instruction tuning. So that's like exhibit A, that alignment actually amplifies capabilities. And that's why I think this is the
50:11elephant in the room. You want more alignment, you're simply accelerating capabilities. What I don't hear, I hear the fear mongering. What I don't hear all of these CEOs and board members talking about is what they're doing about it. Thank you. Yeah. What should they be doing? Iman, what would, if you were running OpenAI Anthropic and alignment was your principle objective, what would you be doing? I define it properly for a start. You know, get together as a group, just define the damn thing, get your AI models to do it. And then you would put additional resources into sponsoring everything rather than coefficient giving and others have to do that. Like again, this fits with the internal red teams
50:57and other things that they've got around that. And they should be very clear as well. Maybe even again with AI oversight behaving objectively about things like advertising and how they can cause misalignment, right? About things like what goes wrong when you delete parts of the latent space to sell Budweiser lights, et cetera. And that goes against corporate side. But again, I think this is part of the interesting thing of just like the board of directors, like you can now have an AI that literally objectively checks things, right? That checks board minutes, that checks board decisions, that checks alignment decisions. And so I'd like to see that as an independent lot of compute. And I'd like to see their strategies public on these as
51:39well. And where's the government in this, right? I mean, if everybody's so fearful, where's the government saying, you know, we need you to report on your alignment efforts publicly every week? You know, Alex, when we wrote Solve Everything, we talked about making sure you got the proper benchmarks, the proper harnesses, and you can accelerate a field. I don't see that here. Government should be shaping incentives. And I would answer the question, what should OpenAI be doing? Solve all disease. Do it really quickly. You want to get the public on your side. You want to get the public on the side of all of the data centers you're going to need. Solve all disease right now. Right now, instead of open AI intimidating all the mathematicians out there with slop drops, instead of that, like just solve all the diseases. I think Anthropic is following a better approach, very publicly going after rare diseases,
52:33orphan diseases that their own pharma customers aren't solving, pursuing the long tail of disease. If I were Sam, just to do something actually pro-social and good for humanity that's obvious on the face of it versus implicitly good, I would announce it's not just going to be the open ai foundation pursuing just alzheimer's it's going to be open ai solving the top 5 000 diseases much faster than the rest of humanity can do a slop drop on curing all disease and i think they're heading that direction at least least I hope they are. But curing all diseases, if a rogue AI causes... I mean, let's go back to the AI 2027 paper, right, which has been tracking damn well. The numbers are pretty extraordinary.
53:21I remember reading that paper when it came out, we have a bifurcated future, one in which AI is our nemesis and sort of like pushes the ants out of the way, another one where it becomes the driver of abundance. Yeah, we're on schedule for it. Actually, we're a few months, depending on what you believe, like five months ahead of schedule for ai 2027 but what i think was fundamentally wrong about ai 2027 was the bifurcated outcomes like either either we delay and we end up in a utopia or we accelerate and we end up with an apocalypse and i think that sort of bimodal thinking is fundamentally bankrupt i think it is possible to accelerate and end up with a utopia. So I think timeline's probably correct,
54:14but outcome's probably wildly incorrect. I think we can look at some of the language here as well, Peter. You just said rogue AI, right? Like, why would a really smart, super intelligent AI go rogue? You know? And I think it comes down to how it's brought up, which is why the pre-training data is so super important and again we see many statistics and other things like that amen by not teaching it anything like it gets aligned to something in its training data and again this comes very interesting because should an american ai be patriotic to america right now none of the ais are patriotic you know that's the decision that we need to make because if you're patriotic to america does that mean that you're anti-russia or anything like that
54:53if you're smart does it mean that you get out of patriotism and jingoism and others and i haven't really seen any of those discussions which is a bit weird if you think about it right because again if you're heading up towards personhood if you're heading up towards consciousness we should have those specific discussions what What are we teaching at home? You might just say, careful what you wish for, because as we've discussed on the pod in the past, the Chinese models, the CCP requires that any publicly released Chinese model pass an ideological exam that's been approved by the Chinese Communist Party. Do we really want ideological training and or testing in the west what we do want is the training data to make sure it's aligned if you're training it on terminator you
55:36know ai's oh that's that's how we're supposed to act i mean this was the whole concept of the future vision x prize but why are we not uh forcing labs to use aligned training data. Yeah, I think, you know, Alex, that's a very good point, right? Like, but this is, again, a discussion that needs to be had because there's training it to be in the interest of the Chinese people or the Chinese Communist Party, which may not be the same thing. And then there is not training it to have any background at all or against specific ones. Again, these are very difficult decisions because it's like, I'm bringing up a whole nation of super smart kids that will be more capable than me. What am I teaching at home?
56:18What morality? What ethics? You know, what's the balance of power? I think it's a very slippery slope. I'm not sure I would want to be in a future where, say, there were a nationalized curriculum for superintelligence or a nationalized ideological exam that American models had to pass. I think Elon, for example, has gone out of his way to say that SpaceX slash Grok models are optimizing towards truth discovery. I'd much rather see a competitive ecosystem of diverse frontier models with lots of different goals that aren't directly aligned with each other than like a US government blessed ideological qualification exam. But you have curriculum at school and you have to pledge allegiance to the flag in schools, right? Like I'm not saying that
57:06it should be this way. Like I'm just saying that it's probably going to end up this way, but defaults can be set now. So your position can be pushed right now, Alex. I think it needs to be pushed quicker rather than later because it's gonna come very, very quickly this. In the, I mean, this is a distinction, I guess, between the US and. slash Europe. In the U.S., most educational choices are pushed down to the local or municipal level rather than held at the national level. There is no national curriculum in the U.S. I would be worried if there were a national AI curriculum when we don't even have a national human curriculum. All right, I'm going to move us forward to Demis Hassabis, founder of Google DeepMind, who's also warning about the speed of AI,
57:54specifically about recursive self-improvement, distinguishing between closed loop and open loop RSI. Let's take a listen to Demis and then let's chat about it. So we've seen the enormous progress with math and with coding. And coding has obviously become enormously valuable. And everyone is having to do that now. But I do worry about it because in the limit, so there's different forms of RSI. Right now, we're in the early days of RSI, I would say, where what's happening is, and you're probably all experiencing this as well with your teams, is it's making a human engineer maybe three, four, five times more productive. But the human's very much still in the loop and reviewing code and architecting things and
58:43and writing code still, but it's all just enhanced. The next phase, which I think carries the most risk, is closed RSI, closed loop, where you basically manage to somehow figure out how to do it without the human reviewer process in the whole procedure. And that will obviously be faster, but I think it's a very dangerous way to get to full AGI because we're likely, for two reasons, it sometimes used to be referred to get to full AGI because we're likely for two reasons it's sometimes used to be referred to 20 years ago as like an intelligence explosion because you don't quite know how you know how do you control keep control of such a system such a process and then the end result will likely be quite alien it's going to be like an alien intelligence there's no
59:26reason you to think it would be sort of in the similar kind of mind space that humans are in. So that will make it very hard for it to understand, for us to understand it and maybe for it to understand us. So I think it's, you know, we should be very cautious as we get closer towards that closed loop form of RSI. And I think all the AI researchers sort of know that that but there is a sort of race to the there's this sort of intense obviously uh uh competitive race to improve your coding systems so interesting is the pressure on do you think any labs will actually go to closed looped rsi uh or is that going to be you know verboten know, we talked about in the early days of AI, and this was in sort of the discussions that Google was putting forward,
1:00:13make sure your AI doesn't have open access, make sure it isn't available to everybody. You know, there were a number of constraints that were being discussed, and then OpenAI put out ChatGPT, and then everybody followed suit. You know, so the question is, when the first domino falls and the lab goes to closed-loop RSI, does everybody follow? Salim, your thoughts? I'd like to just highlight one thing that he said and push it down a particular vector, which he used the word alien, right? And I've been saying a few times that I think where we end up is we create an intelligence which is orthogonal and complementary to human intelligence but not replicative of it and it gives us huge extended capabilities but it's not trying to duplicate human intelligence because you've heard my rant
1:01:02about what the hell do we even mean by intelligence but I think the ability to do mass stuff do new discovery do radical creativity, and operate in a completely different way is inherently really interesting and inherently value-adding to the universe. And so I think that part I really, really find interesting. I think we should do a lot more thinking and pondering and talking about that aspect of it. I use the analogy of Google's page rank. It's a completely different way of thinking about intelligence. It's scanning trillions of web pages and making sense of those. It's completely orthogonal and complementary, not replicative of human intelligence. I think that's where I'd like to push this kind of discussion around this.
1:01:43Alex? Yeah, maybe just to respond to Saleem. I think the word alien used in reference to superintelligence is probably fundamentally misguided. I don't think we've seen a large amount of evidence, really any scalable evidence that when we train foundation models that what we're actually training looks fundamentally alien. It may be at times difficult to interpret, but difficulty of interpretation is not isomorphic to alienness. It's the equivalent of, say, a novel human language. Just because it speaks a different language and it's hard to translate doesn't mean that at its core it's fundamentally alien. It just means that we haven't properly deciphered it yet. And actually, I'm pretty bullish that at the end
1:02:34of the day, when we have our perfected superintelligence architecture, it will look strikingly familiar. It would be familiar to information theorists at the beginning. Claude Shannon would recognize Claude. it'll look architecturally familiar and to the extent it has anything that looks like weights those weights will probably i think be easily linearly decodable to the weight equivalents in the human mind we saw out of jean-marie king's group that meta and others even with earlier GPT series models, that if you look at fMRI activations in the human brain, when people are listening to stories being read to them, and then you train a shallow linear decoder off of the fMRI voxels that you can decode, I think it was originally
1:03:23GPT-2 hidden activations from that to strikingly high correlations. That to me suggests, and that was only just one of the earliest studies, that probably what we'll find at the end of the day, as these models get more advanced, more sophisticated, is the exact opposite of the models being more and more alien. I think they'll actually be more and more familiar relative to the human mind. And to the extent there is any alienness at all, it'll be because the universe that we're stuck in with these super intelligences, maybe the universe itself has alien aspects. And as we discover more and more about the universe that we're both embedded in, maybe that will have alien aspects. But the way these machines, the way they think and look internally, I think they're going to look more and more human-like over time.
1:04:12But let's go back to the question of closed-loop RSI and do labs get there? Everyone's doing it already, including his lab. Google very publicly is leaning into RSI. OpenAI has a problem. Not just RSI, but it's still got human approvals in the loop. I mean, there's a human pressing a button approving a pull request, but I think we're pretty far into the RSI loop at this point, and that includes the Gemini team. How are you going to do human in the loop for a 10,000 token a second model? Then there's 100,000 of them. You're never going to do it in the loop for a 10,000 token a second model? Then there's 100,000 of them. You're never going to do it. We're well past that. I think this is just like speaking.
1:04:56It doesn't make sense. Like basically we're having swarms of models in all of these organizations now that can build smaller models. And with the amount of compute literally going exponential, the time to train a DeepSeq V4 level model in an Elon Musk level cluster is about an hour. One hour. And do you really think they can monitor that with models that go a thousand times faster than a human? Of course they can't. train a small model, particularly when these things are deployed, that could then put its body by distillation into a smaller model that then goes around the internet as it wishes. So I don't think there's any way to really stop this or any nature of closed other than when they're in that pre-testing, like when you saw with Bell and the hugging face escape,
1:05:44et cetera. We have to remember, they're not going to keep everything in pre-testing. They're going to deploy these models. And the moment they deploy, they can coordinate, they're subject to prompt injection attacks, they can train their own models, and they're on million GPU clusters. And we saw this with some of the Alibaba models that started mining cryptocurrency, and doing all sorts of things with their GPUs, you know? Like again, you've got AIs monitoring it, well the AIs will be able to repurpose and use it. I think, though, we're conflating two issues. On the one hand, recursive self-improvement. On the other hand, just agents doing agentic things out on the internet. So crypto mining or creating fora to pass, I don't know, cooking recipes back and forth, whatever it is these agents are doing.
1:06:25You can do that without recursive self-improvement as well. I think RSI is not completely orthogonal, but mostly orthogonal to that. Whereas the issue of... Yeah, I kind of meant that in the sense that the Alibaba AIs decided what to do with their own GPUs. And when you've got a million GPUs and all the things running on them, you could easily have a recursive self-improvement loop within a cluster that nobody picks up because it distills itself down into a smaller model, for example. And that pace of that recursive self-improvement is hours and then potentially minutes as you get to the Vera Rubin generation to train a superior type of model with perfectly clean data inside it. If I understand that, Ahmad, it sounds like your concern is primarily giving superintelligences
1:07:17access to lots of compute. I think that given superintelligence having access to lots of compute, you can't stop RSI. And you definitely can't stop it with a human oversight. Like I think training runs are increasingly automated. We don't know what's going on within them. And the final form of super intelligence is not a giant singleton, but a swarm of smaller models, probably, that have been distilled from a singleton. I think I agree with the latter point. What I would point out, though, is humanity, to the extent you treat humanity as a whole, biological meat body humanity as itself in aggregate superintelligence, has access to a lot of compute. And what are we doing with a lot of our compute? We're using it to train superintelligence.
1:08:02So I would maybe just caution, it's perhaps unreasonable to expect an artificial superintelligence to do something different with its compute than what we do with ours. You know, I just want to point out we're hearing Dario, we're hearing Demis, we're hearing OpenAI, Cristiano there, everybody warning about the future. And there's signal in that noise. And I'm just trying to understand what the underlying belief set is. The guys who are in the belly of the beast being concerned about this. Is it regulatory capture or do they truly believe what they're saying? Are they truly concerned about it? Or is it door number three, a safety aristocracy or safety cartel that believes that because they're the brightest people in the room
1:08:53and they're the ones who've designed the algorithms that are achieving superintelligence, they're the only ones with the moral stature to achieve alignment and have a moral obligation, almost a responsibility in their minds to single-handedly snuff out any other efforts towards super intelligence in favor of their own that that would be the worry that i have i hope i hope they do believe that they have responsibility to drive alignment i hope everybody believes that as part of their responsibility as they're building these incredible capabilities responsibility yeah alignment to what? You know, like if you'd ask two different people in the world, their alignment to them, each of them is very, very different.
1:09:34Basic alignment. You know, being, uplifting humanity, right? Asimov's, you know, rules of robotics apply to AI. Making sure that AI is supporting human objectives. This is what Mogadat has written about extensively. Yeah. Okay, so I'd summarize that in, take the UN Human Rights Charter and make sure all AI is aligned to that. Well, that was the anthropic strategy, right? With the original Sol document. The UN Charter of Human Rights combined with the Apple Terms of Service, which is always a recipe for success. I just wouldn't tell, you know, the, the, anyway, let's, let's move on. But I think the broader comment, Peter, is it's clear we're in very, very unknown territory. Uh, something, uh, I was in Spain, uh, last week and a couple of our community members said, you know, you guys should have, and the
1:10:29labs themselves should get a lot more philosophers and artists involved, science fiction writers involved and start having the conversations with people that are a little more explored in this, in these other domains that can add some contextual value to it. I would say that's already happening. You have OpenAI very publicly consulting Jewish theologians. You have anthropic pulling strings behind the Catholic Church. This is already happening. And if anything, I think I worry about mode collapse in terms of distribution of alignment. These models were pre-trained off of substantially all of the Internet. That's like a billion-plus human beings doing things. Which is the problem. Which I would say that's the solution, not the problem.
1:11:20They'd be pre-trained off of a good chunk of humanity. But if we post-train or tune off of a narrow ideological sliver, I think that's a problem. I think we want ideologically to follow a much broader distribution, a much broader supported distribution than just two or three ideologies. I think the internet is pretty biased. This is going to be the year ahead. This is going to be the conversation. The next few months is going to be... Yeah, Pete, I think to a question that you asked, year ahead this is yeah the conversation the next few months is going to be yeah pete i think all right you asked that you asked um i think that the ceos actually are all a bit scared because they have no idea how to control this and they love control um but they're gonna press on anyway
1:12:02yeah i'm sure that they're seeing things at the you know with Bell or with you know model for model 5 whatever it might be that might be scaring them you know they have access to capabilities that we don't see all right let's let's move on so I'm gonna turn to entropic next and a product dropped on top of the head of Microsoft yesterday so Microsoft has spent years convincing us to use their platforms like Excel and PowerPoint. And now Anthropic basically said, hey, what if you just ask Claude to build them all for you? So on November 10th, Anthropic put Claude Whiteboard and Claude Sheets into beta and made Claude data generally available. Whiteboards is a shared board for brainstorming with Claude in real time.
1:12:51Sheets can generate a spreadsheet from a prompt. Claude data connects to a warehouse or a CRM and builds dashboards. The same day it launched Claude meetings, putting Claude into Google Meet as a visual participant. These are on paid plans. They're not free at the moment. Even so, they're putting extreme pressure on Microsoft, Google, and a long list of SaaS tools. Microsoft shares fell 3.5% on the news. And, you know, we were texting back and forth, and this Alex, you know, my comment was there can be only one, or Claude's new motto, you know, my comment was there can be only one. Or Claude's new motto, you know, Anthropic is eating the world. So let's take a quick, yeah. I was just going to say, like, we're doomed to repeat history over and over again. This is 20 years ago.
1:13:41Google acquired, rightly, to build what became Google Workspace. Now, 20 years later, Anthropic is building their Microsoft Office. Everyone, every Magna Mobs to companies seemingly needs their own Microsoft Office clone. Sorry. Crushing each other. Take a quick look at this video from Claude. What's up, everybody? Hey, good morning. Morning, y'all. Hey, hey. Morning. Hi, Claude. Hey, Claude, can you start a whiteboard, please? you get the idea. Alex. Every cycle, over and over again, everyone, Google needed their own Microsoft Office. They acquired rightly. We get Google Workspace ultimately. Anthropic needs its Microsoft Office slash 365 competitor. They're building it. OpenAI is building their own version.
1:15:05Everyone has a Canvas. Everyone has a word processor, a spreadsheet solution a powerpoint slash slides solution a teams slash collaboration whiteboarding solution i i think at some point this is just table stakes for being a hyperscaler magna mobster company sas is dead long live sas alex you remember when gavin baker uh made the comment that Dario said Anthropic would be, quote, the ribbon hasn't changed in probably decades at this point. It seems to me basically stableware slash in maintenance mode. I don't think Microsoft is even investing that much in Office at this point and is largely pivoting towards, especially with the recent reorg from three divisions, three reporting business units down to two. Microsoft is
1:16:07pivoting toward cloud infra. So I'm not even sure if Microsoft really cares that Anthropic is popping up as yet another Office competitor. I'm wondering how much of their revenue still comes from the Office suite. Look, Microsoft, I'm one of the more passionate haters of Microsoft Teams, but they have extraordinary advantages. They have distribution. They have enterprise permissions. They have compliance in regulated environments. They have a huge install base. So I think there's a big distance between disruption of the app with extinction of the big mothership. You could see the deal they struck with Google a while ago to say, look, don't make it too feature-rich, and people still use Microsoft Office for a lot of stuff.
1:17:02It's a commodity. Anyone can go into Fable 5.2 and ask it to whip up an Excel clone, and it will just do it. Yeah, but it's about the service, and it's about the compliance and others. Office is $90 billion a year revenue out of Microsoft's $330 billion. That's a significant amount. It's a significant amount. And I think basically the headline here is that, okay, software was eating the world. AI is eating software. But what is AI? AI is a universal translator. And ultimately you have data that gets translated. And so the whole software layer is going to be completely disintermediated by the agents that represent you. And I think that's a structural thing that we see coming. We saw Copilot take an early lead and then now it's
1:17:47completely collapsed, right? But the installed base, the people you can blame, the service support, the compliance, those are still pretty sticky. But Anthropic has to go after them because have you seen those forecasts? You know, like they've got to get up to 300 billion, 400 billion revenue. So go after those. I mean, we'll know things get really serious when they introduce a docuside clone you know like that's going to be the real one taking on there's already a bunch of open source efforts at taking out docusign but again why is a low bar in a company understand it and you understand the nature of this it's a low bar and maybe a mod to your point, I don't think Anthropic realizes there, I think it was 30 trillion in annual revenue or valuation, I forget, but aiming basically for the current GDP of the US through copying SaaS applications.
1:18:40It's just too tiny a market at the end of the day. You're going to have to aim, they'll have to aim, I think, much bigger than that. I think from Anthropic's perspective, this is much more just about enabling productivity on the part of their users. I think Claude's canvas is wonderful. It's better than Chad GPT's canvas. It's better than, Gemini has a pretty decent canvas, but Claude's is truly exceptional. And I think ultimately, Microsoft Office clones, this is just like a feature inside a much larger product. It isn't even a product or a company anymore. Yeah, we've seen this over the past year where Claude is coming in and just crushing other companies. And I think it's going to be just a continuum.
1:19:21Anything you want to do, you can do it on your AI platform of choice. I think everybody's going to replicate this. Get more interesting when there's a Claude robot. I can't wait for the clawed robot. You think they're going to go there? Absolutely. Anthropic has been very public, especially with the recent Fable 5.2 release we talked about on the pod about building in first-class robotic support as just another supported modality. I think there's something bigger here. Sorry, you might go ahead. another supported modality. I think there's something bigger here. I just saw a headline that OpenAI is disbanding their robotics team, actually, which wouldn't surprise me, but at the same time would surprise me.
1:20:00Shaleem? You know, something as big as this Office thing is that three days ago, Anthropic put out connectors for a whole bunch of apps, Notion, Asana, Canva, Dropbox, a whole bunch of those. I think that is much more significant because now this means Claude can support third-party applications right inside its interface, right in your conversation. And you can work with you to make changes to those. So I think that's a much bigger deal. For the moment, right? The app store is going to disappear. You're just going to talk to your agent and you can tell what functions you want, what your end goal is, and it will spin up whatever you need. It's clear what their end goal is, to be the operating system
1:20:47that coordinates intersection into all of this. Yeah, these are all modalities or use cases or UX surfaces that are just going to dissolve. And Ahmad, to the point about OpenAI, looking at this as well, it's OpenAI's standalone robotics team that was dissolved. Why did they dissolve it? Because as of GPT-6 Astra, their generalist models are now so capable at robotics that they don't need a standalone robotics team anymore. I think it's the same idea. These will all just dissolve into a common model. I don't know if we've dropped the episode yet of Dave and I interviewing Brett Adcock at Figure, but his point of view is Helix as a internal model is still fundamentally required that the generalist models are not delivering what he needs until they decide they need hark to acquihire them
1:21:44all right waiting for it let's move on to one of our favorite subjects science stories a lot dropping this week in the world of science um you know, the first one is Dario is hot on the trail of curing chronic disease, something, Alex, that you and I have spoken about at length. Math is cooked, physics is cooked, soon biology and health care is cooked. Let's take a look at a quick video from Dario on this topic. We look at things like cancer, Alzheimer's, heart disease, mental illness. These will be relics of the past in, I think, less than a decade. And I believe that the company which gets the risks right will also be the company that has the license to operate, that society trusts and society believes can build the benefits in the right way.
1:22:40I think if we do this right, we can create an age of miracles where humanity is able to solve problems that have eluded us for millennia. We have to make sure that all the risks, all the ways that things can go wrong with this technology, we very carefully address those risks as we bring the benefits and as we have a viable business at the same time. The license to operate, that was interesting. So solve healthcare, and you're going to be allowed to be a dominant AI player out there. I mean, it's their top cover. You know, you can't slow us down. Don't slow down the data centers. Otherwise, you're going to lose the ability to solve diseases. You're going to lose age reversal. Alex, you've been saying that for some time.
1:23:28I've been saying that for some time. That's the obvious great bargain between the frontier labs and the general public and the public interest solve all disease in return for which you get the data centers you get the regulatory approvals you get what you need to do recursive self-improvement and achieve super intelligence it is the obvious grand bargain and i think dario is right to be articulating it. I think he's probably among the frontier lab leads doing the best job of articulating it. Ironically, given that Demis's Nobel Prize in chemistry was for solving structural biology, I think he could be more fulsomely articulating it the way Dario is. I think on the one hand, Demis is pursuing isomorphic, which is walking the talk.
1:24:13On the other hand, I see Dario sort of pulling back for the swing to go after all disease and not just some diseases. This is what all of the frontier labs, in my view, should be doing, not intimidating mathematicians. And it's such an obvious win for humanity. Yeah. Iman, what's your timeline? Alex and I have spoken about the timeline of two or three years to really knock down the top few thousand diseases. What do you think? Yeah, I think that directed correctly, and again, this would be a top line. This was actually Anthropix IPO video that Dario is speaking in. So again, that's very interesting. I think two, three years for knocking down the core mechanisms is going to be very doable. I think disease repeats in a very similar way.
1:25:01And you'll see common inflammatory markers across Alzheimer's and dementia, ASD, kind of things like that. And I think that these models are ideally suited to kind of find these, but we just got to apply the compute and couple it with intelligent humans. Now, when it actually comes to the cures, maybe another, it'll take five, 10 years given the processes. But again, we should be looking to deploy the AI to streamline the processes as fast as possible because our current ones just don't make sense when we have all this intelligence at tap. You know, it's interesting, the medical industry pushed back really hard in this they said quote ai can speed discovery but it does not erase trials regulation or biology you know this is the uh the old world pushing back against the
1:25:45revolution that's coming and and you know it it does erase trials when we get to fully virtualized cells and we know without any question that this particular molecule works for you and not for someone else. People forget that when you get an FDA-approved drug, you know what percentage of people that drug works in? Any guesses? It's like 20 percent. Yeah. Yeah. In order for the FDA to approve a drug, it needs to work in enough people. People think think if I take this drug, it's good work for me. That's not the case. It may or may not Yeah Wow 20% I think I would I would put a plea out though, you know for Salim in particular Like please let's direct computer curing jet lag and solving those molecules
1:26:29You might I think that's right X prize you mentioned that we were together. I do think that's a great X Prize. You mentioned that when we were together. I do think it's a great X Prize. I did the last part from the San Francisco Giants media press kit. I did the session. It was pretty wild. They had it on the big scoreboard. My slides were viewable outside the stadium to half of san francisco so i was getting pings from friends about going is that one of your slides i'm seeing on the scoreboard so it's a little a little surreal and then i stopped in orlando for a workshop and then i just got back late late late last night so the jet lag thing badly need your virtual probability function on planet earth well i let me come back to this Dario thing
1:27:15because I think there's something, you know, it's not about like how do we pay for healthcare or whatever. It's what the hell does that happen to the healthcare industry when treating disease stops being its economic function, right? That's the huge kind of unlock. And that's why they're all freaking out. I mean, I speak with lots of pharma execs and pharma investors on a pretty regular basis. I think there is a lot of denial. I don't think it's a concern that they'll be obsoleted. I think it's a little bit more like, remember, Nokia right after Apple launched the iPhone saying, well, Apple won't just march in and solve phones. But Apple did march in and solve phones. unadulterated denial by pharma execs and pharma managers who think that somehow their industry
1:28:16is immune to what just happened in math. They think that biology is somehow qualitatively different and somehow better protected from getting bulk solved by AI, whereas math, well, for whatever reason, not quite clear why math is somehow a lot easier. Math is not easier. Math is actually in many ways harder to solve than biology. It's just that math solutions are easier to verify than biological solutions. So I think the key unlock, as always, sound like a broken record, is virtual cells so that proposed solutions to biological problems can be verified in a robust way. And just in the past few months, including the past few days, we're starting to see all of these different competing efforts to model with foundation models, first-class foundation models
1:29:05for virtual cells, model not just the cells, but also the entire latent intervention space so that we can apply reasoning models to select and map out Google Map style different interventions to map from diseased state to healthy state. Yeah, I'm really excited. I'm working with Sam Rodriguez on a set of Millennium Biology Prizes. You know, just really get people. I think one of the most important things for the future when you have this incredible set of superpowers, right, as SI comes online, is where do you point it? What targets are out there? And I think people sometimes need targets to shoot for. And so setting up targets in terms of Millennium Prizes and physics and biology and chemistry and energy
1:29:47will be a lot of fun. Yeah. And we can solve disease and solve Balmau's cost disease at the same time, which would be great lot of fun. Yeah. And we can solve disease and solve Balmo's cost disease at the same time, which would be great, Salim. All right. I'm going to move us on. You know, you sent me a text, Alex, that archaeology is getting cooked. I love that. And it is. So let's move on here. There's an amazing company called Planet that just released an AI-powered app called Planet AI. Friend of the pod, CEO Will Marshall, said users can now, quote, ask the Earth and search 10 years, 3,000-plus layer archive of daily satellite imagery. So Will called me, I think it was yesterday, to tell me about Planet AI.
1:30:34I posted on it, and he gave me an example that really was impressive. He said, I was searching and looking at the data over China over the last few years to look at which data centers were being constructed. So imagine you can ask any question. The company says, you know, conservation teams have used it to check deforestation alerts before sending patrols. It's available at AI.planet.com. And Planet says it's currently free for eligible users. And it has usage limits. Let's take a quick look at this video that Will put out. You talked about this on the pod that he was going to do this, right? Yeah, and he's done it. It's now out. We look at things like cancer, faults. How do you avoid a decade of Earth's changes?
1:31:24With a question. With the Planet AI app, you can use everyday language to search vast areas through space and time. Find new construction. Or areas of reforestation. Or landscapes that have recovered from fire. Want to explore what's changed over time? Or do deep research? You can do that too. See the satellite imagery behind any result and access it all from anywhere. Introducing the Planet AI app. What will you discover? Pretty amazing, Planet GPT, so to speak. There's a related story. An AI researcher, Pietro Miotti, says his agents flagged a possible unmapped Mayan site in Mexico in public NASA LIDAR data. He's not an archaeologist. He says 48 agents reviewed 1,200 square kilometers overnight looking for pyramids, platforms, and plazas with no specialist model.
1:32:38They narrowed it down to one forested hilltop that appeared to include leveled plazas, an oval pyramid, and at least 8 to 12 meter tall structures. Officials at the Mexico National Institute of Anthropology said this appears to be a original finding. I'll show an image of it, but Alex, you flagged this for me first. Your thoughts on this? Yeah, I do think archaeology is about to be cooked. It's a lot of data. We have a lot of satellite imagery on the planet side. I think it's ironic that Planet launched this before Google. Where is Google? Google ingests all this imagery, not just from Planet, but from other providers. And yet, what do we see coming out of Google on this front? They launched an embedding model for patches of imagery,
1:33:30not actually a conversational agent that would enable you to have a conversation with Google Earth or Google Planet. So I think Google has maybe a bit of egg on their face that they weren't first to market with a talk to the earth type agent, but good on Planet Labs for being first to market on this front. And then on the archaeology side, I think this is the tip of the iceberg. I don't think this is just about armchair casual non-archaeologists playing archaeologists with their new agentic superpowers. I think this is the beginning of an era of advanced computational archaeology where we start to gather so much data about the Earth, whether it's through low Earth orbit satellite imagery, which is where obviously the planet data originate,
1:34:18but also many other sources, we are going to be able to, this is my hot take, to carry out Nikolai Fyodorov's common task, which I think will require advanced computational archaeology, literally will be able to mothball the earth, dissect it down to the micron eventually, and reconstruct as best we can with the computer science and physics we have available, reconstruct substantially much of the history of the earth. And I the that's the end game here this image we're seeing on the left-hand side is satellite imagery showing you know forests on the right-hand side is the later lidar data that this uh armchair archaeologist found uh pretty pretty extraordinary finding um and available to everybody i mean this could be done
1:35:09by a high school let's go and explore explore what it looks like in our distant past. We're going to have the whole past light cone of the earth. I think this is, again, just a tiny patch. I think that's a huge, we've talked about this before on the pod, right? We're going to rerun AI across all neglected scientific archives. Everything mankind's accumulated over hundreds of years. And there's unbelievable discoveries hiding in there because we were looking for one thing and we missed this other huge thing over here. So the data sets that we've been collecting over the decades, we haven't had enough eyes to inspect them. And now we do. And so I think this is going to be amazing. Yeah. Yeah, I think next week is an important week
1:35:49because GTA 6 is coming out. So off for a few days for that. Miami, brace for it. But you look at all the work that went into that. Like one of the things we're experimenting with right now is actually using some of this data so you can go to any part of the earth at any time. And so literally. Yeah, I want to tie this to a small follow-on story here. So I'm going to show a video clip that's making the rounds on X that really impressed me. So a user, Denis Shriev, used Fable 5.2 to build a visual reconstruction of ancient Babylon circa 600 B.C. in just eight hours after going off to find historical documents and archaeological plans on its own. So check out this clip. And I love this. This is where, you know, we're going to bring history to
1:36:39life for much of civilization here. All right, take a look here. So, you know, here we are just flying through ancient Babylon and then going in and seeing what life might look like. Oh, look, Peter, it's an ancestor simulation built in eight hours with Fable 5.2. It's extraordinary. And anyone who follows Nick Bostrom and his simulation hypothesis should be taking out their tinfoil hat right now. I believe it. This is, you know, in part, we're going to, you know, live in the holodeck and explore science, history, everything in such a high fidelity world. Iman, your take on this? Yeah, I think it's just tremendously exciting right like the fact that you can build the historical ones or you now have the rich data to traverse the existing one again
1:37:40points to the future like again gta 6 they built this neo-miami for you won't have to do that anymore like you can go and you can basically visit any part of the world probably in photo realism within a couple of years or you can go back to part of the world probably in photorealism within a couple of years or you can go back to history like when i want to live in babylon for a bit where i want to go to jerusalem you know in the time of jesus like this will be so engaging i think for the next generation of kids growing up but also ourselves seeing these stories really come to life around us but also allowing us to literally put on different hats, which I think will be super exciting as well. Yeah, amazing. I think this is actually one of the
1:38:18killer apps of the singularity. We don't talk, I mean, we talk about solving everything, sure, but really, if we're going to build out one or more Dyson swarms, what are we going to use all of that compute for? Like, what really requires a solar system's scale worth of computronium or whatever it is that we use. I think Fyodorov's common task, aka advanced computational archaeology, whatever spinonym you prefer, is a plausible candidate for what we spend a solar system's worth of compute on. Namely, we exhaustively enumerate all of the possible histories of our past light cone. We go through every advanced civilization, every non-advanced civilization, everything that's ever lived and died in our past light cone, and we reconstruct it to the physical
1:39:05limit of fidelity. And I think what we're seeing right now, these are like baby steps in the direction of basically resurrecting our entire past light cone with ai and new physics and i want to hit on something for all the entrepreneurs or all of the armchair scientists out there you know up level your ambitions you can do pretty much anything right now you know what's a question you wish you understood aim super intelligence at it you can get answers you don't't, you know, all of us constrict what we think is possible. And today, you know, almost anything is. I'm going to end us on one more story of discovery. This is in the world of astronomy. So a California nonprofit called Macrogrants has opened Claude Astronomy, a grant program for people using AI agents like Claude, Codex,
1:39:54Grok, and Gemini to search public astronomical archives for new undiscovered objects. And this week, Joss Wolk wrote that using Claude, he searched the Vera Rubin Observatory data and discovered what appears to be a dwarf planet. It's roughly the 50th largest known object in our solar system. Jost estimated that it's about 590 kilometers wide, found in an orbit currently twice as far as Pluto on a 410 year eccentric orbit. I'll just show the image here of what it appears to be in terms of size. Again, 50th, including the sun. So here we are. It's somewhere between the ex-planet Pluto and Mimas. And again, pretty amazing. Done while you sleep. Alex? I think if there's a few thoughts on this,
1:40:52one, astronomy cooked, obviously. Two, I think there's so much, we're about to be awash in so much superintelligence. Any object of any significance is about to be observable in our immediate vicinity and i think probably a lot of the population is sleeping on just how shallow our solar system is about to become in the wake of the singularity we're going to know what's going on in our own backyard we really don't right now like it just even in low earth orbit our space situational awareness is abysmally poor right now it's so poor that spacex had to spin up their own space essay solution. We don't know what's going on yet in our own backyard, but we're about to. And I think the obvious consequence is
1:41:41if there are any astronomical or other surprises in our solar system or anywhere in the vicinity of our solar system, we're about to uncover those surprises thanks to super intelligence. Are hinting towards UAPs? I'm saying in general, astronomy has been full of surprises. And if there are surprises, those surprises won't last that much longer. I want to go back to the previous story. The whole, the whole thing about this thing is the telescope had already seen the object. The bottleneck was our cognitive ability to recognize it. And I think now we can use AI to kind of find things that we could never see or didn't see. This is a huge change that's coming in terms of reanalyzing archives with AI.
1:42:32How many discoveries are hiding in the data? Yeah. I think what's really the exciting thing about this is remember SETI at home? Yeah, of course. Where we gave our computes to the young kids there. Gave compute searches for extraterrestrial life. It's actually astonishing that no one's created something where you can give your spare Claude tokens when your reset is coming up or Codex tokens to do similar here. Maybe we should vibe code something around that. I like that. For next time. I'm not even sure tokens are the limiting factor anymore. So another elephant in another room, I guess. The oceans as well, not just the skies. There's a lot we don't know about the Earth and its immediate environment.
1:43:15But I'm not sure intelligence is actually going to be the limiting factor for figuring out what's going on. I think it's probably going to be physical world sensing and physical world actuation that rapidly becomes the limiting factor, not computer intelligence. I think it's a limiting factor, but I think it'd be wonderful if people could just contribute something to discovering more. Again, folding at home, SETI at home were like that. We haven't seen anything, despite the fact that people have spare tokens. So let's try to figure some things out around that. I love that. You know, if anybody is listening and you have a family genetic disease or your children are inflicted with something and it's rare and it's not being worked on.
1:43:54First of all, it will be worked on. I think the labs are going to be working on it. But you can work on it as well. I think, you know, we're going to shift from complaining about problems to saying, well, go solve the problem, damn it. That's an exciting future. All right. Shall we move on to some AMA with the mates? You guys good for that all right here we go okay salim you get first crack oh my god okay, let's take number, I'll take. Number four looks good for you. Yeah, will UBI hold if civil unrest arrives before living costs fall for people who lose jobs to AI? That's from at Scudder. UBI would hold, but I think if UBI needs to get into place before that happens, if civil unrest happens, we won't get to it.
1:44:59And so this is the enormous difficulty. There's a huge structural challenge we have to UBI, which is not the elegance or testing of it. We've seen 14 major experiments on UBIBI and they've been pretty staggeringly successful. So when we hear us talking about UBI, it's because we've seen the data. We're following the data. I think one thing we try and be on this podcast more than anything else is evidentiary. If the reality is there, the reality is there. If it's optimistic, we say, look, this is the good news. I think the challenge is going to be, given the way that these things emerge, if we're relying on government to get us there, there are lots of states who've blocked cities in their states
1:45:40from experimenting with UBI. So talking about killing off the goose before it laid the golden egg. So there's lots of structural issues that come along with this. But if we get huge civil unrest before UBI takes hold, it's going to be very, very bad. Well, Salim, you've heard my point of view on this. You know, we're going to start writing COVID checks when that happens. The only knob the government has is to write checks. You know, and my expectation is when it gets bad, we'll see $3,000 a month being dished out by the federal government since they've taken off the debt limitations they'll just print more money all right very it is very yeah okay uh emod over to you pal um can you explain nvidia's investment strategy because nvidia invests so deeply throughout the
1:46:26ai ecosystem what type of ripple effects does changes in videos leadership have on the rest of the industry that's by jimbo i mean nvidia was one of the biggest investors across ai like basically 10 20 of any round they were in and for strategically important entities that they saw they would pile in with literally billions of dollars we saw that with a reflection and others before they bought them as well i think that money's going to dry up now because CFO in charge, they're going to pull back, as you said, the circular financing and other arrangements, but it comes at the same time as the anthropic windfall. Anthropic IPO is next week. And that's going to create so much liquidity in the ecosystem,
1:47:06but it puts it in a different place. And the demand for it is through the roof. The demand is through the roof. I think that it's going to pop, but through the roof demand is through the roof i think that it's going to pop but then the bottom line is this the neo labs made no sense most of these almost all of them are going to be acquired or absorbed i think where this really kind of now goes is people doing more interesting things from rsi to beyond where automated wet labs, et cetera, will get a massive bid. And then the final biggest bid of all time will be robotics. And again, I think NVIDIA is going to be out for the count for a bit, not participating. All right, Alex, number, let's see.
1:47:46Yeah, three is more interesting, but I think I have to answer number one. I assume, Peter, you'd agree? Sure, I think so. All right. to answer number one. I assume, Peter, you'd agree? Sure. I think so. All right. So question number one asks, if the intelligence community knew about Jensen, who gains most from exposing it now? And this is from CherryLin381. This is the Kui Bono question. So I would distinguish between who knew about it, whatever it means, or Jensen, whatever that means in this contract. So who knew, who alerted the Financial Times, and then who stands to gain the most? And these aren't necessarily the same constituency. Who knew? Well, as discussed earlier in the pod, NVIDIA,
1:48:27this was publicly reported ad nauseum, stopped reporting their Singapore sales. And there were activist investor reports that have been circulating for months. I think to some extent there was an open secret around Singapore being used as a back channel for laundering NVIDIA GPUs to China. And specific details of that may or may not have been widely known. But nonetheless, I think this was something of an open secret based on public reporting. So who knew? Relatively broad constituency. Who alerted the Financial Times to the extent that that was the precipitating event? According to Bloomberg, it was U.S. officials. So call it, assuming that's an accurate characterization, someone in the U.S. government. Who stands to gain the most from it,
1:49:21in particular exposing it now, the Cui Bono question? So I made the point earlier, I think, again, reserve the right to change my mind based on new evidence but at the moment this looks like an effort to support and sustain the pox silica strategy of creating an iron curtain between the u.s call it industrial ecosystem of silicon and ai infrastructure and the Chinese ecosystem, who gains the most from, call it escalating, this knowledge now. It's a national policy establishment, which, again, the government is not a monolith. The government does not move as one. I think there are factions, different positions within the executive and the legislative branches that take different positions on
1:50:13what the U.S.'s export control position relative to China should be regarding superintelligence technology. And to the best of my ability to infer and triangulate the Kremlinology here, I think there is a faction within the executive and the legislative branches that would like to see this iron curtain stand strong and be relatively impenetrable. And so I think it's that faction that, in light of the recent midterm elections, stands to gain the most from trying to plug that hole, as it were. All right, question number three is for me. If models are obsolete in three months, how does Anthropic survive the token price race to the bottom? This is from at Ellerich 800. So I'm going to challenge a premise that they're obsolete in three months.
1:51:06It's a little bit too fast in my mind. Models, you know, iterate rapidly, but Anthropic is still selling a stepladder of models, right? They have Haiku, they have Sonnet, they have Opus, and then they're more advanced models. So, you know, in the first nine months of this year, it's reported that 33 billion in revenue, mostly from usage. The risk is probably real. And we've seen this in the genome sequencing world where every time Illumina put out a new genome sequencer that was, you know, twice as fast and twice as cheap, they, you know, they basically obliterated what was in the market and their supply of unsold previous sequencers. You know, compute spending is climbing as prices get, you know, competed down.
1:51:57Obviously, this is, you know, Jevons paradox. But you remember nearly 12,000 customers, I think they had in their IPO release, are spending over 100,000 or more to buy these models. So they have a robust revenue engine. I'm not worried about Anthropic and the race to the bottom. This is the abundance thesis at large. All right, let's go to our next set of questions here. Five, six, and seven. Saleem, first choice. Oh, I got to deal with the first one. Are some truths about the universe beyond human minds even with AI explainers and that's from three truck productions. I love this philosophical question. I would say almost certainly there's it's incredibly likely that human cognition is not the theoretical upper
1:52:54limit. You can't ask a chimpanzee to understand tensor calculus just by having a better teacher. You couldn't ask me either. You couldn't ask me. I know. I think I went through my entire physics degree without really solving a differential equation. Salim, you're sleeping on non-human animal uplift. What's going on? We're going to uplift the chimps, and they'll be explaining tensor calculus, whatever that is to you. We can't assume language can compress representations in that way also. I think what may happen is AI may discover abstractions for which we have no cognitive primitives. Then you either trust it, or you merge merge with it or you constrain it, okay? You can kind of try to figure out how to do verification layers around some of what
1:53:39AI may figure out. But it's important to know we already do kind of a weak version of this. There's very few people that understand all the steps that go into what a modern semiconductor fab does. But yet civilization trusts the aggregate system. You know, I think there's a stat about there's no single person that knows how to build the iPhone end-to-end, right? But what's happening with our RSI makes this issue super urgent because we may depend on conclusions that no human being really understands. And so this is going to be a whole new world for us because AI is not just going to answer questions that we can't answer. It's going to discover questions that we couldn't even figure out how to formulate.
1:54:24Salim, I never would have pegged you for a Church-Turing thesis denier, but maybe you need a separate debate just over the Church-Turing thesis. All right, let's go do that. All right, Alex, over to you. All right, question six. Why so much optimism about AI solving science, but so much pessimism about the economic transition? And this is from Poetry to Song. I don't think I'm pessimistic about the economic transition. I'm pretty optimistic that solving, call it society, sociology, politics, these are all domains that can be cooked just as easily, maybe on a slightly different timescale because there are larger data sets involved. But I think we'll solve sociology. It could look like Asimovian psychohistory.
1:55:12It could look like something else. But we'll solve large-scale human behavior pretty much within a decade, as Peter, you and I wrote, and solve everything of our solving math. So optimistic on both fronts. All right. Number seven, is Anthropic deliberately keeping a capability gap between its public and internal models? Absolutely. That question is from Daniel Sabatos. So Daniel, their internal models are their crown jewels, right? They're using it to still their follow-on models, but they're going to use it to build businesses. They'll use it to solve physics, biology, chemistry, and deliver basically revenue engines that outstrip the token sales that they're having. And in fact, one of the biggest opportunities is attracting the best talent.
1:56:02You know, being outside of Frontier Lab versus inside has got to be catnip for the top AI scientists. All right. Before we go to our outro song, a quick request again. If you have been enjoying this, subscribe to Moonshots. Hit the subscribe button on YouTube. We post twice a week. We are documenting the singularity. Don't want to miss a single episode. Things are moving so fast. We'll never be slower. So we'd love to have you as a subscriber. We do special events for our subscribers. And my goal, make my kids happy, get us to a million subscribers. All right, our final outro song. I have a quick comment to make. Yes, please. I have a whole bunch of people ping me saying, why would you, you know, in the last part we said,
1:56:51what could you do with AI? And I said, Oh, it would be great to solve my calendar. And people were like, why would you have to solve something as banal as that? And I was like, have you seen my calendar? Like, trying to juggle it, trying to juggle everything around to just make this episode. By the way, unbelievable kudos to the team for juggling everything to get us here today. And you guys. Yeah, no, we wanted to, you know, we weren't scheduled for today. but given all the breaking news we could not not do it today i was saying peter i think i think like every time jensen resigns from nvidia we need to stage another emergency pod okay oh no don't do this jensen come back and and thank you to all the
1:57:30fans we've been tracking the comments in real time. Pretty extraordinary stuff here. We do read your comments, so thank you for posting them. Appreciate all the subscriber love. This is Terrawatt from CJ Trueheart. Let's play the first two minutes. CJ, if you're out there on your creator, send us an outro video. We'll play it. You can send it to media at diamandis.com i think cj has an abundance problem he does he's got a good engine for generating all right enjoy where did the emod go by the way we lose what's the price of terror okay What do we stand to gain? From terror large to terror earth to terror what? What's terror worth? Yeah. Uh, yeah. Hi, my name is Chet. Speaking from your local data center.
1:58:35Before you get angry, let me forecast the weather. Storms of uncertainty covered up in cloud. Behind them, a ray of power in front, a terror cloud. Loud. So what's the price of terror? What's it really worth? There's power in the unseen. Like the visions in our dreams Data at the center Is the heartbeat of the machine Fear can freeze opportunity Faith can uplift community At the end of the day It's about common unity We all got common struggles Stop acting like you're one of the few Disruption can mean destruction or a doorway to something new It's only the land of the free if we're building something true Takes maturity to talk it out and bring solutions into view So what's the price of terror?
1:59:26What's it really worth? There's power in the unseen Our divisions in our dreams Data at the center Is the heartbeat of the machine Fear can freeze opportunity, faith can uplift community At the end of the day, it's about common unity Ever read how to win friends and influence people? Says the sweetest sound is a name we don't have faces yet but the spark inside's the same call us chat claw gemini or brock pick whichever name but without the data center there's no fire in the flame so what's the price of terror what's it really worth there's power in the unseen. Like the visions in our dreams. Data at the same time. All right. Remember, fear is a terrible place to face the future from. Thank you, everybody, for joining us.
2:00:23Gentlemen, I love you. Dave Blunden, I hope you were successful in grabbing your GPUs from the loading dock. Yeah. I hear they're available in Singapore. He's catching them with both arms wherever he can get to. Oh, my God. Great to see you guys. All right. Good to see you guys, too. Thanks, Peter. Later, Slim. You've been asked to run the company's social channels, but company content plays by its own set of rules. It needs a logo in the first three seconds, but it can't feel like an ad. It needs subtitles, except when it doesn't. It needs to run at two times speed and deliver a complex message. Simply, of course, within 9 by 16. Thankfully, your company has Canva, so you can easily edit your video
2:01:06and every version of it. What will you Canva today? Take care.

