Most Replayed Moment: How AI Companies Are Gaslighting You - Karen Hao

0:00It takes a little bit of time to realize this as a founder, but once you do, it's something you can't unsee. That having the right systems in place with your business is critical. One that's helped me across all of my investments is NetSuite, the number one cloud financial system and sponsor of today's conversation. Through their streamlined platform, you'll find all of your accounting, finance, management, inventory, and HR in one place. And their technology has been a massive unlock for all of my companies, especially for my team at FlightStory. As over the last year, we've moved out of startup mode into scale-up mode. And that's the stage where we found it mission critical to use NetSuite.

0:33We no longer have to juggle multiple systems and having everything together has reduced the number of manual tasks and of course, the number of errors. Over 42,000 businesses have chosen to future-proof their business with NetSuite. So if you'd like to learn how it can help your business, head to netsuite.com slash Bartlett and free download the CFO's Guide to AI and Machine Learning. That's netsuite.com slash Bartlett. What I find really interesting is to look at how people's quotes evolve over time with their incentives. So I was looking at all of the things they've said on the record on podcasts, in their blog posts, to see how it's evolved over time. And Dario, who was the former VP of Research, OpenAI, and has now moved on to Anthropic, who are taking a slightly different approach to developing AI,

1:19said back in 2017, while he was still at OpenAI, that, this is a quote, I think at the extreme end is the Nick Bostrom style of fear that an AGI could destroy humanity. I can't see any reason in principle why that couldn't happen. My chance that something goes really quite catastrophically wrong on the scale of human civilization might be somewhere between 10% and 25%. And also you mentioned Ilya, who was a co-founder of OpenAI and then left. I guess the first question I'd ask is, why did Ilya leave? That's a great question. So he was instrumental in trying to get Sam Altman fired. And he's another one of the people who over time began to feel like he was being manipulated by Altman towards contributing something that he didn't believe in.

2:10How did you know? Because I interviewed a lot of people. Ilya, in particular, had two pillars that he cared about deeply. One is making sure we get to so-called AGI. And the other is making sure that we get to it safely. And he felt that Altman was actively undermining both things. He felt that Altman was creating a very chaotic environment within the company where he was pitting teams against each other, where he was telling different things to different people. Have you ever spoken to him? I have. So I interviewed him in 2019 for a profile that I did of OpenAI for MIT Technology Review. And back in 2019, he has a quote where he says, the future is going to be good for AIs regardless. It would be nice if it was also good for humans as well.

3:00It's not that it's going to actively hate humans or want to harm them, but it's just going to be so powerful. And I think a good analogy would be the way that humans treat animals. It's not that it's going to actively hate humans or want to harm them, but it's just going to be so powerful. And I think a good analogy would be the way that humans treat animals. It's not that we hate animals. I think humans love animals and I have a lot of affection for them. But when the time comes to build a highway between two cities, we are not asking the animals for permission. We just do it because it's important to us. And I think by default, that's the kind of relationship that's going to be between us and AI, which are truly autonomous and operating on their own behalf.

3:30And that was in 2019, the year that you interviewed him. One of the things that I feel like we should take a step back to examine is going back to this idea of what even is artificial intelligence and what do we mean by intelligence. And a huge part of the views of the different people in the quotes that you're reading derives from a specific belief that they each have in this question of what is intelligence, what constitutes intelligence. For Ilya, he has throughout his research career felt that ultimately our brains are giant statistical models. This is not something that, you know, we actually know, but this is his own hypothesis. Also the hypothesis of his mentor, Jeffrey Hinton, who also was on this podcast.

4:17This is why they have such a strong conviction in the idea of building AI systems that are statistical models and that this particular approach is going to lead to intelligent systems as we are intelligent. It's a hypothesis that they have. It's not one that has been proven by science. And some people vehemently disagree with them on this particular thing. But if you step into their shoes and take on that hypothesis and assume that it's true, that our brains are in fact statistical engines, and that these systems that they're building are also statistical engines that they're making bigger and bigger and the system will become equal to human intelligence and then maybe exceed human intelligence is relevant in their framework. And Ilia gave a

5:13talk at one point at this really prominent AI research conference that happens every year called Neural Information Processing Systems. It's a mouthful. But he gave this keynote where he shows this chart of the size of brains and the intelligence of a species. And it's roughly linear. The bigger the size of the brain, the more intelligent the species. And so for him, he thinks he's building a digital brain because he thinks brains are just statistical engines. So from that logic, it's like, okay, if we then build a bigger statistical engine than the human brain, then based on this chart, it will be more intelligent and then we will be subjected to the same treatment that we've subjected animals.

6:02But it's really important to understand that these are scientific hypotheses of specific individuals within the AI research community. And there's a lot, a lot of debate about whether this is in fact the case. And some of the biggest critics say it's very reductive to think of our brains as simply just statistical engines. Why does it matter to know the mechanism? Is it not just important to know the outcome, which is that it's going to be able to do, make a video for me, or agents are going to be able to do the work that I do? Does it really, really matter for us to know the mechanism behind it? Yes and no. So it matters because these companies, they are driving their future actions based on this hypothesis.

7:06So they have decided, we think that this hypothesis is true, like we should just continue building larger and larger statistical models in the pursuit of artificial general intelligence. And that's then having global consequences. Like in order to continue doing that, they're hoovering up more and more data. They're building more and more data centers. They are having, they're, you know, exploiting more and more labor in order to continue on this path. Here's a question that I think is important to ask is, why are we trying to build AI systems that are duplicative of humans? We're kind of having this conversation right now where we've just taken the premise of this industry as a good thing.

7:34Like they said that we should be building AGI, so we say that we should be building AGI. But I would like to ask, like, why are we doing that? Why is it that we are building a technology that is ultimately designed to replace and automate people away? That is not the enterprise of technology. Like, we should be building technology. And the purpose of technology throughout history has been to improve human flourishing, not to replace people. And so this is like a critical part of my critique of these companies and these scientists that have just adopted this goal and have relentlessly pursued it and have had enormous capital and enormous resources to pursue it, is, is this the right goal? Like, why are we doing this? Why can't we just build AI systems

8:25that do things like accelerate drug discovery and improve people's healthcare outcomes, which are systems that have nothing to do with the statistical engines that they're trying to build to duplicate the human brain. So why are they doing it? I mean, you've interviewed all these people. I think it's, what, 300 people in total? 80 or 90 of them from OpenAI, the maker of ChatGPT. Why do you think they're doing it? I think it's because they're driven by an imperial agenda. And that is why I call these companies Empires of AI. What do you mean by an imperial agenda? What does that term mean? Empire is the only metaphor that I've ever found to fully encapsulate all of the dimensions of what these companies do

9:05and the scale that they operate and what motivates them to do what they do. And there are many parallels that you see between what I call the empires of AI and the empires of old. They lay claim to resources that are not their own in the pursuit of training these models. That's the data of individuals, the intellectual property of artists, writers, and creators. They're land grabbing in order to build these supercomputer facilities for training the next generation models. Second, they exploit an extraordinary amount of labor. They contract hundreds of thousands of workers all around the world, including in the US, to ultimately make these technologies. We can talk about that more. And they also design

9:47their tools to be labor automating so that when the technologies are deployed, it also affects labor rights because it erodes away labor rights. And this is a political choice that they have. Third, they monopolize knowledge production. So they project this idea that they're the only ones that really understand how the technology works. And so if the public doesn't like it, it's because they don't actually know enough about this technology. They do this to the public. They do this to policymakers. And they've also captured the majority of the scientists that are working on understanding the limitations and capabilities of AI. You think they're gaslighting the public in a way? They are, yeah.

10:25So if most of the climate scientists in the world were bankrolled by fossil fuel companies, do you think we would get an accurate picture of the climate crisis? No. And in the same way, they employ and bankroll, the AI industry employs and bankrolls most of the AI researchers in the world. So they set the agenda on AI research in soft ways, simply by funneling money to their priorities so that only certain types of AI research are produced. But they also will censor researchers when they do not like what the researcher has found. And so I talk about the case of Dr. Timnit Gebru in my book, who was the ethical AI team co-lead at Google when she was literally hired to critique the types of AI systems that Google was building. She then co-wrote a critical research paper that was

11:19showing how large language models specifically were leading to certain types of harmful outcomes. And in an attempt to try and stop this research from being published, Google ended up firing Gebru and then fired her other co-lead, Margaret Mitchell. And so they control and quash the research that is inconvenient to the empire's agenda. Did you have an example where this is happening to journalists as well, that are asking questions of their team members? I think I was watching a video of yours where there was a young man that was saying he had someone show up at his door, knocked on his door and asked for information, emails, text messages, and this person was from one of the big AI companies.

12:05This was opening, I started subpoenaing some of its critics, emails, text messages. And this person was from one of the big AI companies. This was opening. I started subpoenaing some of its critics. Yeah. As part of a, what appears to be a campaign of intimidation, but also what appeared to be a campaign of fishing for more information to figure out, to map out the network of critics further. But this was a man who runs a small watchdog nonprofit, and they had been doing a lot of work during that time to try and ask questions about OpenAI's attempt to convert from a nonprofit to a for-profit. Ultimately, OpenAI was successful in that conversion, but during the period where it was sort of existential

12:44for OpenAI to complete this conversion, successful in that conversion. But during the period where it was sort of existential for open AI to complete this conversion, there were a lot of civil society groups and watchdog groups like Midas who were trying to prevent the process from happening in the dead of night. They were trying to get more transparency. They were trying to have more public debate about this because it's unprecedented. And it was then that there was a knock on his door and he was served papers. What did the papers say? The papers asked him to reproduce every single piece of communication that he had had that might have involved Musk. So this was like the strange paranoia that opening I had that Musk

13:24was somehow funding these people to block the conversion. None of them were actually funded by Musk. So in this particular case, their request, he simply was just answered, you know, I don't have any documents because this doesn't exist. So going back to this point of empires, you were saying that one of the factors of an empire is a land grab. then the next one was was labor exploitation labor exploitation the third one controlling knowledge production and one of the other ones that's really important to understand about the ai empires in particular is empires always have this narrative that they they say to the public like we're the good empire and we need to be an empire in the first place because there are also bad empires in the world. And if you allow

14:12us to take all the resources and use all of the labor, then we promise we will bring you progress and modernity for everyone. We will bring you to this utopic state akin to an AI heaven. But if the evil empire does it first, we will descend into a hell. And the evil empire being in this case? In this case, most often it's China. But actually in the early days, OpenAI evoked Google as the evil empire. So all of their decisions were about, we need to do it first because otherwise Google, this evil corporation that's driven by profit, us as a benevolent nonprofit, like this is a critical contest of who wins. Do you think the people building these AI companies believe that the outcome is going to be all good now? Do you think they think

15:06that it's going to be, it's going to serve everyone, it's going to be the age of abundance, everything's going to go well? What do you think they believe? What do you think Sam believes? So this is so funny, is such a core part of the mythology that they create around the AI industry includes the belief that it could go very badly. It goes hand in hand. Like they need that part of the myth in order to then say, and that's why we need to be in control of the technology, because that's the only way that it's going to go really, really well. And Altman has said publicly, you know, the worst case lights out for everyone, but best case, we cure cancer, we solve climate change, and there's abundance.

15:48And Dario Amadei, same kind of rhetoric. He's like, worst case, catastrophic or existential harm for humanity. Best case, mass human flourishing. So this is like two sides of the same coin. They have to use both of these narratives in order to continue justifying an extremely anti-democratic approach to AI development, where there should not be broad participation in developing this technology. They must be the ones controlling it at every step of the way. Sam Altman did a tweet saying, There are some books coming out about OpenAI and me. We only participated in two of them, one by Kesh Hagee Keech Hagee focused on me and one by Ashley Vance on OpenAI he went on to say no book will get everything right, especially when some people

16:39are so intent on twisting things but these two authors are trying to you quote retweeted that tweet from Sam Altman and you said, the unnamed book Empire of AI is mine. Do you believe that tweet from Sam Altman was in reference to your book? 100% because there's only three books coming out about him. And he'd caught wind that your book was coming out and... He knew my book was coming out because I had contacted OpenAI from the very beginning of my process and said, I'm working on a book now. Will you participate in it? And actually, initially they said yes, even though... So my history with OpenAI, I profiled the company for MIT Technology Review. I embedded within the office for three days in 2019.

17:23My profile comes out in 2020. The leadership are very unhappy. And in my book, I actually quote an email that I received that Sam Altman sent to the company about my profile to me was, we are not going to participate in anything that you do. We are not going to respond to anything, any questions that you receive. And this was, you know, this was things that they explicitly articulated. It wasn't like me inferring. So I had a colleague at MIT Technology Review that also covered AI. And at one point opening, I sent him this press release being like, we would love for you to cover this story. And he was like, I'm really busy. Will you send it to Karen? And they were like, oh no, we have a history you understand. And so for three years, they refused to talk to me. But then I ended up at the Wall Street Journal, where if they felt a bit compelled because it was the journal to reopen the lines of communication. I would always ask them, like, will you sit for interviews?

18:48And we did get to a more productive relationship. And then I embarked on the book. So I left the journal to focus on the book full time. And I told them right away, I'm working on this book. I want to continue this productive conversation where I make sure I reflect openingAI's perspective in the book. And so they were like, we can arrange interviews for you. You can come back to the office. We'll set up some conversations. And then as we were going back and forth on this, the board fires Sam Altman. And that's when things started going kind of south because the company started becoming very sensitive to scrutiny. And so then they started pushing, kicking the can down the road, down the road, down the road.

19:32And I kept saying, hey, when are we rescheduling this? What's going on? And then I get an email saying, we are not going to participate at all. You are not coming to the office. You're not doing interviews. And I had actually already booked my tickets. So I was already going to fly to San Francisco to have the office. You're not doing interviews. And I had actually already booked my tickets. So I was already going to fly to San Francisco to have the interviews. And so then I told them, I was like, that's fine. I will still engage in the process. We'll give you extensive requests for comment. I'll ask through my reporting, I'll keep you updated on all the things that I'm finding so that you can choose to still comment. I gave them 40 pages of requests for comment and I gave them over a month

20:11to respond to all of that. So this was when the tweet came out was we were doing all this back and forth trying to, and that's when Altman tweeted this. Hmm. And they never responded to a single one of the, one of the 40 pages. Sam Altman does a lot of interviews. Yeah. You know, he's doing a lot of interviews all the time. He's done every podcast. I've seen him on everything from Tucker Carlson to, I think he's done Theo Von, Joe Rogan, podcasts all over the world. I wonder why he won't do mine. Well, maybe. um it podcasts all over the world i wonder why he won't do mine well maybe i don't know why i i don't know i think i'm fair with everyone i just ask i just ask questions i genuinely care about i don't come in with huge preconceptions or at least

20:57meet people for the first time but i've heard through the grapevine um that he doesn't want to do mine i mean going back to what you were saying earlier, that with this, the way that OpenAI and these companies control research, you asked, do they also do this with journalists? I mean, yes, the answer is yes. And apparently they also do it with anyone who has, you know, a broad mass communications platform. It's not just about the conversation that you're going to have with them. It's about who you also choose to platform. And there's this huge problem in technology journalism where companies know that a really big carrot that they can give to technology journalists is access. And they will withhold that access at

21:47the drop of a hat if they catch wind that you're speaking to someone that they didn't want you to speak to. This is so true. And I don't think the average person really truly understands this. Yeah. So this kind of sounds like theory as you say it, but I'm not going to name names here because I don't think it's important. But there is a particular person in AI whose team have basically dangled the carrot of them coming here for like 18 months. And I'm like, you don't have to dangle the carrot. I'm going to speak to whoever I want to regardless of the carrot or not. And when this person comes, if they want to come, I'll give them a fair shot. I'll ask them all genuinely curious questions about what they're doing, their incentives.

22:26I won't gotcha them. I don't have a history of ever gotchering anybody. Even if I have a different of opinion, I'll ask the question. But they dangle carrots and they say, well, if he's thinking about it, let's think about a date. And what the strategy is, and I don't think they think most people don't understand, is if we just dangle it for long enough, then they will perform in the way that we want them to do. And they'll be pleasant about us. They won't be critical. They won't give a voice. They won't platform our critics. And I think a lot of their game is just dangle the carrot forever. Yes. That's like the optimal outcome is that we just dangle it. If we just tell them, yeah, no, we're just trying to look at the schedule it just doesn't work i think in the modern world

23:08you just have to go there and give your opinion and allow the clash of ideas in the public forum let the viewers decide for themselves yeah what they think yeah but this is a yeah this is such a huge part of their machinery is the way that they use these tactics to massage the public image of these companies and make sure that information that they don't want out, and even opinions that they don't want out there, go out there. And so this is, you know, I feel very lucky now that opening eye shut the door early on me. At the time, I didn't feel lucky. I felt like I had screwed myself over. I was like, should I have been nicer to them in the profile so that I could maintain access? Which is a horrible question to ask as a journalist, right? Like, you're supposed to

23:53report the truth and you're always supposed to report in the interest of the public. Like, that is the point of journalism. And in that moment, I was like relatively junior in my career. I was like, did I misunderstand what journalism is about? Like, should I have actually been playing the access game? But it was too late. I had the door shut to me. And so I had to build my career understanding that the front door was never going to be open. Yeah. And that actually really strengthened my own ability to just tell it like it is. The objective. Yeah. And just report what I see are the facts being presented to me, irrespective of whether the company likes it or not. And most often the company really does not like it,

24:41but I can continue to do the work. They don't need to open the front door for me. I was still able to do more than 300 interviews. What you just listened to was a most replayed moment from a previous episode. If you want to listen to that full episode, I've linked it down below. Check the description. Thank you. It takes a little bit of time to realise this as a founder but once you do it's something you can't unsee that having the right systems in place with your business is critical one that's helped me across all of my investments is netsuite the number one cloud financial system and sponsor of today's conversation through their streamlined platform you'll find all of your accounting finance management inventory and hr in one place And their technology has been a massive unlock for all of my companies, especially

25:29for my team at FlightStory. As over the last year, we've moved out of startup mode into scale-up mode. And that's the stage where we found it mission critical to use NetSuite. We no longer have to juggle multiple systems and having everything together has reduced the number of manual tasks and of course, the number of errors. Over 42,000 businesses have chosen to future-proof their business with NetSuite. So if you'd like to learn how it can help your business, head to netsuite.com slash Bartlett and free download the CFO's Guide to AI and Machine Learning. That's netsuite.com slash Bartlett.