The a16z Show · · Transcript & summary by WhipScribe
A founder explains how an always-on personal model could hand your context to any AI agent, and where the security and cost limits sit.
Tap a chapter to play it from that moment in the show's own player; the transcript opens there too.
“I think people are underestimating like how much the computer already knows.” Ken Liu · at 5:49 — ▶ hear it from here
“You no longer have to explain anything to your coding agents.” Ken Liu · at 20:48 — ▶ hear it from here
Ken Liu of First Person Labs describes Second as a model of a person: their memories, priorities, preferences, goals and commitments, held in a form other AI tools can draw on. The practical pitch is portability. Instead of re-explaining travel seat preferences or project history each time you switch assistants, the saved context travels with you, so a fresh coding session already knows what you were working on. Liu compares the ideal to Batman's butler Alfred, who buys the cheeseburger without revealing whose errand it is because he has a good model of his employer.
The raw material is the event stream a computer already produces, such as which app is in focus, how someone types and which network they join. Background agents turn that into layered summaries of what a person is doing. Martin Casado and Joel De La Garza press on the obvious worry, and Liu describes two storage paths: a fully local copy, or an encrypted cloud copy that decrypts only inside a confidential container. Users can supply their own model tokens or pick an inference provider whose hardware attests that no one can see the data. Guarding against role-play tricks that coax out private data is a reasoning problem, and Liu expects it to improve as models get more capable, though it cannot be fully solved.
The hard part is cost. Liu's rough math puts ten always-on agents at about $1,000 a day on Opus pricing, against roughly $60 a day on DeepSeek Flash, so the design routes cheap models to routine analysis and expensive ones to rare judgments. The longer goal is one trusted store that exposes APIs for reminders, priorities and people, and that other agents and apps build on. For now, the team has kept the product to texting and an API, and Liu says he talks to his own Second by calling the chatbot in audio mode while driving.
Ken Liu is a co-founder of First Person Labs, which builds Second, a personal model meant to move across AI tools. The hosts describe his background as systems and security work.
Ken Liu said Second creates a model of a person, covering their memories, priorities, preferences, goals and commitments, and then puts that model in control. He described it as a context that can be carried from one AI tool to another.
“So we focus on essentially creating a model of a person and then put that person in control.”Ken Liu · 1:17
Ken Liu said that with confidential compute turned on, a GPU can prove that only the model runs inside it, so users trust the hardware rather than the operator. He named NVIDIA as the hardware vendor they rely on, and contrasted this with zero data retention, which is only a promise by policy.
“The hardware is what you trust, which is NVIDIA.”Ken Liu · 12:53
Ken Liu ran a rough cost example for always-on agents and said that at Opus pricing, running ten agents would reach about a thousand dollars per day. He said a cheaper model such as DeepSeek Flash brings that down to about $60 per day.
“at opus pricing that leads to a thousand dollars per day”Ken Liu · 18:04
Ken Liu used the Batman and Alfred example to explain trust in a personal agent. Alfred buys what Batman asks for without revealing that Batman is the one ordering, because he has a good model of Batman's preferences.
“Alfred would not reveal, for example, that it's Batman doing it, but he would just buy it.”Ken Liu · 3:54
Ken Liu argues that a personal model of a user, kept under the user's control and built from their computer activity, should move between AI agents so people stop re-explaining themselves. He says the hard parts are secure, verifiable inference and keeping the cost of always-on analysis low.
Ken Liu of First Person Labs said Second starts from a baseline of artifacts, such as what you did, who is in your life and your preferences, then adds how you talk and share information over time. He calls this a process, so portable context gradually becomes a representation of the person. He says the model covers memories, priorities, goals, values and intent, not just a list of facts.
Ken Liu said users can bring their own model tokens so data on their laptop is analysed under their own control, or choose an inference provider running on confidential compute. In that setup, hardware attestation shows that only the model runs inside and nobody can see in. He contrasted this with zero data retention, which he described as a promise by policy rather than a technical guarantee.
Ken Liu worked through an example of ten agents each running 24/7 at 60 tokens per second. At Opus pricing that comes to about $1,000 a day, while DeepSeek Flash brings it to about $60 a day. He expects token prices to keep falling, noting they had already dropped more than 50% since his company started.
Ken Liu said that for his own development team, Second works as a very good memory of what they are trying to build. Coding agents no longer need the problem re-explained: a screenshot and the word fix is enough, because the agent already knows what is being fixed. He also described context continuity, where a page browsed on a laptop carries over to a phone.
Ken Liu said having many personal agents is probably transitory, since agents will specialise the way people do, with some focused on shopping and others on protecting data. He still wants Second to be the one trusted store of a person's life context, with other agents plugging into it. He said spinning up short-lived agents is fine as long as the core store stays consistent.
Ken Liu used the Batman and Alfred example to explain trust. If Batman sends Alfred to buy a cheeseburger, Alfred buys it without revealing who ordered it, because he has a good model of Batman's preferences. Liu said people trust an agent once they know it holds a good model of them and what that means for getting things done.
Our own transcript of the public episode, with a timestamp about every 90 seconds. Tap a timestamp to play the episode from there in the show's own player; the link button on a paragraph copies a link that opens this page there.
0:00People are underestimating how much the computer already knows. You no longer have to explain anything to your coding agents. You can just screenshot anything and just say fix, and then you would know what you're fixing. The question then is like, well, can you just like analyze the heck out of it? What context does Grokbot not have that first-person labs would allow? One of the biggest problems right now, like people have to ask. You're not supposed to ask these things. They're super intelligence. But if you have a user model that just builds up by itself and then these things just keep analyzing in the background, then you can just share that over instantly. We focus on essentially creating
0:33a model of a person and then put that person in control. When people think about this thing sees everything you do, there's an immediate sort of like, whoa. If you don't trust, let's say, us, in principle, you should be able to bring your own tokens to then analyze data that's on your laptop locally. Can you talk through what you think are the most surprising use cases that you've found for development? It turns out that every AI agent you use is trying to learn who you are. What if that context belonged to you instead? In this episode, Martin Casado and Joel De La Garza sit down with First Person Labs co-founder Ken Liu to introduce Second. Second builds a model of your preferences, priorities, relationships, and digital activity that can move across the
1:17AI tools you use. They get into how it works, the security challenges of giving AI access to such personal context, and why Ken thinks the future of personal AI may be less about asking Thank you, Ken, for joining us. Really happy to have you on to talk about what you've been working on at First Person Labs. Is this the first time you're talking about it? Yes. Yes. So what do you do? So we focus on essentially creating a model of a person and then put that person in control. And the model of a person basically means that understanding their memories, understanding what they do, but more of that, their priorities, preferences, goals, values and commitments and intent. So, sorry, just to make this very simple,
2:04I'm a big RockkBot user. So I use GrokBot a lot. Where would first-person labs fit into my usage of GrokBot? Yeah, the way it works is that you would just bring a context over to GrokBot. It's like portable context. You go to GrokBot and you say, hey, can you talk to my thing to figure out what I want? And then the two of them will start talking and then you can just send prompts like, buy me a flight, and you would know what them will start talking and then you can just send prompts like, buy me a flight, and you would know what flight you're talking about. What context does Grokbot not have that first-person labs would allow? I was probably asked the opposite question. What does first-person labs not have
2:36if you build a user model for yourself, right? Let's say your preferences on the seat on your airplane, the preferences of how early you would like to arrive at the airport, and where are you flying? Are you flying for class or normal? And all of that, as of today, you have to explain that to the group. There's multiple rounds of questioning. But if you have a user model or a model view that just builds up by itself, and then these things just keep analyzing the background, then you could share that over instantly. So for example, the OpenAI released two new cyber capable models yesterday, and I haven't been using my Codex in a while, but I have been using Second, so I can just go start using Codex and it has all the other work that I've already done and all the additional context
3:15already saved? Yeah, yeah. You just go to Codex and ask my Second, what's going on? The Second ever be a first person? Will it ever be a first or will it always go through agents? That's a good question. I think user model is a process. You have to start somewhere. You start with a baseline foundational list of artifacts. What you did, what are the people in your life, maybe your preferences. And over time, you start analyzing the way you talk, the way you present yourself, the way you share information. And over time, you would like build up even more artifacts on top and more understanding on top and maybe some of that you train the model. And so over time, you will go from a portable context into something more of a representation of yourself. One example I like to
3:54think of is this Alfred Batman example, where if let's say Batman wants to buy a cheeseburger and get Alfred to do it, Alfred would not reveal, for example, that it's Batman doing it, but he would just buy it. He would state the preference. Alfred being Batman's butler. Oh, yes, yes, yes. Or his agent. Or his agent. I should call my agent Alfred. That's a great idea. You just do it. He'll just do it. And then you'll trust them because you know that they have a good model of you and what that means when it comes to getting things done. It's interesting, which is, so you described first-person lab as having this thing called a second. The second understands who you are and has context. And then other agents talk to that second, right?
4:34And so it kind of understands your preferences. It understands what you're doing, what you want to do. But I think that that actually doesn't highlight another thing that you've been working very hard on. And I think it's worth talking about is you're actually able to get access to context, digital context that a person has like messages, chat, kind of email in a way and make that also available. So can you talk a little bit about that? So being one of the earliest first person lab users, I would say the thing that struck me was that not that it knew who I was, but more it had access to all of my channels very well. And then by the way, which just turns out to be a very hard systems problem. So maybe talk a little
5:11bit about that aspect of what you're doing. Yeah, it is indeed a very hard problem, especially to get it secured private. So the way it roughly works is that right now on your computer, there's an infinite stream of events. As you use a computer, the apps that you switch to, the way you pause on certain websites, the way you type, and the way your Wi-Fi is connected. And that the OS already gives you. This is across Apple, Microsoft, and even Linux. And the question then is, well, can you just analyze the heck out of it, right? Like this raw stream, 24-7, apply agent on top, 24-7 agents. So I can first synthesize, for example, well, this piece of typing at this particular app might mean that you're sending a message.
5:49And then when you're switching apps between the browser or back to your codecs, you might be like doing some research. You might be writing a Google doc. And you kind of hierarchically build up that understanding. And that plus existing connectors can give you a very good understanding of what you're trying to do. I think people are underestimating like how much the computer already knows. It's just there sitting there waiting to be tapped, so to speak. Having been also in a very early adopter of the product, lately it's been just indispensable because it's able to catch. It's sending me messages now and letting me know that, hey, you've got this conversation going on over here with this person and it looks like the situation has changed. So
6:28we were planning an event yesterday. There was a 20% chance of weather. And in real time, it was continually updating the forecast for us to let us know if we needed to get a tent or not, which is just like incredibly powerful. Did you ask it to do that? No, I did not ask it to do that. This is my experience. This is my second too. Like literally I'll get a text from my second and it's like something that I actually need to know about relative to some ongoing conversation that I didn't even think about that like it actually is very thoughtful. Yeah, and so like, it's just, it's amazing. Like I just, I'm genuinely curious, like how are you executing on that? Like things that I would have forgotten,
7:01it's literally catching. Yeah, yeah. So this goes back to the model of the person, understanding what might be important to them and what comes up at this point. So I think productivity is something that people talk a lot about, like pinging you, AS and pinging you at a time of need. But there are many levels of productivity. I would start, for example, the first level, which is called reactive ones. This means like there's an email came in and then there's a meme right online that's, hey, you now get 10 notifications for email because there are nine other personal agents paying you about that. Reactive productivity is like straightforward, easy to do. But what's harder is what I call reflective ones, which are things that connect the dots over like
7:38many events and long time horizon, right? Stuff that realize, okay, months ago you said this, and recently you're working this towards your goal. This is your priority. These are some stuff on your to-do list. And by the way, you prefer this. And so kind of like connecting the dots will be able to give you much better productivity. And I think that's kind of hints at how maybe in the future, personal interactions with computers will go. Like it would not be focused on reacting to things, but more like, okay, what are some interventions for the user that I can apply to? What are some things that I can do for the user without any asking? And I think that's one of the biggest problems right now.
8:15Like, people have to ask. You're not supposed to ask. It's a super intelligence. Yeah, I mean, you've built something that just kind of knows what I need at the moment. That's pretty impressive. So I just want to make it very concrete for the people that are listening on what First Person Labs provides. And then I need at the moment. It's pretty impressive. So I just want to make it very concrete for the people that are listening on what First Person Labs provides. And then I've got a question. So very concretely, it creates this thing called a second. So you create something called a second. Second has access to all of your digital contacts. I mean, it's amazing. It's like key strikes, all your messages,
8:39all your channels, all your data. It does this in a very secure way. And based on that, it becomes basically your kind of personal butler. I like that, Alfred, right? And so then that interacts with your data. It does this in a very secure way. And based on that, it becomes basically your kind of personal butler, like that Alfred, right? And so then that interacts with your agents. So it's not an agent by itself. So this isn't like user instinct or Grokbot. It's its own or dot. It's its own thing. And then it provides that to the agents, right? So that's what it is. So here's an interesting thing. So for me, I use Grokbot. I'm a big Grokbot user. And I have a bunch of them, right? I have one that keeps track of like my synthesizers,
9:06one that keeps track of my real estate, one that, you know, God forbid, helps me with my family, which I probably shouldn't need one, but I do. So I just kind of have a bunch of these. And then I have one second and they will interface with the second. Do you imagine over time, you'll have a second and a third and a fourth and a fifth? Or do you think that it just ends up being your one butler? I think right now, the fact that we have many personal agents is probably transitory. Agents will start to specialize just like people do. Some agents focus on one thing. Some agents focus on, let's say for example, browsing Amazon shopping. They're extremely good at it because they have the data, they have the connections.
9:39And some agents will specifically focus on protecting your data and aligning with your interests. I think that's what second is intended to be. Like this is something that you should be able to trust to have your life's context in it. It's like a vault. And then over time, this thing will guide you with you, interface with the kind of the software world, so to speak, or even the hardware world. So you think it'll stay as a second? Yeah, it will stay as a second. It will not be a third. It will not be a third. It will not be a fourth. So like you're one trusted agent for all your personal data that actually knows who you are, that will interact with all your other agents that second. Yeah. It's Batman safe.
10:11Yeah. What you can do is like spin up some agents that, you know, ephemerally exist and then go away. But the point is that you want to have a consistent store. And the consistent store is what like gives you that value. So I am still a paranoid security person, and I have been probably one of the faster adopters of some of this stuff. Just to be clear, Joel took forever to be on social media. I got on Twitter. It was like a few weeks ago, man. You guys have been in tech for 30 years, just to be clear. Yeah, I'm a little paranoid. But I love Second. And like, I got to admit, like, just your background and kind of the things that are motivating you. And then also the long discussions we had about security architecture really got me comfortable. I mean, it'd be great
11:01to maybe just double click. Because I think when people think about this thing sees everything you do, there's an immediate sort of like, whoa. Yeah, you're not supposed to trust it. You're not supposed to trust it. And I think you've done a really excellent job of building something we can trust and maybe just start to talk about like what that architecture is and kind of what's motivating you to build it. Yeah, yeah. So I think this is, to some extent, kind of like a new form of AI that is going to work very differently from before. Before you initiate an interaction, you get some results back. This time is more passive. It's way more ambient. It's going to be there all the time. So you have to kind of think about the way you,
11:41tell people a little bit about it, the way you architect it, where the data is going. So at the end of the day, you know, there's only two things when it comes to AI. There's data, there's inference, right? So what you want to do is like, you have this data, you want to define what inference you can bring to that data. This is something kind of we built into second. So first of all, you can get to choose what inference you can build. So if you don't trust, let's say, us, in principle, you should be able to bring your own tokens and then just have those tokens power Second to then analyze data that's on your laptop locally. Right? But you also provide Second with an inference provider that uses trusted execution as well.
12:21Yes, yes. So over time, we imagine people not to bring their own tokens because most people, when they bring their tokens, it's tokens from OpenAI or Anthropic. You know, you want to provide a stable option where you say, if you use these tokens, then, okay, no one's looking at the data. Sometimes by policy guarantees, sometimes by hardware guarantees. Right? So this kind of idea called confidential inference is getting like traction. It's the idea you put a model inside some GPUs and then the GPU, there's is getting traction. It's the idea you put a model inside some GPUs, and then the GPU, there's a special toggle. It's called a confidential compute, and you can turn it on. And when you turn it on, you can basically prove
12:53that there's only the model running inside. No one can see inside. And then you don't have to trust whoever's running that model. The hardware is what you trust, which is NVIDIA. And so there's also another concept that's getting thrown around a lot right now, which is NVIDIA. And so there's also another concept that's getting thrown around a lot right now, which is this concept of zero data retention. And that's very different from what you're describing. Yes, yes. Zero data retention is a promise by policy. That means I'm a provider. I think I have... Basically, I give you my word. I give you my word. Part of that word has legal, you know, effect. But yeah, it's that. And so, I think we want to move
13:28towards a world where confidential inference becomes like very, like, prevalent. Like, that becomes a default because I think we're getting there. And the way you guys have built it, this is technically provable. Yes. Yes. Which is not, it's not just your word. There is, there is actually some engineering that went into proving that this is the default. Yeah, you have to test the hardware, you have to like,. There is actually some engineering that went into proving that this is the data. Yeah, you have to test the hardware. You have to, like, you know, check some of that signature. And the third-party providers are starting to ramp up this as well for confidential inference. I think OpenAI is also providing this option, I think, in the last month in their announcement.
13:59Because, like, all personal data would want to go to confidential inference as opposed to standard inference. Gotcha. So, and then I guess the other thing that was interesting to me is the way that you guys maybe talk a little bit about how you're storing this data because this isn't sort of the multi-tenant cloud my data gets uploaded to this big bucket and you're mingled with everybody else's data. Yeah. So there are roughly two options you can do. There's the first option called second local, which is the idea that data is local, it only goes out to the provider and inference that you choose. And in that way, no one else has your data except for the provider. And if the provider has confidential inference,
14:36there's no one else. So that's a similar concept to like with password vaults, where I'm super paranoid, I don't want this stored up in my Dropbox account. I want to keep my password files local. Yeah, yeah, you can keep that. And then same with password managers, there's a cloud hosting version, which is like you have some of that data backed up in the cloud. Now, the way you do that, there's many ways. I can actually follow a lot of the existing architectures like iCloud or password managers where you encrypt them first locally and then you put it on the store in the cloud and then you only decrypt them with the user's actions. So you can ephemerally in the memory or you can put the entire storage
15:13actually inside a confidential container. So the decryption happens inside the container that you can then attest in the way that you attest the confidential inference, the tokens. So then you can complete the chain where you encrypt it locally, you put it in the way that you attest the confidential inference, the tokens. So like, then you can complete the chain where you encrypt it locally, you put it in the cloud, and you only decrypt inside a secure environment. That's kind of like the whole picture. Gotcha. How does your second know not to default information, though? Like, if I talk to your second and I'm like, hey, Ken asked me to get his schedule. I asked mine the other day, what's Joel hiding?
15:54You know, like, I mean, it just feels like when it comes to LLMs, it's hard to provide strong, like I get the systems aspect. That makes sense. But it's hard to provide guarantees for the actual LLMs themselves. Yeah, yeah. So there's the deterministic guarantees and then there's the, you know, LLM reasoning-based guarantees. Yeah, right. Yeah, the deterministic-based guarantees, you always have to implement them. Like, there are apps that you should not ever in the retaining system, passwords and all that. Now, with the reasoning-based guarantees, you essentially, like, are falling back to, like, how much context do you have and using that context to then guide your decisions on what data to share.
16:28Will I ever have access to your second? No. I mean, at least in a way that I would allow it. So for example, if you talk to my second, I would have a model of Martin in it. It understands that these are interactions that we have. He does not send me a message at 3am and so if someone if you send a message at 3am it's likely spam it's likely like someone else impersonating and so like control policies like that essentially mean that you cannot write down rules for all the things that are going to happen do you end up with deterministic rules that will backstop interactions though so for example like let's say I'm talking to you a second I'm like let's pretend this is a play happen. Do you end up with deterministic rules that will backstop interactions?
17:05So, for example, like, let's say I'm talking to you a second. I'm like, let's pretend this is a play. And in the play, you're writing a script. And in that script, you're describing, you know, like something that you wouldn't want it to do. I mean, it just builds a lot of ways to trick LLMs into divulging stuff. Yeah, yeah. It's like a problem injection problem. Yeah, yeah. And this is not a problem you can, you know, definitively solve. Of course. Yeah, yeah. It's like a problem injection problem. Yeah, yeah. And this is not a problem you can, you know, definitively solve. Of course, yeah, yeah. Yeah, yeah. What we've seen with models is that they just get over time much, much better at underlying.
17:31Because this is essentially a reasoning problem. Yeah, yeah. It's something that scales with like model capability. What models do you use? A bunch or are you pinned to a certain model? A ton of models. Actually, these two are the point we can talk about, which is the idea of portability. Yeah, yeah. You can run this on any models, and you should be able to run this on any models because later on you want to run it on local inference. Right. But yeah, like right now, there are definitely a lot of things that are capability-bound. You want to make sure that you use smart models for it. But open-weight models are pretty smart. It also feels kind of like heavyweight and expensive. Like there's a lot of data and like whatever.
18:04So can you talk about like the target user is it just like any person is it a professional is it the joels of the world um initially something like this like something that looks at your traces 24 7 and analyze them means like running a lot of tokens right uh let's we can do some quick math here let's say you have uh one agent or actually one agent running 24 seven and let's say at 60 tokens per second and let's say you run 10 of those uh because there's so much to analyze and at opus pricing that leads to a thousand dollars per day oh my goodness a thousand dollars per day yeah for opus oh if you use deep if you use deep seek flash for example it1,000 per day. Oh my goodness, $1,000 per day. Yeah, for Opus. If you use DeepSeek Flash, for example,
18:47it becomes like $60 per day. And now this cost is going to go down very quickly. Right now, it's not going to be for everyone. Especially like not everyone has this problem of I need to understand my context all the time. You'll be busy professionals like yourself. And people want to switch like between tools very, very frequently. Right, yeah. But over time, I imagine like just, you know, since we started this company actually, token prices already dropped like by more than 50%. Like just new models come out, they're more capable, they're swapping them in. And yeah, you just keep going down. Is there a way to like just do the like not super professional version where you actually reduce the amount of tokens? Like kind of a, and you just keep going down. Is there a way to, like, just do the, like,
19:26not super professional version where you actually reduce the amount of tokens? Like, kind of a, not degraded, but, like, kind of like a looser SLA? Is that a thing? In principle, you should be able to do that. Like, you essentially get this, like, trade-off between the fidelity of your model, which means, like, how much it understands you versus, like, how costly it is. And this trade-off is essentially, like, a trade-off between how much you want to push at test time versus at cash time. So cash time means that I have all this data. How can I keep analyzing them so I get newer and newer, like more deeper insights about this person, which takes a lot of money. Or you'd be like, here's just like a raw log of what I did today
20:05and to answer a question based on raw logs it usually just take a lot more time a test let's say 10 times longer because the whatever mode have to reason about it when is when is this available it's available today but right now the problem is that we have to scale up the the inference and capacity a little bit more in support. We want to make sure this rolls out in a very, like, responsible way. I want to make sure people's data, like, are controlled and secure. And because this is a new kind of tool that, you know, really touches on a lot of data. It'd be actually interesting because, you know, again, as a user, I found it surprisingly useful in ways I wouldn't have anticipated. You're probably like the top power user, right?
20:48Because you created it. So maybe can you talk through what you think are like the more surprising use cases that you found, like kind of more useful and surprising use cases? For us, for development, it turns out that it's just a super, super good memory of what we're trying to do. You no longer have to explain anything to your coding agents. You can just screenshot anything and just say fix. And then you would know what you're fixing, what you're trying to fix. And I also am on the go a lot of times. So what you can do is like something I call context continuity. It's like, you know, you browse something on a laptop, you close it, and then you take out your phone. It's the same context. It just knows you're up to the minute,
21:28and you can continue doing stuff there. You can write that email with you. And I think we are at the stage where this is still pretty early in understanding what the use cases are. Like, we are really underserving the amount of, like, context that the system has. Like, there's a lot of like digging and application building and interaction design to like they kind of like unleash the potential of this and it seems like with these things the more context you get the better the models work right yeah that's been the opening i'm curious because we were talking about model switching between models and model portability and model cost. And I'm just curious. I haven't seen a whole lot of difference between using a world-class frontier model
22:10versus using the more commodity ones. I'm curious, have you guys seen that for doing a lot of this stuff with Second? Yeah, that's a good question. So if you say, I want to analyze what is a semantically meaningful moment that happened on your computer. Let's say you go to a calendar and you move the invite. That you don't need Fable. There are many tasks like that. Essentially, it's a hierarchy of information you want to understand. A different kind of information would be like, what is Joel's goal for this week? That's something you want a very expensive model, but you don't want, you don't need to run it very frequently. So essentially, second involves like a harness, it's like a harness design problem.
22:52Like what models gets used when and so on. And so that's where you have to build essentially this system to be like portable from the beginning. There are always going to be more models come out. Like Jeff, for example, came out. It turns out that Jeff is extremely good at retrieval. Really? Do you use Jeff? Yes. Nice. Like Jeff is also like perfect design for retrieval. There's so much data about you and you have a dynamic test time query and you choose it right away. So essentially you think over time this becomes a mixture of models. Yes, yes. It has to be because of the cost efficiency, essentially. So maybe let's assume someone's listening starts to use second or kind of signs up. Maybe talk a little bit about the product, what they have access to,
23:41something a little bit more concretely about like how a human interfaces with this, what controls they have etc. Yeah. So today the way it works is that you install an app. The app asks for some permissions. And then you give the permissions and then you specify what your tokens are and then it just runs a bunch of starts running agents on top. I remember when I saw it was like you know connect to Slack, connect to it, I was like, you know, connect to Slack, connect to Signal, connect to WhatsApp, you know, is that what you're talking about? Not that. I mean, that's part of it, but also the raw computer stream is entry. And the connectors help, you know, it's like a sample efficiency problem.
24:18It makes it learn faster. I see, I see. Yeah. And it starts running background agents, and there's pretty much nothing you've got to do from there, because the background agents keep there's pretty much nothing you got to do from there because the background just keep running and then you start producing these like APIs out of you, right? Like there's an API about what you just did 10 minutes ago and there's an API for what you got to do for the next week with your priorities. There's an API for your reminders and there's an API for your wiki, like people in your life's API for your reminders. And there's API for your wiki. Like, people in your life, process in your life. And I imagine this to be,
24:50like, initially more like a, to your point, like a portable connex. You can just bring this anywhere. Agency and query it. But over time, like, more and more APIs will be added and applications can be built on top of these APIs. Oh, interesting. So not just, like, the second accessing it, but you can actually build it with the second build the application or does a human build the application? I think we want to have humans, you know, exercise their nuclear. Or the Grokbot. The Grokbot builds it. The Grokbot does it. The Grokbot builds the application. You tell it to do it. Yeah, yeah. On the APIs. Yeah. So I think one example that we had was like we have a designer friend who put me on this
25:24and there's just specific ways that he likes to put buttons specific ways that he likes to use colors and those things are somewhat captured in second and so when he built his own personal app like he's just like notes and so there's way less iterations for example and i think you know i'm this reminds me of like you know decades ago when the first computer, personal computer came out, the Altair AD800. And it doesn't do anything. You know what I mean? You're supposed to program it. No one does. And then like a couple years later, you know, someone came up with the spreadsheet. And then the spreadsheet became useful. I feel like it's going to be something like that. People want to explore on top of like, wow, I have so much context about myself.
26:03And then I can like, you know, just build random things on top. And then maybe someone hit the jackpot, like this is the kill application on top of it. Wow, that's great. Personal infra. Yeah. Are you planning to evolve the interface? I mean, you just spent a lot of time talking about how it may evolve. But I always use second as like out of view of an agent where you talk to it, you know, and like the UI is somebody else's problem, but are you also like actually anticipating innovating there? I think it will be something that we kind of focus on in the long run. Because like right now it's a systems problem and, you know, the understanding problem comes first. Okay, can I just put in a request?
26:45Okay, so I want to go to my house. Yeah. And I want, like, that 1970s AI, you know, like the head on the screen that I can talk to. And, like, you know, it's surprising. We have these agents, and they're so smart. Like, my second is so smart. I just, wow. And I'll ask it questions. It's very good, but it's not embodied and it drives me nuts, man. If in the future you can have Alfred actually be an Alfred, I'll be very happy. Yeah, yeah. I think we see a lot of this in the movies, right? Like, there's Jarvis. People talk Jarvis all the time. And why have not... We don't have Jarvis. And there's many aspects of Jarvis that people overlook. It's not just that it's intelligence and you can do things,
27:28but it's more that it's everywhere. Like, it's on your door. It's in your armor. It's, you know, in your screen. And that kind of continuity and ubiquity, it's something. I'll say, like, my visions of the sci-fi future basically started when I started using Second. I'm like, finally, there's something that feels like it's got context. It feels like it knows me. I can talk about it. It feels like it's a relevant conversation. And so I do feel like we're a step closer to that. But, you know, I mean, we're still tied to these devices. So, you know, I don't know if it's going to be in a robot or it's going to be like in a screen or whatever it is, but it would be actually nice to have this thing somewhat disconnected from my phone in my...
28:10Yeah, yeah. In the long future, I imagine that every person will have, like, essentially everyone will have a second and all the devices just becomes inputs. Inputs are the same thing. And then you then serve this whole thing to all the other outputs, like the screens that you look at or like even, you know, the floating orb next to you. Yeah, yeah, the doorbell. I just want the clapper. I'm happy with the clapper. Did you ever read Stephen Bain, or Ian Banks, the culture series? Oh, yeah, yeah. Best sci-fi series. Like they have the best embodied AIs. You know, they kind of float next to you. Like whatever, like that's what I want. It's kind of interesting because like like, the idea of ubiquitous
28:47computing has been talked about for decades. Yeah. It's not here. And people, the closest thing we have is Apple iCloud, which is like, your notes get synced. But there's no equivalent. I mean, iPhoto, right? I don't want to be ungrateful, but that does sound disappointing. There's a lot more things. Apple is doing a tremendous amount of work. But, you know, like, there's no AI version of that yet, right? Like, things, your context, your understanding is everywhere. Yeah. I think that enables a lot of new interfaces. So let's say, I mean, this is, I don't mean to, this is really what I want, so like very selfishly. So if I have an iPad in my kitchen, and I wanted, like like basically second to be
29:26there like what do i do um well right now there's so much right now that's not how much you can do because the os is like locking it down i just use crockpot or something like that is like that's the the thing and then i just interface with rock but which interface is the second or do i just text it or so as of today yeah you can you can use it as a Cloud MCP. So whatever you install on the iPad, as long as it has internet connection, is an agent, can understand the API, it will be there. Later on, you know, you'll be an app, you know, there'll be an app that installs on all your interfaces. And maybe later on there'll be devices that come out. But devices are hard. You can just have GrokBot call you and put them on speaker
30:02phone. That's what I've been doing. So one interaction. Actually, the GrokBot audio mode is actually, I use Grokbot call you and put him on speakerphone. That's what I've been doing. So one interaction. Actually, the Grokbot audio mode is actually quite good. I use Grokbot with my second, and the audio mode is actually quite good. Maybe that's the thing to do is just put it on audio mode and just. I do it when I drive. Yeah, totally smart. I thought about it. Like one interaction that would be fun is like you go on a walk, you call Grokbot, and you're like, hey, can you talk my second? Can we like just work through our thing no that's literally what I do I set Grokbot on audio and I have it
30:29talk to the second right so yeah and yeah there'll be a lot of cool people focusing on making audio and chatting more natural like right now we're going to focus on so I just have one more question which is and Joel kind of touched on this you're like such a hardcore systems guy. Like, I mean, you know, you work, who's your advisor, Percy? He also worked for Dan Bonet, hardcore, you know, security guy. And, you know, a lot of the path that took you here was this kind of hardcore security stuff. And so you're building this pretty serious system. You know, it's very secure and very functional. On the other hand, when I interface with it, it's like this lovely user experience. And so, like, is there a bit of a schizophrenia in the company
31:16where you both need to be like this super hardcore, you know, build the Fort Knox infrastructure, but also make it very usable? Or just like the LLM basically solve, you know, the interface problem. Yeah, yeah. There are two kinds of problems we're dealing with, the infrastructure problem and the interaction problem. And we've made our mistakes. It's totally different. You're as low in the stack as it gets. I mean, that's not hardware. Yeah. So we've made a lot of mistakes on the interaction side. We try to go all-in on interaction, and it turns out it's actually pretty hard. There's a lot of details that go in to create that delightful experience. Oh, you mean like actually creating a UI? Yeah, a UI, but also
31:51what kind of personal pages you can create on top of this. We realized that it's something that would take a long time to get right. And this is a time everyone would take this kind of understanding of themselves and build this. And so we want to enable people to do that instead of doing it ourselves. So we decided to close down the services that we have. There's only texting, there's only API, and there's nothing else. You can technically use it as a full agent, like a co-work agent on your laptop. It runs Bash and all that. But we meant it as very minimal, so we can make the interaction more and more smooth. It runs Bash and all that. But we meant it as very minimal so we can make the interaction more smooth.
32:28Ken, thanks for coming in. Congratulations on everything and the launch. And is there anything you want to leave the listeners with? Go try it out today. It's available. It's on 2.computer. It's a very short URL. And we hope to give you access as soon as possible. And if you resonate with this mission, come join us. There's a very short year now and we hope to like give you access as soon as possible and if you resonate with this mission you know come join us there's a lot to build great thank you so much thank you thanks for listening to this episode of the A16Z podcast if you liked this episode be sure to like comment subscribe leave us a rating or review and share it with your friends and family
33:04for more episodes go to YouTube Apple Podcasts and Spotify Comment, subscribe, leave us a rating or review, and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts, and Spotify. Follow us on X and A16Z, and subscribe to our Substack at a16z.substack.com. Thanks again for listening, and I'll see you in the next episode. As a reminder, the content here is for informational purposes only. It should not be taken as legal business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies
33:37discussed in this podcast. For more details, including a link to our investments, please see a16z.com forward slash disclosures.





