ChatGPT, One year anniversary reflections

Introduction

Fraser Kelton: we get signs of life that a RLHF version of 4 is actually amazing. And we say, this is the moment, it, the model has crossed a threshold where we can actually start to build the vision of having a first party product, which is your AI assistant. And we,

uh,

Nabeel Hyatt: an accident. You don't know how big it's going to be, but, but there's a feeling of gravity there. There's a

Fraser Kelton: oh yeah. Yeah, yeah, yeah. And, and Greg, Greg starts

Nabeel Hyatt: just a happy accident.

Fraser Kelton: not a, no, no, no

Nabeel Hyatt: Hello, everybody. Welcome to Hallway Chat. I'm Nabeel.

Fraser Kelton: I'm Fraser.

Nabeel Hyatt: the agenda for this week writes itself. Which is that

Fraser Kelton: Because we don't have one.

Quotes from the ChatGPT 1 year anniversary

Nabeel Hyatt: No, that's not what I was going to say. It's because it's a year since ChatGPT. Uh, and so I have a special guest this week. The former head of product at OpenAI who was part of the team that launched ChatGPT. Fraser what amazing words of wisdom can you impart to everybody about how you saw The beauty of ChatGPT before everybody else.

Fraser Kelton: I thought it was so lovely that Liam. Somebody who is on the on the, the modeling side, started his anniversary tweet by saying. Consumer behavior is impossible to predict. What a year.

Nabeel Hyatt: No doubt, I think Alyssa Rosenthal from the, from the go to market team I saw a tweet from her that said a year ago tonight I got a Slack letting me know that we were silently launching a low key research preview in the morning and that it, it shouldn't impact the sales team.

Fraser Kelton: I wonder if it impacted them at all.

The um,

Nabeel Hyatt: team know, Fraser?

Fraser Kelton: the, well, cause it was supposed to be a low key research preview. We, we did not think that it was going to be a significant deal. Everybody's been very open about that. Greg had a tweet this week as well, that was very honest was. It was a low key research preview where we had diverted the team that was working on the first party product that was going to go out with the GPT 4 launch in March to push this out.

And then we were going to get back to work on the first party product. We, we know how that turned out.

Nabeel Hyatt: I'd love to just unravel a little bit. I think we folks know it was a surprise. It has to be the fastest growing consumer internet product in history. There wasn't, there's an exactly a way you could have predicted that. Was it on the roadmap at all? Like when you were building product roadmaps for the team

Fraser Kelton: well, this is the part that's hard to Disentangle. We had a long term roadmap for what has now been publicly called the Assistant adjacent to the API. We wanted to have a first party product that was your AI Assistant.

Going back to the start, that's been part of the product roadmap and the vision. ChatGPT was, was a, a research stepping stone, if you will, on the path toward that first party product.

Nabeel Hyatt: So, so in a way it was a tease for what is now called assistance?

Fraser Kelton: no. We had a vision for a first party product that was supposed to be the AI assistant, and the AI assistant is something that you Talk to and it talks back to you and it helps you with your tasks as you go on with your day.

Nabeel Hyatt: Mm hmm.

Fraser Kelton: Sounds very similar to, to what we all use chat GPT for today. We, well this, this is a great story. Maybe we go back to GPT four's training, just to tighten, tighten up the narrative. Four gets trained. And we get the base model, and it was rather underwhelming. It was rather underwhelming, and we're sitting there going, Oh no, what's just happened?

And Greg, who is truly a remarkable technologist, and, an incredible person, says, let's get to work with instruction fine tuning of this, and, and he's so, well, there was a tweet, just as an aside that says he and, he and Sam run the company, but they haven't found the right way to reflect that, and I thought that that was a beautiful thing for Sam to say, because internally Greg is a superhero.

Greg says, let's get to work. And he says, we're going to have a new aligned version of four on a weekly cadence. Let's get going. And so the team starts the RLHF pipeline

Nabeel Hyatt: Right.

Fraser Kelton: every single week and we're it, it gets better and it gets weirder and it gets worse, and it gets better and it gets weirder and it's worse week after week after week after week it starts to look really good. And we realized that this is the time to have a first party product, uh, which was the Assistant. And we wanted to go out in

March with the GPT 4 launch.

Nabeel Hyatt: was, oh, four is the assistant. This is the thing that's good enough now that we've gone through fine tuning that's where the assistant's going to really come through.

Fraser Kelton: The model is now good enough that we can do this.

Nabeel Hyatt: I see. So now if you backtrack to why you're underwhelmed with ChatGPT as you're working on it, because it's still 3. 5. Like it's,

Fraser Kelton: it's 3. 5! We, we

had seen, yeah, we had seen 3 go to slightly better, go to the instruction fine tuned version of 3, go to 3. 5, go to a slightly better version and we thought this is interesting. It, it is interesting research because now it, it is specifically tailored to chat.

Because of the, what we had started to see in the summer around the quality of 4. We had stood up what we called internally a prototyping team to try to figure out what the first priority product that we bring to market as part of the 4 launch should look like. we're doing all sorts of. Crazy stuff, like as you should with a prototyping team. We're going broad. It's scrappy.

Nabeel Hyatt: But very specifically in it sounds like the read internally was without 4. This is not even an MVP. This is, this is a prototype. This is a test. But it's not the minimum viable product because at that point in time, I think if I had asked you, you'd said, well, minimum includes GPT 4,

uh, as viable. And so then why, so then why did you guys end up releasing it?

Fraser Kelton: Let me, I'm going to take a pause right here.

Nabeel Hyatt: Yeah.

Fraser Kelton: Here's what I would love to find a way to, to convey. We trained for, your insight is spot on where four looks like it's the, the thing where it's finally good enough for our product vision and roadmap to invest in that direction. The other piece that's missing from that is we had a taste of first party relationship with end users with DALI.

Nabeel Hyatt: Mm hmm.

Fraser Kelton: so we do DALI, we're like, this is, this is intoxicating. We're getting real feedback from end users. We think that that's not just useful for the research roadmap. It's just really great. We're not, we're not sitting between the end user with the developer.

Nabeel Hyatt: that you feel when you're working with consumer. That

Fraser Kelton: That's right. That's right. The

model

Nabeel Hyatt: with customers versus in between. Yeah.

Fraser Kelton: that's right.

And so then the thing that I want to convey here, there was a team of like eight people who ended up shipping ChatGPT, but had been working on what is now ChatGPT for months in advance on the prototyping team. It just happened that we took the UI that we, we had at that moment and we're like, yeah, let's ship out a chat interface.

Nabeel Hyatt: hmm. Yeah.

Fraser Kelton: then that lit the world on fire.

Lessons from ChatGPT development

Fraser Kelton: There's a couple of lessons for entrepreneurs. One is the one that you were goading me for earlier around, like we had to break free from the organizational planning process as well as their structure. So we had purposely for.

That prototyping team and the, what became ChatGPT was we, we smashed some people who were on product development and product engineering together with research for the first time.

Nabeel Hyatt: Wait, ChatGPT was the first time that happened?

Fraser Kelton: yeah,

Nabeel Hyatt: Then, because Dali was just a couple people just doing it. There wasn't

really

Fraser Kelton: was, you had your research org.

Nabeel Hyatt: Yes.

Fraser Kelton: your product development org and they they tossed it over the

fence.

Nabeel Hyatt: tossed some stuff over the transom, product took it and ran with it, which is,

Fraser Kelton: That's right.

Nabeel Hyatt: anybody who's gone through this process knows that's a remarkably different process than in this situation, it was research working hand in hand with product.

Fraser Kelton: That's right. That's right. John Shulman's team, these guys from Google show up who are all of a sudden motivated by product, not by research, but they're researchers.

Nabeel Hyatt: Yes.

Fraser Kelton: But the narrative that gets told right now, understandably, is hey, we, we didn't know, like, this was a happy accident.

Nabeel Hyatt: Mm hmm.

Fraser Kelton: But it was a happy accident after months of smashing these things together, getting rid of OKRs, iterating on a UI that was simple. We had explored all sorts of silly UIs for the prototyping thing, and we kept stripping it away.

We kept stripping it away. And then the two weeks that was the launch was basically.

Nabeel Hyatt: Now, let's pause. Before we get that, I want you, I want you to rewind for a second.

It's not a happy accident

Nabeel Hyatt: if I'm hearing what you're saying correctly, if I've heard the background that you've talked about correctly, it's not just that sometimes people can't understand what is gonna hit or not hit.

Um, it's that it does take some intentionality on how you build things. And we are going to go through a phase right now where a whole bunch of research scientists are going to try and work on startups in AI. And a whole bunch of founders are going to go try and run and grab research scientists. And you're just saying that this was actually the first time in OpenAI's history that product and engine research had worked this tightly together. That mattered.

Fraser Kelton: Yep

The Birth of the AI Assistant

Fraser Kelton: so we, we get signs of life that a RLHF version of 4 is actually amazing. And we say, this is the moment, it, the model has crossed a threshold where we can actually start to build into the roadmap and the vision of having a first party product, which is your AI assistant. And we,

uh,

Nabeel Hyatt: an accident. You don't know how big it's going to be, but, but there's a feeling of gravity there. There's a

Fraser Kelton: oh yeah. Yeah, yeah, yeah. And, and Greg, Greg starts

Nabeel Hyatt: just a happy accident.

Fraser Kelton: not a, no, no, no

Forming the ChatGPT team

Fraser Kelton: . So in the summer, Greg, Greg starts getting individuals all over the org in the most crazy of places. So, a guy named Eric from the Supercomputing team, comes over and says, hey, I'm familiar with iOS. I'd love to build a prototype of an iOS app that we can use. And, and Greg's like, cool, that sounds awesome. Let's do it. And so Eric, Eric, like, hacks on, on evenings and weekends and comes back with a very simple version of an iOS app where you're chatting with four, you're, you're talking to four. And somebody goes, wouldn't it be great if, if Whispers put into this And so, sure. Let's put Whisper into it. There was somebody who was on the inference team comes over and says, I think it would be awesome to have a Chrome extension that would allow us to do these types of tasks. Wouldn't it be great if you could do summarization of PDFs as you browse around the internet and builds this little applet that does that. Somebody else is building what internally was called MeetingBot that would join your, it's, it's super powered and all the other apps that are out there where it

would join. Yeah, that's right. That's right. And it would join your. Meeting, and then it would send you the GPT 4 summarization of what just happened.

And we were trying to figure out what first party product assistant would feel right. And, and I think importantly, these were not people who, were assigned to a team that were then given You know how it is. You have a great Here, this is a great time for you to talk about your Startups shouldn't have OKRs and follow the lead from Google.

Like, give us, give us 30 seconds on that.

Nabeel Hyatt: Well, it's ironic that you asked me to give that rant because, , I almost never go on podcasts other than just wanting to chat with you, Fraser. But I went on one yesterday and with, with Fareed and Brian Balfour called Unsolicited Feedback. And Brian reminded me that back in the day when I was a very young entrepreneur, I came out of the Zynga process and I actually had a presentation that I presented to him and a handful of other.

Product leader, CEO, founder types in Boston, this is like early YC, Boston days about how to run OKRs and startups.

Fraser Kelton: Oh, what?

Nabeel Hyatt: Yes, so how, how we've evolved over time

the core thing about OKRs is that they were developed originally.

Actually it's called OOGA, they were originally developed in the military as a way to organize, as you can imagine, thousands and thousands upon tens of thousands of infantrymen in a battle. And if you're Google, that metaphor is maybe not wrong. You're trying, you're a massive cruise ship and you're trying to figure out the next year of your life

what it really tries to do is ostensibly, in a way, rip away taste from the leaders of the company and and learning, I don't, I don't mind setting goals and actually don't mind OKRs as a kind of general precept, as long as we all agree that every objective or key result or goal or thing that you put up, has a cone of certainty That expands outside of a month in very, very rapid succession.

So by the time we get to a year, no one knows in an early stage series, a startup has any idea what the real revenue is going to be, let alone, these are the five milestones. And, and so it's fine to set up waypoints. It's important.

Fraser Kelton: Yeah.

Nabeel Hyatt: unfortunately is used as a bludgeon generally to keep people who have insights from doing the work they really believe in because it's quote unquote not, not on, on plan.

Fraser Kelton: Yeah. That's right. So we, we had a research org that had a. Year long plan that then worked back to individual research tracks that had OKRs or similar. And that makes sense because research is a long lead planning effort, right? Then we had an applied org where we're shipping APIs and inference and fine tuning, et cetera, et cetera.

And we, We had a year, we had a roadmap, we had a year long plan that after six months got quite fuzzy, and then we had really loose OKRs on a half and a quarter basis. And those are the two orgs, right? Observation was, oh, well, this is something very different than either of those. This is not a place where you take the model, put it into a, an API. and serve it to developers who are then building products for consumers. This is, we are taking a model that looks profound, feels profound, and we're trying to figure out, uh, how to bring it to market as a first party product. What is the right first step of delivering on the long term vision of an AI assistant,

and

Nabeel Hyatt: but you're like, that could be a Chrome extension. That could be an iOS app.

Fraser Kelton: Right,

Nabeel Hyatt: this supposed to be? How much,

Fraser Kelton: that's right,

Nabeel Hyatt: interfaces, what are you supposed to do with this thing?

Fraser Kelton: that's right. So there's, there's two things to call out here that I think are interesting as as insights for, for this experience from startups is we. We aligned people who were working on the product development piece with researchers. So we literally had them sit adjacent to one another.

We fractured the reporting structures and they were a team that was rowing together, so to speak, to try to solve this problem. We also broke free of resource planning and resource allocation. You can imagine if you're going to say, well, Hey, listen, we need two backend engineers for this task.

People are like, well, listen, we have a an OKR for Q2 that we're going to miss if we give this up. So we can't give it up from this side. And we were, we were begging, borrowing and stealing resources. From any part of the organization that we could get them from. I mentioned the guy from supercomputer, Eric comes over, right?

Because he knows how to hack on iOS and we're exploring all sorts of different UIs. Noah, the designer, working with Nick on the product side comes up with like this beautiful thing where it can emit good JSON, so why don't we have it, create UIs within the product for the task that you're working on. Right, and we prototype that and it looks awesome. Except it adds an additional level of brittleness, right? So, sometimes it failed because it hallucinated. But sometimes it failed because it didn't generate the UI correctly, or the, the code correctly that then had to be rendered into the UI. And so it just was, it was additional complexity that added brittleness and broke the user experience.

And so we're like, hey, that's cool, but not today. And we, we end up, I don't know, well, it, there was maybe eight people, like you can name them, it was Val, it was Chelsea, it was Eric, it was Arun it was Nick, it was Noah,

Nabeel Hyatt: designers, like what, how, what's the kind of loose structure?

Fraser Kelton: a couple engineers. A designer, product manager and then, and then a whole bunch of researchers. John Shulman is, is like phenomenal. And his team around him is phenomenal. And then the code interpreter and the code gen team is pulled into that because it looks like that's going to be a very valuable tool and. Tina's working on it on their team, it's just a small team of really high caliber individuals and these four individuals from Google Brain show up and this is a hilarious twist. Mid, mid to late summer, they're like, hey, Sam and Greg said we're working on something called The Assistant, is awesome. And the story is they, they were, they were out raising capital for their own startup where they were going to try to build what what in essence was an assistant and Sam, who is just a magnet for talent, said, guys, come on in here and build it. And so the. These

four people show up.

Nabeel Hyatt: resource allocation problem where they're like, we can't hit our OKRs? We don't, like you can't take these three engineers away. And so I don't know, the only way I'm going to get you is like, here's some new headcount.

Fraser Kelton: Oh, it was, it was a blessing from so many different perspectives.

Nabeel Hyatt: Right.

Fraser Kelton: bit of some complexity to figure out who these people were and why they're all of a sudden showing up one day. But they were phenomenal. So they, they were. Not just exceptionally talented researchers along with the rest of John's team, but they were exceptionally motivated to deliver a great product experience.

And so, as were

Nabeel Hyatt: oriented researchers, which is

Fraser Kelton: that's right, that's right,

Nabeel Hyatt: that is so rare.

Fraser Kelton: that's right. Barrett, who was one of those individuals Tweets out a week after the launch and says end user interaction is the ultimate test set. And, and Liam tweets out something like there's a reason we're not at, at NeurIPS.

And it was just two wonderful moments from, from people who, who played a big part in the success of, of, launching and then scaling ChatGPT

Differences between polish & overengineering

Fraser Kelton: the, the thing that is also hilarious in this is. Scaling is hard, right? Anytime. Anytime with that type of load. Scaling from what was supposed to be a prototyping team is all the more impossible.

So, I think that the code was quite sloppy, if we're honest, because we were moving fast and just trying to explore ideas. And then we throw it out into the world in a low key research preview. And the rest is history. So I think that there was a lot of firefighting, not just because of the scale, but because of the history of where the prototyping team had started and the quality of the code

Nabeel Hyatt: you advocating for spaghetti code?

Fraser Kelton: Oh, a hundred percent, a hundred percent, right? . Imagine if we had been iterating on ideas and we push out one of the one of the silly ideas that we had been prototyping, uh, but we only do that after we make the, the code elegant

Nabeel Hyatt: Yeah.

Fraser Kelton: we realize it's a flop, right?

Like a hundred percent, people took the wrong lessons. When this first went out, people were saying, look how crummy the, the login page is. People don't, these people don't even care about the quality of the login page, which is fair. if, if the core of what you're shipping is so, so compelling, people are going to jump through all sorts of hoops to get there.

Nabeel Hyatt: The other conversation I had this week was why is Midjourney on Discord? And why did that work at all? And it's a long complicated set of reasons that I think the specifics of a paid product and, and that particular demographic of users mapped well to MidJourney. But the other bit is that you have a second time founder who although I don't believe in 25 OKRs that everybody's got to hit that doesn't mean you don't use goals to focus.

And I just find like the fewer the goals, the better because the fewer the goals. The more latitude you have to hit the, hit the goal. And, and for him, if the idea is, this model is revolutionary, get it in people's hands and it's better experienced socially with a community or a tribe or a scene, then you're like, what's the fastest way to do that?

And then just ignore every other voice in your head about what that strategically means or whatever else other naysayers on the internet are going to say, what the kind of like hacker news community is going to bloviate about and just get it out there. And, and so in your case, it's like, yeah, the login page sucks.

Everything else about this can suck. If you interact with this product, it's magic. That's the thing. And everything that's not the thing, Can be duct tape and glue, only you cannot compromise on the thing. And I think a lot of people, misrepresent or don't understand what polish truly means, especially in a, in a startup.

And it's I think of it like,

Fraser Kelton: great way of putting it.

Nabeel Hyatt: I think of it like when you're, I don't know if you've ever done any painting, but when, early people learning to paint. They focus on the craft of the painting itself, so they're worried about the way the face looks. And so what you get is these situations where the face looks totally overworked because they're obsessing about it because that's what they're looking at the most, and they miss the big picture, that the concept of the painting sucked, the layout of the painting sucked, and then when you look at a lot of the layers of really good Painters that kind of like they strip away the layers of them.

What you'll see is that they actually worked on a loose sketch form of the whole thing over and over and over again until they got the right feeling of the overall painting and then you increase resolution

Fraser Kelton: Right.

Nabeel Hyatt: it's like They know that the core of the painting, they trust their own ability to execute later on. They're gonna make it look pretty, it's fine.

Fraser Kelton: Right.

Nabeel Hyatt: concept right? And, and

Fraser Kelton: Right.

Nabeel Hyatt: that's gonna then measure through. Obsess about that, and then get to the other stuff. And it's okay to be bad at the other stuff for a while.

Fraser Kelton: Yeah. Yeah. I love that. I love the idea that people confuse what polish means when it comes to a startup and it is not the, the funnel all looks beautiful. It is just that one core experience. And people will, as we've seen, jump through a lot of different hoops to try to get to that if it's so good

Nabeel Hyatt: well, we should talk about any products we've been playing with this week. I don't know if you saw, but it somehow got in the water. I love it when this happens. You remember in AI, there's this, that month where everyone's like agents.

And then everybody like worked on some kind of agentic behavior or layered it into whatever previous AI product that they had launched. Well, the one that feels like it started to crop up this week

Realtime AI Painting

Nabeel Hyatt: is and I slacked you about it earlier this week is, is real time AI painting which probably some people have seen and it's all over their feeds and for other folks, it's still in the phase where you probably haven't seen it at all and it'll start cropping up a little bit.

I, I saw Martin Nebelong, who, who did a completely kludged first setup where he was I think pointing a webcam at a screen that he was on one screen drawing while he then had the model running in real time over and over again against some prompt on the other screen.

And since then, now it's starting to get integrated into software.

Fraser Kelton: So listen, I don't know if you fully know this, but I've been getting my master's in biochem this week, and so I haven't spent any time I saw that you slacked me on that. I didn't even click through. Give me, tighten it up. Like what is, what is it that we're even talking about?

Nabeel Hyatt: Well, the simple idea is. People suck at the English language as a way to visually describe what they're talking about. So the idea of MidJourney and these other things of DALI of me saying what I want to do is, is a hard way to talk about what you want. And so, meanwhile, if I could just have a pencil and a paper, or in this case a mouse and just say, I want A stick figure in the upper right hand corner and a stick figure in the bottom left hand corner and put the sun in the upper left hand corner and I want a plant right in the middle because the thing is supposed to be a plant.

The bottom line is I, I draw my little stick figure drawing and to the right of it beautiful stuff comes out that matches the prompt that I gave it when I'm,

Fraser Kelton: Yeah. Love it.

Nabeel Hyatt (2): combination between English language and drawing.

And I think in particular, what was surprising to me in using it, because some versions of this have loosely been around since the controlnet paper , ten months ago. But the ones that are happening now in real time it's just speed sometimes is,

Fraser Kelton: Oh yeah,

Nabeel Hyatt: speed.

Speed changes the texture of the experience. It makes you make different things. And, and that's the, actually the feeling,

GUIs make it easier, so prompts can get harder

Fraser Kelton: Yep, yep. Yeah, yeah, yeah. Thi this feels, doesn't this feel exactly where a lot of different mediums are going to go where? People struggle and they're like, nah, what I mean is actually like this and, and you wave your hands around or you get a piece of paper and a pencil and you're, you're scribbling.

And, and then my guess is that's where natural language can also come in, right? Nah, make the sun a little bit more golden.

Nabeel Hyatt: yeah. Or, or the simple one of like, I want, if I, if I want to move the sun a little bit to the right. Let me just move the sun a little bit to the right. I can pixel perfect move it to the spot I want it to be in versus describing in English text. Move the sun slightly to the right. And inverse, can you make the whole room more moody is probably better spoken about in text than with a UI tool.

A couple of things. One is, I think it is a continued evolution of us just rediscovering. The fact that GUIs matter, that, that our, our textual interfaces with these models is a little bit like we're in the MS DOS era of computing and then someone figures out Xerox PARX originally, and then copied by Apple afterwards as they met the first GUI graphical user interface.

And then of course, later on windows is it turns out that, it's easier for people to move around and click things. I, I simultaneously hold two beliefs. One of which is people are bad at the English language and that this Interfaces will become more real time and they will become more graphic.

But at the same time, that the language with which we talk to a model,

aka These prompts will actually not become easier. They will probably become more esoteric over time. And I just feel this a year into MidJourney where the way I use incantations for MidJourney is just, it's just in a different world, than where it was a year ago.

And the last thing I want, by the way, is for them to take away That amount of specificity and the way that I can augment and change that model versus say, where I was when I use DALI now I have this problem, right? When I use ChatGPT for DALI, they do a bunch of prompt cleanup against my prompt to

Fraser Kelton: Yep.

Nabeel Hyatt: which makes it prettier, but.

Not if I'm a pro, if I know exactly the words I'm supposed to use to make the model do what I want it to do, it's now some diluted version of what I really wanted and it makes me angry. I have loss of control.

Fraser Kelton: Yeah, it's like you're trying to form an elegant elegant sentence on this podcast. And I come in to try to finish your sentence and stomp all over your words and like butcher it. Yeah, I get, I get what you're saying.

So this is interesting, right?

Nabeel Hyatt: mean is

Fraser Kelton: Yeah, that's right. what Nabeel is actually saying here is this really silly thing.

The, so this makes me think about some of the conversations we've had across this year where we would ask the founder, well, what has ChatGPT done to your business? Like, it can't be good. Remember that the first time when they went, no, no, no, let us show you the chart. And you just saw the inflection point.

You're like, oh, that was November 30th, huh? And they're like, yep. And it was because people all of a sudden understood what AI was possible of and then sought out products with a UI that was specific to their need. Which is

not too interesting.

Nabeel Hyatt: talking about people who might compete with ChatGPT. Did, did the release of ChatGPT hurt their product?

Fraser Kelton: Yeah, that's right.

Yeah. If you're building a, yeah, if you're building a product that allows you to use AI for writing wouldn't ChatGPT be problematic for that? And the answer is no, because they've. curated built, crafted experiences within their product using AI that writers want within a UI.

But the thing that I thought was really interesting and surprising was that they said, but guess what? Every user now says that they want some way with natural language to give feedback at certain times within the product. And so I don't think that we're going to have a world where you are only Talk to your computer or your products.

I think that we're going to, as you said with this drawing tool, there is going to be all sorts of different UI layers that allow you to do certain types of things, and then you'll find very natural ways to fall back into Conversing with your product, which is such a, which is a, such a funny

Nabeel Hyatt: that's true.

Fraser Kelton: with your product, but it's not a chat product, it's move the sun over here and you can legitimately just like draw an arrow with where you want it to go.

That's far easier than say, move it to the right. Nah, a little bit more to the right, a little bit more to the right, versus make it far more moodier or make it, make it a sunnier.

Nabeel Hyatt: The other thing that this makes me think about is not just how these interfaces will affect drawing and just creativity, but what does that say about how we edit other forms of creativity?

Fraser Kelton: Huh.

Nabeel Hyatt: Kapano had a piece of writing that sticks in my head from now a couple of years ago where he said we think about, when we think about editing images, we have this vast array of options that come to mind. Stuff like the blur tool, the sharpen tool, saturation contrast. That's how you think about Photoshop. And what does it mean to have the Photoshop for text? And, obviously the only way we interact with text editing should not be prompts.

Fraser Kelton: Mm hmm.

Nabeel Hyatt: other ways to manipulate the text. And I don't think we've even thought at all about what those interfaces are going to be.

And especially when they get to, to what does it feel like to real time manipulate big blocks of text.

Fraser Kelton: But still having a piece of, of conversing with, with text as well. Right? Like, I think that that's, that's going to be here. Yeah, that's right. That I think if you have the success that they've had with ChatGPT, you rip up all roadmaps and just start with a blank slate.

But the thing that always felt very compelling to me on the roadmap was. OpenAI Auth, in the sense that you bring your ChatGPT profile to that text editor that you just discussed, and it When you're conversing with it, it has all your history of your AI assistant and it just brings it into that product, uh, rather than having to have that product builder recreate all of those pieces.

I mean, they're using OpenAI's API anyway, in most cases, right?

Nabeel Hyatt: hmm.

Fraser Kelton: So why not take the API and the account, your AI account, in essence. Such that is now this, this platform layer going out into the world so that when you are conversing with the products, it happens through, through that mechanism.

Nabeel Hyatt: Yeah. Agreed. mY one last thing on this live painting stuff that it reminded me of is an old rant that I've had, which is

Twitch meets the creative arts?

Nabeel Hyatt: Twitch was originally built for gaming. And then they tried, just like YouTube Live and other folks have tried to expand into things that are non gaming often failing.

And I do think this is a really interesting opportunity to take people that are truly creative and truly artists and remake. What it means to do these creative arts in a live format. Because when Twitch, the reason Twitch is really compelling is that it's often passive activities, so the bar is lower.

And that's very different from the bar to play a video game or write an email, the active activity. It's just, it's usually a second screen thing that's going in the background. It's like, it's like friends playing in the background, right? And the thing that, that the video game does is I can stream it. And the weight of entertaining the, the person watching it is not just on me alone. Because as, as, as you and me know, when we try and just do half an hour podcasts, it's hard to be entertaining the whole time. And we're just having a conversation. And so this way, look, a game is inherently built to be entertaining on a regular basis.

And so the person could just play the game, which is doing a lot of the work for entertaining, and then just speak over it and hang out with people in a lobby setting, right? And if you think of most creative pursuits, Drawing, or doing other things. They've never been designed for consumption.

Like, nobody designed drawing to be interesting to watch. Right? Nobody, nobody But I still think there's a way, like television production studios have found a very repeatable way To make certain types of activities interesting to watch, generally by turning them into competitions. Okay, cookie's not, cooking's not interesting, but if I take ten cooks and I give them weird ingredients and I turn it into a reality show, now it's a competition, right? There used to be an old dance company online social company called Dance Jam, which MC Hammer started, that tried to do YouTube dance video competition things, and I don't know if that's the right structure, but I do think there's probably some kind of, like, baking show meets Twitch, meets live, where The process of doing it is quite different because AI is the assistant that is co piloting next to you, that

Fraser Kelton: Mm hmm.

Nabeel Hyatt: you do things that you could never do in real time, live.

Like, it's,

Fraser Kelton: Right.

Nabeel Hyatt: the replacement for the video game in this equation. It's doing things that still give you agency, it's still you. But then you, you get to not just experience video games, but lots of other hobbies that people do on a regular basis. That's my weird like someone's going to figure this out thing.

Fraser Kelton: yeah, because think about it. Photoshop ping pong. What was that thing called? That was like a hit for a while where, you would add a layer to it, to an image and then send it to me. And I would come back with a new layer added to it. And then you would watch us as a fan every couple of days, up would be built this crazy photo and each turn somebody would try to add the, the player on the one side would try to add something more creative or weird or bizarre, and there's no reason for that not to be live.

Nabeel Hyatt: That's right. And, and the problem has been the, it's

Fraser Kelton: Yeah,

Nabeel Hyatt: and you'd want it to be visual. Visual mediums are best indicated for this.

Nabeel Hyatt (2): except that the speed of creation is so slow . And so that's where you get AI trying to make live work here.

Fraser Kelton: Oh, that's so amazing. We have talked at length about how the idea that the work product now is instantaneously created and then, infinitely edited. Right? So you can say, write me a memo about these five topics and the memo appears. I'm now getting what you're saying is like that same type of, speed is going to permeate all these other things.

You're going to see that in music too, right? Let's, let's create music in real time with audience feedback based on what we're doing and see where this goes.

Nabeel Hyatt: That's right. And not just dropping one beat, but starting to really put together more complicated, interesting orchestrations, which, to go back to the UI conversation we were having earlier, isn't just at the track level, let me drop let me drop in synth right here, but can be one level above that, like, let's make this moodier.

What if the, the ending felt like this? And so you can just pop up and down in altitude in a way that I have to imagine for audiences would be like 10 times more interesting.

Fraser Kelton: Very interesting.

Nabeel Hyatt: That's, this is about entertainment and about people enjoying things with their audiences and so on and so forth. I'm not trying to say it's all art. I'm not trying to draw the line that , this is supposed to go in the Louvre or anything. But people love making. And this might be a new idea or a medium way of using these products.

Fraser Kelton: Yeah, go and call Bob Ross. ai.

Nabeel Hyatt: Fluffy little clouds. I think we should end, we should end on Bob Ross. By the way, I should just say, I was just at a bar last night, called The Ric in Berkeley, which I deeply love. They at this bar had, had TVs all around, which were just playing Bob Ross.

Like that's, instead of, instead of being a spo instead of being a sports bar, it's just

Fraser Kelton: Out of, like, to be ironic?

Nabeel Hyatt: We don't have to judge, Fraser. You can choose to enjoy or not. if you're gonna, if you're gonna be in Berkeley or Oakland I think you gotta know your audience.

Fraser Kelton: That's fair. That's fair.

Nabeel Hyatt: Okay, let's be done for today, man. Well, thanks everybody. If you have a topic you want us to cover or just want to send us, some, some happy notes, we'd love to get them. And otherwise we'll chat with you all next week.

Take care.

Fraser Kelton: ya.

Nabeel Hyatt: Later.