Designing AI Hardware, Prompt engineering vs prompt writing, Suno AI

Nabeel Hyatt: Sure not compelling, but a lot of the, should it be an app? Arguments are also not compelling. Look. There's a tension between the statement. The customer is

[00:00:08] ing

Nabeel Hyatt: always right. And that the audience doesn't know what they want.

Not being understood by some set of customers indicates a problem, but it doesn't mean you should let the customer design your product. if you're going to do something as a piece of dedicated hardware versus this pane of glass that sits in my pocket.

I think you want to play to the weaknesses of your competitor

Nabeel Hyatt: Welcome to Hallway Chat, starting the year 2024. I'm Nabeel. I'm Fraser. we are here to talk about all the things we're talking about in AI, venture, startups, and.

[00:00:47] The era of AI hardware?

Nabeel Hyatt: And now we're an AI hardware podcast, because apparently everybody's releasing AI hardware.

Fraser Kelton: not just the rabbit that we saw this past week. But we've seen a whole host of curious, interesting hardware pitches over the past couple of months. Where do you think

Nabeel Hyatt: this is coming from? Yeah, I mean, some of those are public, but we've had, the rabbit got announced this week at CES.

20, 000 units right now, which is like, man, good for Jesse, good for that team. Humane, a couple months before that, Rewind had some product, Tab. There's at least three more that I know you and me have seen that are not yet out publicly, why now?

I was asking you the same thing this week, right?

[00:01:30] Why it's the right time for AI Hardware startups

Nabeel Hyatt: I think the negative side is that the iPhone and Android, over the course of the last decade, have gone from relatively open platforms to slowly being more and more constrained, as both the large oligopoly clamps things down to benefit themselves, but also for real privacy concerns, right?

They start shutting down APIs, you can only have certain types of access, you can only run things in the background for so long. so part of it just might be the pendulum swing between open and closed and verticalized and horizontal. Right? If you're, if my iPhone can only do so much nowadays, then I might feel constrained as a founder and I want to solve that with hardware.

Yep. Yep.

Fraser Kelton: And you know, maybe a historical piece that we can come back to, but I think on the positive side, my observation is that. AI is just a beautiful new capability, right? it is equivalent to electricity.

And I have to imagine that there were all sorts of crazy gadgets that were tinkered with early on because of electricity. I remember automatic can openers. Plug in and they would open them up. And I think we're going through the same thing where, you know, previously you couldn't do whisper like transcription, if you were a software developer, but now you get that basically for free.

You get things like ChatGPT and Claude and other types of LLMs. Not for free, but like the sophistication for fractions of pennies per query. I think we are seeing people try to understand how to use this technology in novel new ways, free from the constraints of current devices.

Nabeel Hyatt: Counter to that is some of these ideas should probably just be apps. And if you're a founder, a lot of the hurdle is just the inertia to get started with something.

And so obviously tapping an icon on the thing that's already in my pocket is much easier inertia to build a new behavior than buying an entirely new device online. Right. Right. Right. Right. I think Jesse Liu, the founder of Rabbit, had a thread on, like, why isn't it just an app? Because people were obviously asking him that.

And I'm just gonna, I'll pull it up, I think I'm reading, like, his answer here was, like, Apps are easy to build but easy to copy? So that's the first one. apps are hard to maintain, although if he thinks apps are hard to maintain, wait until he's trying to maintain hardware over time. I mean, I've been through that as a founder and that is a pain.

Fraser Kelton: I'll tell you, I'm overly optimistic on these types of experiments. I thought that his arguments for wanting to do it as a hardware rather than an app wasn't entirely compelling.

Nabeel Hyatt: Sure not compelling, but a lot of the, should it be an app? Arguments are also not compelling. Look. There's a tension between the statement. The customer is always right. And that the audience doesn't know what they want.

Not being understood by some set of customers indicates a problem, but it doesn't mean you should let the customer design your product. if you're going to do something as a piece of dedicated hardware versus this pane of glass that sits in my pocket.

I think you want to play to the weaknesses of your competitor. For example, where, an investor in a company called remarkable. And it is an e-ink tablet. That competes ostensibly with the iPad, for note taking, and it obviously doesn't do a lot of things that the iPad does, but precisely because. It is good at some of the things the iPad is bad at. They've really grown very quickly and it's a good business. And for them, it's easy ink, which is important because it's a reflective and non-abusive screen. So it feels more like paper. Uh, and it's single purpose, it's just for note-taking. And it turns out that, you know, for an iPad, it can do a million things, but the fact that it can do a million things and it has this night nice, shiny, bright. Oh, led screen is great for 90% of use cases, but not great for this thing that plenty of knowledge workers do all the time, every single day. In their work life.

Every device in our life this way it has to have. A reason to exist. That is not just 10 or 15% better than me running it on an app. It's going to benefit a lot or get a benefit of the doubt. A lot if it plays to the weakness of a product already have. In our life.

And in that world, is

Fraser Kelton: it an appendage to the phone still? Like it's tethered to the phone and it interacts with the phone

Nabeel Hyatt: One of the other areas, I do think that AI has a real benefit is the battery life. And my worry about, you know, the iPhone or my Android device running out of batteries, if it's always on all the time, which is going to lean itself into AI devices that want to be always on all the time. Right. My guess is the AI stuff actually competes more with the Apple Watch over time than it does the phone. That's for me a lot of the weaknesses that get played. out of the phone are already trying to be solved by the Apple Watch and or by glasses and things like that, that people wear a lot of accessory items that you're always supposed to have around.

I think there's something like 45 million Apple watches sold last year. It's a crazy large market.

[00:06:43] YouTube, and building on top of another platform

Nabeel Hyatt: if you are deeply ambitious as a founder, you're trying to get out from the yoke of Apple and Android, then I can see why you have the conversation about why you need to build LTE cell phones into your device. And that means pay a monthly fee and all the rest of that things because we're going to build the next Apple.

And how do you build the next Apple if you're dependent on it? I tend to think you got to be customer centric first, and then you'll solve the business and strategic problems. Later.

I like to remind people that like, YouTube started out, first of all, massive IP infringement that Google had to work out later, but second, it started out as a, widget of MySpace.

Like, does anybody think of YouTube that way right now? Like, absolutely not. Sometimes it's where are my customers? What do they need? and yes, that's going to create dependencies and worries that will end up being the death of the company if I don't fix them over time. But We get to survive in the first place and be relevant in the first place, which matters.

And I'm not sure I want, well, I will buy another device that costs a thousand dollars with a new monthly fee because I try everything. Right. I don't think the average consumer is looking for another thousand dollar device with another subscription in their life. It's a, the hurdle just became very large.

Whereas the rabbit, I think 200, no subscription. Which creates lots of questions about how in the world they're actually doing that effectively. But at least it hits this price point that I think you mentioned earlier this week, like reminds me of the early mobile phone days, which was also weird and playful. And I am personally just really excited for us to go through a period where hardware is weird and playful again. I love it. I love it. I started a weird and playful hardware company a long time ago that, you know, that didn't fully work out. But man, did we have fun along the way.

so

Fraser Kelton: excited for the return of relatively low cost Experiments within hardware especially with this new capability. I think we're going to see a lot of weird, novel,

and

sometimes interesting things.

[00:08:46] Is AI Hardware destined to be a phone appendage?

Fraser Kelton: Going back to the, question of, you know, is it, does you have a completely standalone device or an appendage?

came to mind when I was listening to you talk is. The iPhone itself started as an appendage to the computer in a sense, like you couldn't set it up unless you tethered through iTunes into your own computer.

And I think easily, more easily imagine a world where you have something that brings this new capability onto your, whether it's a watch like thing, a band, something that goes in your ear, and you still benefit from the capabilities of the phone, but it extends it in novel new ways because of AI.

And maybe that matures into a place where it then earns the right as it becomes more and more capable, to become a standalone thing.

Nabeel Hyatt: Think we're even skipping, we forget the antecedents to the iPhone. So the antecedents to the iPhone before you got this one thing to take over everything. You had so many layers of meeting customer needs where they are, and then moving forward in the constraints of the ecosystem.

it was, it's iTunes first, and the iPod. You release a dedicated device that does one thing, it's not trying to compete with your mobile phone. It's trying to do an friends who said they're not sure they carry another device at the time.

You carry another device because it did one thing super well, right? Right. Then let's not forget that the first version of the phone that the Apple released was not the iPhone. They actually partnered with Motorola on a product called? Rockr? I think it was called R O K R, which came out in mid 2000s, which was touted as the first.

Phone and it was a feature phone the first phone with iTunes built in and that was Apple's way of course of Testing the market . What did it feel like to partner with a carrier? And what did it feel like to be embedded inside of a phone like all that stuff which obviously informed the development of the iPhone which then Of course, famously took over everything, but also, of course, cannibalized one of their own previous products.

I empathize that You want to jump forward five steps as a startup, but I'm in the one step in front of the other kind of camp.

So if we like back up and set some context here, we have a world where startups are trying to do absolutely every single thing from scratch. Forgetting that even somebody, the size of apple actually took some incremental steps in the market to learn what it was like and get really good at those things before they went all in.

[00:11:17] Nobs, dials, and the joy of tactile

Nabeel Hyatt: But on another note, I don't know what the devices are going to be, but it does feel like teenage engineering has nailed the design language of the next stage of consumer electronics. The next 10 years is 1990s nostalgia designed hardware.

Because of exactly what you said earlier, it kind of just reminds us of an optimistic time.

Nabeel Hyatt: Yeah. Yeah. I mean, it's

Fraser Kelton: also the starkest way to differentiate yourself against a flat sheet of gorilla glass, right? Is that you have these aluminum milled knobs that are beautiful and tactile, right? It's, just different. Nostalgia aside, nostalgia then gets sprinkled on and it's profound. I'll tell you.

One, one funny thing for Christmas my kids got CD players, and they're blown away. They're blown away that you can burn music and then put it onto the CD player. And I think that within a couple of months, I'm going to dig up my old iPods and give them to them because that will be the next natural progression where their mind's going to be blown that you can actually have a thousand songs on this little small

Nabeel Hyatt: little thing.

Do you still have? An iPod with the tactile click wheel, because I am still angry at Apple for getting rid of the tactile click wheel. I don't know.

Fraser Kelton: I version three, which is where they introduced the scroll wheel

Nabeel Hyatt: that was not tactile. That was all digital. Terrible, choices.

I'm happy to have us come back to a bunch of tactile products where I'm turning little knobs and so on. Give it, a

Fraser Kelton: month and somebody's going to have an iPod like. device that you'll carry around in your pocket for some new AI capability. It's gonna be cool.

Okay, something else that launched this week.

[00:12:54] The GPT Store: Should founders care?

Fraser Kelton: GPT's, the GPT store. The place to start is if you're So, young startup, do you bring your product to the GPT store? And how should founders think about that distribution channel? Certainly the groups that were early to the App Store, benefited from that handsomely.

And then the groups who were early to, I don't know, the iOS watch store or whatever, the Apple watch store didn't benefit in the same way.

Nabeel Hyatt: Yeah, most app stores don't work.

The iOS and Android being a consummate counterexample. So the default is it's not worth it. But I gotta tell you, if I load up GPTs right now in the store, the very top left corner is, AllTrails to help, because apparently the entire internet needs to find ways to get out into nature and go on your next hike.

And I'm sure AllTrails. com is pretty happy about that placement, right?

Look at the categories and see if that doesn't inform us about what might be working, right?

Fraser Kelton: The categories are, writing, productivity, research, programming, education, and then kind of tucked in at the end, lifestyle

The first categories that we read are workplace like Productivity Tools, and then I think Lifestyle is here's Consumery type GPTs. If you click through, it's Tattoo GPT, it's What Should I Watch, it's Astrology GPT.

Nabeel Hyatt: This is

Fraser Kelton: a tool for people who are business productivity focused, or are students and don't forget it can maybe do fun stuff as

Nabeel Hyatt: well. It's very knowledge worker oriented in its tool set. .

If I was a founder trying to figure out, should I launch my own GPT?

I think there's probably two categories, that would make me think about launching one. I think the first one is If my company can be boiled down into an API, it's a natural language interface into any API to search some kind of database.

And for me that's what AllTrails is, right? Instead of you using some GUI that some random designer came up with some set of sliders that it decided are the things that matter at alltrails. com can I just natural language talk through the things I want and then you go figure it out.

And there are a bunch of examples of that.

The second reason would fall into a metaphor that I heard on Dan Chipper's podcast a little bit ago, and I really liked. Which is to think of LLMs as more of a muse. Then as an Oracle. . ChatGPT interface itself is a little bit of an analogy to this because I'm talking back and forth to it. Um, I. I'm not supposed to get the answer to the first time.

That's what Google is for. And so if you have a chat interface that is trying to be iterative in that way, a little bit more muse, and then am I trying to use the GPT store to try and get really quick market feedback? and use this as a way to gain distribution

Fraser Kelton: I wonder if the Muse versus Oracle delineation is more a question of time

Nabeel Hyatt: horizon,

Fraser Kelton: right?

Nabeel Hyatt: I mean, what you're saying is, Hey, can we reduce the hallucinations over time and we'll just find the right answer and give it to you. That will certainly happen.

I still find myself less intellectually excited about that path because it seems like it's fighting on the same territory as a search engine does, which to a certain extent makes sense and, should happen.

Like the, we've had conversations multiple times about perplexity as a product and its value and, how it's whipped its way into our lives. But a use case where the hallucination is a feature not a bug feels like It has many more years of deep exploration and creativity and many more surprises that might lead to fundamentally different ways that we have a creative partner on a computer.

So that's why I think Muse, not Oracle. And that stuck with me. It's not just the capabilities of these systems today, but just where's version two and three and four. Where are they likely to still yield dividends? Where are they likely to still surprise us?

Fraser Kelton: You used this on me earlier this week when I told you that Neil at Betaworks had given me a great answer for where consumer agents value is likely to unfold. And it resonated, you know, so back to the question, I don't think I'm bringing my product the GPT store. If I'm a founder, I'm probably watching it closely.

I still think the real value

Nabeel Hyatt: of this approach is helping

Fraser Kelton: lower the friction for discovering the capabilities of such a broad horizontal product. and then making it easy to derive value, within that use case, right? So you, I'm looking at like AI PDF, it's one of the top trending ones. I think it's very hard for many people to come to ChatGPT and appreciate that you can upload PDFs and then interrogate it.

It teaches you how you're supposed to use this, what you can use it for, and then it lowers the friction for you doing that.

Nabeel Hyatt: The thing that I've actually practically started using this for is that I have a spreadsheet of saved prompts.

that are useful for different scenarios. it's just a better interface than an Excel spreadsheet full of interesting prompts to do different interesting things. So I think I have six GPTs built. All of them are basically things that used to exist in Google Sheets Yeah, So they also released

Fraser Kelton: the new Teams plan, which is a self serve, kind of enterprise like experience, where you can have your own Teams GPTs administered to just the users on that team. And this is where I think it starts. My guess some of those prompts that you have saved in your spreadsheet make you a little bit more productive day to day.

And the fact that everybody on our team then gets to benefit from, you know, quote unquote for free. We don't have to go tune it to make it work. It teaches us that this product can do this for our day to day job. And then it reduces the effort for us to be able to get that little incremental bit of utility out of it.

That's right.

Nabeel Hyatt: for instance, know this. I put it in Slack. I made one gPT that is a Spark design mentor, which ostensibly looks at our PDF brand guidelines. And then you can give a PDF to it or a document and it tells you how you're supposed to design it.

And I had another one that a friend of mine had put together, which was a set of prompts that do a much better job summarizing research papers in a way that isn't overly pedantic and juvenile, assumes some level of technical capability so you actually know what you're reading.

We'll see what comes out. Hopefully I get to retire my Google sheet of prompts you got to share, your GPT

Fraser Kelton: with me. Okay. So the other thing that uh, top of mind for us is, we're going into the 2024 and we're doing yearly planning.

Uh, and then there's a wave of layoffs happening from big tech all the way down to large startups. What's going on? How should, how should founders think about this as they think about planning

Nabeel Hyatt: for their year?

[00:20:25] New year starts with a thud, layoffs

Nabeel Hyatt: Yeah, we unfortunately opened up the year with tens of thousands of layoffs that I think were really a hangover of the massive over-funding of companies over the last couple of years. Kind of ZIRP environment, wherever they've got too much money and got too fat for their own. Good. Didn't really ask. Fundamental questions about their business. And then the subsequent freezing in public markets and growth markets last year.

And so there's just a lot less capital available and folks need to try and get slimmer. They need to try and. Get more efficient and they try and get profitable. And a lot of that is good. Um, But sometimes I do worry that it goes too far. Ultimately I'm in this industry. 'cause he startups invest in innovation. Not because they're straight efficiency, plays an arbitrage.

So first of all, we should acknowledge that's, uh, it's, painful for the people now out of work , right?

It's probably a lot of very good people that are out of work and, um, I think last year was really a pernicious kind of crisis because when COVID happened, there was a moment in time where there was a shock to the system and you knew what you were supposed to do.

You had to rally the team. You had to rally the, the board. Um, and you had to do things very quickly for your survival. Unfortunately, like the last. Year is probably as deep a crisis. As COVID for many businesses who needed to raise capital, but it just, it, because it's not acute, I think it's snuck up on a lot of founders.

I think many of the cookie cutter V C level advice leads too heavily in one direction, just like the grow at all costs approach from two years ago was like wrong for many, many companies, I'm kind of worried as an ecosystem of startups that now we're like the pervasive desire to just cut all spending and break even immediately is also misguided,

[00:22:24] How founders can pitch taking risk in 2024

Nabeel Hyatt: I spoke to a founder. Who's got a solid business or not in a smart portfolio sitting on tens of millions of dollars of capital. They're set to make double digit millions in revenue this year. Um, but they feel pressure to reduced spending to avoid running out of money, even though you have 36 months of cash, uh, and consider further layoffs to get profitable.

And I don't know, the founder's energy felt very different. It's not what the founder wants with their business.

And so in that situation, I encourage them to explore whether there's a potential business within a pivot. Is there a business inside of their business that would get growth again? And man, if there wasn't like a strong visceral reaction that's like, I don't, how am I supposed to spend money on, on experimenting when all I'm supposed to do is cut?

And, if you're trying to talk to a board in 2024 about assuming you can get the energy to want to innovate and experiment and try to see if you can get growth in the business again, is to think about that risk as a fixed cost, instead of as an ongoing cost, it's a lot. Easier to talk people into you know, CapEx versus OPEX. And what I mean by that, not CapEx in that, you know, how, what the servers I buy or the buildings I buy.

But to almost think of a new experiment for growth as a CapEx expenditure. So, Hey, I'm going to cut my business down. Fine. It can be, it can be profitable if we want it to be. And proving that to you and your team probably relieves a lot of pressure, which is great. And maybe gets everybody bit in a slightly better mood.

And it's going to take six months or one year or two years and a certain number of employees. And you know, it's actually not CapEx. It's just fixed budget accounting, but we're talking about what are the emotional leaps you can take with a larger group to get them to be okay with taking risk. And because without it, the inclination is to wait until you're profitable.

Wait X number of months later, and then try and lean into risk, which might just be too late.

Fraser Kelton: Yeah, I think so. The risk of the pendulum swinging too far feels very real and I will also say it feels like the big tech companies and the ones that had scaled certainly needed, they needed to trim . They had, they had gotten way too large and that was impending their velocity, and their ability to be creative and ship new novel things.

But you, you have talent on the team and creativity on the team that should be, I don't know, invested

Nabeel Hyatt: into.

Ideally you have earned insight, as well . You've been doing this for two or three years. You understand things about this market and this industry and this product that the rest of the world doesn't because you've been in it.

Fraser Kelton: Yeah, you know, and then if we go back to the discussion today about AI hardware, uh, we framed it as an entirely new capability of the likes that we haven't seen in a long time.

And so you have the earned insights. You, you have grown your business from, you know, something that's fragile that doesn't even belong. It doesn't deserve to exist. You have those earned insights right when a brand new capability comes along and you decide to be pumping the brakes rather than exploring novel new ways to capitalize on this.

It makes me think in, in monetary policy, who would have guessed we'd become an economics podcast. Um, there was an era of the Stability and Growth Pact. The European Union were too late to apply the appropriate policy and then, and then applied it too long, when the economy was back, recovering was right when they decided that they had to pump the brakes a little bit. And so they, exacerbated the issue. And things became a little bit too hot, a little bit too cool. And I wonder if we're going to see similar types of things here where the advice is too long, too late to cut, uh, and then you're going to miss investing into the

Nabeel Hyatt: opportunity that exists.

Yes, I think you're talking about the EU mandated austerity policies . Uh, which there's many, many essays about online that we don't need to go through in detail here. I love, I love the analogy of.

Of current venture capital board member advice to, to EU austerity policies of, the 20 thousands and 20 tens.

Fraser Kelton: I Switching topics, I gotta say you had a moment recently , with your call that prompt engineering is going to be a thing for a long time, and that is complex and hard and brittle.

[00:26:54] Prompt Engineering Lives On

Nabeel Hyatt: Oh, yeah, that's right. We had talked before about anthropic releasing some prompt guidance and, uh, and of course, open AI, never to be one up to release an entire prompt engineering guide, six strategies for getting better results on your prompts. It's stuff like specifying desired length of output, which kind of makes sense.

And I think people have naturally done that. But also, you know, asking the model to adopt specific personas, which I haven't always really thought about specifying the steps required to complete the task. In other words, trying to give instructions on specifically what they should be doing step by step. I don't know.

Anyway, it's a great set of things to read, and it just kind of reminds me that there's a lot of work to be done to get your prompts, right. Prompt engineering. I don't believe. E X problem. I think it ends up being a different language that requires a different skill.

And more importantly, for the folks that have tried to get a demo of some internal product, say customer service or some summarization feature for their internal company. If you put an engineer on it for a half a day and you were like, ah, these results are not good enough. It's very possible that you just didn't have somebody write the prompts correctly.

And I suspect that those who take the time and energy to build these props properly for whatever model you're using, Mistral, OpenAI, Anthropic, whatever, there's just significantly better results by the time and energy spent to construct it properly.

Fraser Kelton: Yeah, I think there's a long run ahead of us of prompt engineer as a role that will get hired into companies. And then there will be agencies that help groups pull this into production because they know how to whisper to the, to the models to get it to do what they want to do.

[00:28:31] Snipd's Summaries

Nabeel Hyatt: Oh, my other really simple example of this one, by the way, Fraser, from, that I was just thinking about this last week is, I think we mentioned in our very first episode of Hallway Chat four or five weeks ago, I use Snipped as my podcast recorder now. It does summarizations of the podcast driven by AI.

So it does things like, it ignores the chapter markers that I generate or have us generate for our podcast and it just generates its custom chapter markers and summarizations of the whole podcast so forth. And I've got to say I don't know what they're doing on the back end. But, Snipped. A summarization of a long form podcast transcript is ten times better than anything else I've used in terms of summarization. And I used, I used a lot of different summarization products from open AI or Claudes native summarization products to like Jasper or there are these other voice notes. And nothing is as accurate and captures the nuance of a total transcript in the way Snip does. It's just an interesting example for me of something that I've thought that OpenAI is just like good out of the box at. Like, summarization is Like,

core capability of what these models are good at., and yet, it's kind of good in like a B minus way. And I, you kind of only feel that once I kind of saw it done in a very, very good way.

Fraser Kelton: Oh, I, I totally feel that because , think about what summarization is, right? it is. Condensing down and synthesizing, the content into a context that's easier to consume. , right? And you need to have a ton of different opinions interjected into all of those steps in order to get it to work well for your specific use case. And if you want it to work okay for a bunch of different use cases, you're probably going to be B on any of them. So I'm not surprised that somebody who's crafting a product experience and understands that, okay, a podcast is focused on these types of things with this type of narrative arc, and here's the type of summarization notes that are going to be valuable for somebody who's consuming these, would have a very opinionated, uh, prompt versus if you're just like, hey, summarize this long piece of text.

Nabeel Hyatt: A really subtle example that they must be doing something right, is if I take a transcription,

of

Fraser Kelton: Mm hmm.

Nabeel Hyatt: and let's say I drop it into OpenAI or Claude or something else and I say make chapter titles inevitably, every single, thing will be like, the podcast hosts, which is also stupid, the podcast hosts discuss AI and its implications in product development and then the next one is, the podcast hosts discuss A. I. and a review of SuperPower. it is just this repetitive, and I'm not sure what's, how Snipp structures it, but it feels like English language, wonderful writing that doesn't repeat itself. We should figure that out. We'll follow up on it.

[00:31:19] When do you use prompts vs your roll your own model

Fraser Kelton: To take a big step back, My guess is that as you find increased success and scale, you're not just going to want to do prompt engineering. You're going to have the resources and the desire to sculpt the entire model experience. And you're going to shape the training data that you either fine tune these models with or that you train your own model with. Because you're going to be able to have more control over shaping the experience than if you're just doing prompt engineering. And so my guess is that you'll, you'll see. You know, Snip might not do it today, but if they continue to have success and accrue resources, it will be beneficial for them to train a model bespoke to creating podcast summaries. It might not be a year from now, it might be three years from now, but you will shape the training data that goes into building the model that then is generating the output. You look skeptical.

Hm. No, I'm not skeptical. It is just When does a company use a large generalized model, and when does a company use a small model, and how do I know when to do

Fraser Kelton: Mm hmm. Mm hmm.

Nabeel Hyatt: what? I think that's the question that came to mind.

Is if we can always get better performance and results from not just fine tuning, but from building our own models, then what does that say about how this technology develops over time?

And I'm not sure, intuitively that makes sense to me. Of course, it's less expensive, lower latency, it's a smaller model, that's just tuned to the task that I want to do. And yet, it's often in business, we have found in technology, that people don't do that, because of the resource or expertise required to do it.

And, and a good example of this is just the number of different CRMs out there. If I built a personal CRM just for Spark, which some venture firms do, by the way, then I would get exactly what I want. And in fact, inside of any company, if they built their own bespoke CRM, they would get exactly what they want.

And yet, they don't.

They use some third party provider, like some software as a service, to solve a whole bunch of problems. And for me, the big model, small model problem, when you use which. I don't know how it will all play itself out, but I wish I had a rubric for thinking about when these things will happen and when these things likely won't happen.

Because in lots of other areas of technology, we do yield benefit from bespoke solutions, and yet we don't use those bespoke solutions. We often, in the world of technology, use generalized solutions.

So why is that different here?

Fraser Kelton: Um, I don't know. Does anybody know? I think we're all figuring this out, which makes it so fun. Here, here's a crack at trying to think about it is, um, so if you think about a language model, in this case, is part of the user experience, which is equivalent to the UI. And the output of it is equivalent to the UI. Then it's a question of how prominent and important is the feature that is being generated, uh, by the language model, and how much does Specificity into your use case matter, right? And it may be that in a podcast app, better summarization plateaus or asymptotes in terms of the value that it provides pretty quickly.

let's see if off the top of my head I can come up with an example. If you're only building an app that synthesizes the, uh, transcript from a doctor's conversation, there's probably situations where that data is so bespoke and unique and you have to craft it in a way that an MD wants to consume it. that you would have, benefits from, fine tuning your own model in time. I think there's another vector here where,

When do you take this on? I think that you're probably not hiring your own prompt engineer when you're just trying to find product market fit in your MVP. You might be bringing on a prompt engineer. Because it makes sense for, iterating and eking out improvements beyond that. And then as you're scaling and growing the business, you might find that you get even better performance by fine tuning and training your own model.

Nabeel Hyatt: Right. Yeah, you certainly just start with an understanding and respect for the amount of nuance that there is in prompt engineering and that can either come from external consultants or building expertise internally or just spending the time and energy on doing it and then there's somewhere in the spectrum between writing good prompt to making your own model that we'll see,

I'm just remembering when Stripe came out, everybody feeling like, You know, the core of Stripe, as another piece of middleware software, is actually relatively simple.

It's not that crazy. And you can build it yourself. there was a pretty common point of view at that point in time that it's such a thin layer of the wedge. It's just talking to banks with existing APIs, that anybody who got to sufficient scale would just pull off of Stripe and build a direct integration.

And the early days of Postmates, uh, I remember we were the largest customer of Stripe and Bastian would come in every quarter or two and talk about how much we were. Paying them and how, of course it's something we could build and we'll probably be building some kind of direct integration at some point soon because it's just a little bit of software and. And I just kept scaling. With us.

And obviously, as you scale, you'll You

Fraser Kelton: But, but that's a great

that's a great way

Nabeel Hyatt: But that's not what happened, right?

Fraser Kelton: Right, but why not? Here's a hypothesis, is Incremental benefits on reducing fraud or simplifying the payment transaction accrues very little value to your business. Because it's like plumbing, it's infrastructure, right? Whereas, if you could redeploy those resources to make the product experience that your end users or customers are benefiting from, that's going to be much stronger.

[00:37:04] My model vs yours, it depends on the goal

Nabeel Hyatt: Or maybe even more broadly, put simply, there's just a tension between using an external software provider and what ongoing benefits they're going to provide versus what you would build if you did it internally. So I may get better cost reduction, and better tuning by using my own data and making it smaller and building some small model.

What I'm going to miss is the 30 other things. That a large model AI company might be shipping over time that might help me.

Fraser Kelton: Yeah,

but I also don't think we're, you're, you're not going to have a world where you choose. It's going to be both. And then certain use cases we'll use. We'll use one versus the other. It's been wonderful making this podcast because now I have a real use for Descript, which is the AI supported, audio editor, among other things,

and Bye.

Bye. we know that they have a bunch of different models, woven throughout their product to enable the experience, many of them provided by OpenAI, but they've made the decision to train their own model in certain cases where they can get not just the performance improvement, but that performance improvement Um, extends the valuable user experience in a way that is worthwhile for them.

Nabeel Hyatt: Yep.

Fraser Kelton: Right?

Nabeel Hyatt: That's right.

Fraser Kelton: Okay,

Nabeel Hyatt: Maybe a broader way of putting it is like, It's like the old Steve Jobs quote, You have to start from the customer experience, And then you work your way back towards the technology appropriate for the experience.

Fraser Kelton: Hmm.

Nabeel Hyatt: To an extent, all of this is, It's an interesting bass ackwards view because inevitably if you're talking about a Cambrian explosion because AI just hit, you're having the opposite version than that conversation.

You're not saying, here's the thing I want to make for a person, it's ooh, this thing is a generalizable model that can do lots of different stuff. What's it, what can it do now for my problem?

And uh, that is always a fraught product development

[00:38:54] Consumer Web Agents

Fraser Kelton: Okay, so along those lines, along the lines of what can it do for my problem, I want to transition to a question that I asked at the holiday party that fell fell flat. It was so miserable. People looked at me like I was stupid. The music came to a screeching halt I've spent a little bit of time trying to figure out the answer or at least a set of hypotheses to what you just said within consumer facing web agents. My question

Nabeel Hyatt: take an example to talk about what a consumer facing web vision is for those who are not

Fraser Kelton: sure. At the start of this year, there was a lot of hype around, um, these autonomous AI products going out into the web on our behalf. and accomplishing tasks for us. Sounds pretty awesome.

Sounds pretty awesome. here we are toward the end of 2023 and we have not seen a lot of those take flight in the way that the hype might have suggested at the start of the year. My question to you is, What is the future that people see when consumers are, when people talk about consumers benefiting from a sea of agents going out onto the web and doing tasks for them?

Nabeel Hyatt: I think you specifically said, why do I want an AI agent to book my airplane flights?

Fraser Kelton: why do I want an AI agent to book my airplane flights?

Nabeel Hyatt: Oh, I don't want a, uh, AI agent to book my airplane flights. There's a lot of nuance there that I'm not sure that could get captured. There's two ways to answer this. One is a systematic view of what are AJA agents going to be good at.

And then the more near term view is just taking the flight example. So, I have an EA that, that books flights. for people who have executive assistants who are very close to them,

and they

Fraser Kelton: hmm.

Nabeel Hyatt: their preferences, what they like and don't like. it's not un outsourceable work. booking a flight.

I would not have a new friend that I just met, uh, or a new EA that I just joined that just joined Spark or something like that. I wouldn't just hand 'em the task. And even though they would know technically how to use Expedia, I would not give them the task and be like, Hey, I'm going to San Antonio for a quick meeting. Can you just book this? Because they're gonna mess it up,

right?

[00:41:08] The problem isn't Actions, it's Context

Nabeel Hyatt: I think the important thing is to talk about whether that product has the proper context to make a good decision. And how do we get that thing the right context, right?

The agent could sit at the provider level. So it could be like Expedia 2. 0 Copilot. That's one part of the altitude, right? Or it could be at the aggregator level. So the same way that Kayak aggregated all the flights from across all the other websites. It could be a copilot at, at that level as well. Hey, we're go anywhere and book anywhere.

That's normally how things, people think about it. But, I think the more expansive opportunity is probably something that sits an entirely new layer of the stack that feels kind of more Google like. And it's a value add in terms of its aggregation. In other words, it has not just context about flights, so it's not Kayak 2. 0. It knows that, my wife is pregnant right now. And so, I'd be more willing to take a red eye now than normal because I gotta make sure I get home. My wife's not pregnant by the way, there's no, I'm not

Fraser Kelton: Congratulations.

Nabeel Hyatt: heh, heh, heh, heh, heh, heh, heh. Heh, heh, heh, Uh, just a, you know, just a life note.

Clarity. So, in other words, if it feels like this If this agent has a huge amount of context about my life, that will make it 10x better at doing any one of these particular tasks. Although I will say that I don't know that any consumer agent product has tried to be the context agent so far.

If you just think about our previous conversations about data defining the consumer experience at the end of the day when it comes to AI. Then, if I were trying to think about the best consumer agent product is, then my first task would be, how do I get incredible amounts of context about the wants and needs? much the way of an incredibly good executive assistant works. They know you better than you know yourself.

Fraser Kelton: Yep. Yep. I, I, I can partly get there on that. The, my challenge is

Nabeel Hyatt: You still book your own flights, so I feel like I'm speaking to a very late adopter here.

Fraser Kelton: I might have some control challenges here. no, but from a consumer perspective, like a flight is a very infrequent, risky proposition where you're balancing a bunch of different criteria. Uh, are you willing to go 20 minutes earlier to save 200? The trade offs are complex for most consumers.

My guess is it's either. Uh, sitting at the higher abstract that, you talk about, and, or it's just a, a new interface for interacting with the kayaks of the world, where I netted out on this overall is, it feels like the consumer agent, maybe I'm just riffing off of what you just said around context, is it's really going to be about search and research and then effectively using summarization and synthesis. I wanted to come back and say, based on the criteria that you said here, the four flights and prices that might make sense. And then you're

going to have

Nabeel Hyatt: that just Kayak Copilot then? You would start from basically the Kayak Copilot use case. Nothing more advanced than that for you.

Fraser Kelton: I have a really hard time seeing the future that people see. I think that information density on these flights is so high for consumers when they're trying to make a Complex decision for them and their family. And my guess is it's not going to happen.

Another reason to

Nabeel Hyatt: mean, look, but we'll just, yeah, I mean, like, like, look, there's probably some, you know, 2x2 matrix or a spider diagram or some other chart that can talk about what use cases for agents would be low hanging fruit. And what would be, maybe not the right agent framework, obviously complexity is very good.

Steps where context matter a lot are good.

And some of this, by the way, comes out in the B2B context that we're going through with Adept. As Adept has been working with customers, we're seeing some of these same patterns play out. You can imagine that a task on the B2B side that a person does 15 times a day, that's 35 steps. with just a little bit of nuance and change on step 3,

Fraser Kelton: Mm hmm. Mm

Nabeel Hyatt: An RPA is not good for this. those are perfect for a product like Adept as a,

as an agent, really. And on the other end of the spectrum, of our little matrix is if it's an infrequent activity that is only a couple of steps, with huge amounts of context needed.

Then, probably a very bad example of something you want to use an agent for. A good example of this is go buy a good gift for my wife, uh, for Christmas. There's so much contextual knowledge required that I would first of all need to be able to impart context about my wife into a model.

Might be tough for me to do.

Fraser Kelton: Right.

Nabeel Hyatt: I'm trying to figure out her preferences, let alone try and feed her preferences into a model. Um, and then at the end of the day, the task itself, once you know what you want to do, is incredibly simple. Amazon. com, look for thing, hit buy, hit purchase. Like, why do I need a model to do that,

Fraser Kelton: That's right?

That's right.

Nabeel Hyatt: simple and infrequent, you know, loosely, probably AI architecture around. And repetitive, long, complex, with lots of small decisions inside of them,

you know, you got, you got a good candidate.

Fraser Kelton: Yep. I agree with everything that you just said there.

Where I come down again is like research and search use cases broadly are going to be where consumer web agents thrive and it's because it can go and do summarization and synthesis over a lot of different content. There's no shortage of people trying to provide web agents into travel booking.

[00:47:08] Exploring Bard and Gemini Pro

Fraser Kelton: I tried BARD this past week because it was a good way to play around with Gemini Pro. And one of the extensions, a way that they allow you to bring,, BARD to different experiences is for Google Flights.

And I tried to book a flight through it. It was so painful. It was, it was as painful as the first plugin that ChatGPT launched with Kayak, where booking a flight through natural language is just a terrible experience.

The thing that I want to talk about BARD

Nabeel Hyatt: So your first bit of advice before you get to the BART bit, your first bit of advice is, uh, Hey guys, if you're trying to pitch a consumer agent, can you please use an example other than flights? That's

really the

Fraser Kelton: There's a, uh, a thing to ponder is why, why have the two large players in agent space as well as almost every startup. pitched it as, Hey, and we'll help you book your flights. And I guess it's because it's a super complex problem that we would all love automated. And I feel like if you just push a little bit beyond that, as you said, it requires EA type insight, to be able to get there. And these technologies and products are far, far from having that level of knowledge and, and memory and context.

Nabeel Hyatt: our second Steve Jobs mentioned in a single podcast, which is definitely over the quota. But, you know, if

Fraser Kelton: Careful.

Nabeel Hyatt: understand anything else, then maybe you don't understand how to pick a good use case for a demo. And

flights might not be the right use case

Fraser Kelton: So I'll tell you a good demo that I tried with BARD, was the ability to plug it into. Gmail and Google Docs. I encourage you and everybody else to go try it. It was still rough around the edges, but it was pretty awesome.

I asked it about context that I knew was sitting someplace in my email archive. And it pulled out the right context and then it had a deep link to the specific email, which was great. and it was part of a large project that I'm working on with my wife. And I knew that it was in reference to a document that we had created a couple of years ago in Google Drive.

And I was like, well, how does this compare to the Google Drive example? And it said, I think you're referring to this page. And based on that, here are the five different things that, that, aren't completely aligned. And you just thought, oh, you know, like this is going to be an amazing experience when it gets, uh, polished up and mature.

Nabeel Hyatt: First of all, that's great. It's good to know, especially given our conversation about Gemini previously , that now that they're actually allowing us to play with it, that it's, it's useful to you when you use it.

Fraser Kelton: Yep. I got, I have one more example about BARD, that really highlights the importance of context and memory. Because memory, I think, is one type of context, right?

Nabeel Hyatt: Oh yeah.

Fraser Kelton: think about it. Don't tell my kids for Christmas, we're going to give them a gift of taking them to Disney in the new year.

And so I gave Bard, ChatGPT with both 3. 5 and 4 the same prompt where I wanted it to help me come up with an itinerary for the trip.

I kind of nudged it at the end. I said, and our eldest is really into Harry Potter but we're going to Disney and we're staying at this spot in Disney. and. Uh, I want a three day itinerary.

What, what do you have, right?

All three, so ChatGPT with 3. 5 and 4 and BARD came back with the same itinerary. And all of them left off going to Universal to see Harry Potter.

And I asked Bard, I said, what about Harry Potter? And it said, I'm glad you asked, in the final story of Deathly Hallows, Harry Potter, and then it told me the end of the book! It told me the end of the book, which was so funny, I laughed so hard, I'm like, well, thank you, Bard, thank

Nabeel Hyatt: Thanks for ruining the book for me!

[00:50:42] Perplexity, Search and Synthesize

Fraser Kelton: Okay. Now, now the importance of context, uh, on a LARC, I went to Perplexity to try the same prompt, and it is not the typical prompt that I would take to Perplexity. And I will tell you, Nabeel, with the copilot setting on, it was awesome. Awesome. It was

way

Nabeel Hyatt: did the itinerary thing with Perplexity as

Fraser Kelton: exactly the same prompt with the nudge at the end of saying, my eldest loves Harry Potter, right? Perplexity gets that. It comes back and it says, Do you have any parks already in mind that you may want to go to? And it

showed me a list of the options, and you selected the options. And so I just went, oh yeah, this one. And then I hit go and it was great.

Nabeel Hyatt: And why do you think that was so much better of an outcome for you as a consumer than Claude or ChatGPT in this case. Like, what was it about that need? Even without the context that it had of your whole life, the way St. Barth did.

Fraser Kelton: Right. This was a search and research task. And then it had the wherewithal to pull out additional context. Just a small drop of context from me helped nudge it in the right direction. And then it was optimized around, pulling up the information from the web that was going to be most useful for me, given the context I provided. What did you call ChatGPT and Claude like products previously? You called it

Nabeel Hyatt: Oh, yeah, search is basically three products for me at this point, right? There's Google, which I really only use if I want the one box result. That's the first use case. And then the second use case is ChatGPT. Essentially as a Q& A framework. If have a question I know I'm going to have follow ups for,

Fraser Kelton: Mm hmm.

Nabeel Hyatt: is the best outcome.

And then the third is perplexity, which is my research companion. When I wanted to summarize a whole bunch of data and do a good job, go do the work and present it back. And yeah, so Google's one sentence, perplexity is one page, chatGPT is, back and forth, back and forth.

Fraser Kelton: Yep. Yep. Yep. And this is that, right? This was a research task where it collected a little bit of additional context. The other groups didn't care about collecting that additional context. And then it went out and did the research and the synthesis far better.

[00:53:06] Prompt Engineering vs Prompt Writing

Nabeel Hyatt: Yeah, it's a little, it's reminding me a little bit of understanding when you're writing these prompts or using these products, the difference between prompt engineering and just prompt writing.

Linus lee at Notion,, what he said that resonated with me was he separated out the way he thinks about writing a prompt when he's just talking to ChatGPT versus the prompt engineering that he does when he's writing for Notion is like It's like the difference between scripting and programming.,

software engineering involves writing robust and reliable programs that are shipped to millions of users and run on countless computers. Whereas scripting involves like writing quick and easy programs for one time use .

Fraser Kelton: Mm.

Nabeel Hyatt: Maybe one way of thinking about this is , your problem here, it turned out, when you're trying to look for travel suggestions, was a very, very simple summarization problem.

Fraser Kelton: Right. Right.

Nabeel Hyatt: there are 30 articles that try and answer your question. They're SEO'd, they're terrible, but , there are 35 guides on the internet to what to do at Disney for the weekend. And so, that is less of a prompt engineering problem alright? So, if it's a simple solution to summarization, then I'm going to use Perplexity. And that's why you probably should have been using Perplexity versus BARD. Because the answer is sitting somewhere on the internet across 30 articles.

The problem is you're just gonna take you an hour and a half to click on 13

Fraser Kelton: Mm hmm. Mm hmm.

Nabeel Hyatt: all the SEO juice, try and get out the real nuggets of wisdom, and then form an opinion based on that aggregate data sitting on the internet. Perfect example of when perplexity is useful. then there are other situations that feel closer to almost like programming.

For instance, I was trying to do some sample problem sets with my son last night. he was trying to study for his math final. And, of course I can try and feed one question into ChatGPT or Claude and say give me a sample problem based on this. the issue when you do that, of course, is that the model doesn't have enough context to really give you good problem sets.

So I would give it, I would ask it for a problem that involves You know, X, Y, Z thing, and it would give back wildly wrong problem sets that, for instance, he's in pre calc right now, that are like, calc two questions that have the same concept in them. And so I really started to think more like a programmer than I did like somebody using a search engine after a little bit.

And I had to kind of build a multi step process prompt, like a prompt engineer, that said, I am Studying for a test, I'm in pre calc, these are the concepts I know really well, these are the concepts I don't know as well, just like actually trying to structure thinking. Please think through each question you're going to give me, then I had to actually check each answer to verify that the answer was right before proposing the question so that it didn't get it wrong.

Again, it's the difference between thinking, having to think like an engineer, and just, and just talking.

Fraser Kelton: Well.

[00:56:15] Suno AI

Nabeel Hyatt: of AI products, we should, we should cover our AI product of the week.

Fraser Kelton: Yeah, what'd you play around with? I, I've been busy, playing with Bard. What have you played around with?

Nabeel Hyatt: You played with Bard. I have been playing with SunoAI.

Fraser Kelton: Oh, why am I trying to book flights while you're making music? We, we, I should have switched this around. Tell, tell

us what SunoAI is.

Nabeel Hyatt: fOr context, this is the ex Kensho team, the guys who built Bark. They have built a model for making music. and it just came out of stealth. Although we've talked with them before. and I think there are two bits about this. One is that it's a music AI generation startup, of which there's a bunch of different research orgs that have done some work there.

Then the other remarkable thing about it is it's, it is both real time music generation, so beats, but also real time lyrics and, and singing. So as you make a song, you get, actually somebody singing the whole song.,

and

Fraser Kelton: it, M& M who sings the song or who's singing it?

Nabeel Hyatt: non IP oriented.

Fraser Kelton: Non IP.

Nabeel Hyatt: is a great question when you talk about music or any kind of creative arts. Yeah, you, you're not doing This sounds like Eminem or Beyoncé. You can't hear it right now, but what I'd love to do is, hallway chat has a theme song, which I actually built from some random third party AI music generation service in about 15 minutes as I was trying a bunch of different Kind of first generation AI music startups.

That's what you hear when you first turn this thing on. It's not some massively professionally produced thing. It's something I made in 15 minutes.

Fraser Kelton: I don't think, I

don't think you tricked anybody with that.

Nabeel Hyatt: But it does feel like the only way to get a proper theme song for hallway chat, given what we talk about every week, is to have an AI generate.

Fraser Kelton: Uh huh. I agree. I agree. Let's, you want to do it with Suno?

Nabeel Hyatt: I want to do it live.

Fraser Kelton: Are we, are we going to have lyrics?

Nabeel Hyatt: We absolutely are gonna have lyrics. so what I, uh, first of all have to say is I love that we are in a world where the three account creation processes are Discord, Google, and Microsoft.

Fraser Kelton: and Discord gets prime billing.

Nice. Nice.

[00:58:27] What products lend themselves to Discord communities

Nabeel Hyatt: think it gets prime billing for a very specific and very smart reason from Suno. I don't think everything should. I don't think everybody needs a Discord community.

Um, but I think the specific products that do well on Discord, and MidJourney obviously is the best example of this.

are products which have a very deep amount of expertise and knowledge to

Fraser Kelton: Mm hmm. Mm hmm. Mm hmm.

Nabeel Hyatt: you know, if using your whole SaaS product can be described in a FAQ and a PDF, then there's not a lot to discuss every day on Discord. But if it feels more like a scene where the idea of using the product is constantly changing, then it's probably a really good Opportunity for Discord.

And Mintry is exactly that, right? We're constantly discovering new prompts. We still don't know fully Everything you can do with this product and so on and so forth. Which is very similar, frankly, to video games. Where the meta changes and certain types of cards in a playing card game are different over time.

Those kinds of things with real, real depth, those are really good for Discord. And I think, obviously, music generation is a perfect example of this. So the first thing is, a little bit of a kludgy interface, we have a couple of trending things on the front.

It's fine. I'm gonna hit create. We should

Fraser Kelton: polish in this case is not going to be related to the interface. So I'm going to forgive, forgive this at this point.

Nabeel Hyatt: Oh, well put. The, the polish doesn't need to lie in the interface level, because ideally the polish Lies in the model level, right?

Fraser Kelton: got it. Well, model, yeah, model and how you interact with the model is all that's going to matter right now.

[01:00:07] Live test of Suno AI

Nabeel Hyatt: Yeah, okay, so Create. We're at Create, and we're immediately given with a very typical Prompt Chat Experience . I'm supposed to type in a song description, and I'm supposed to describe the style of music using genres and vibes instead of specific artists and songs.

You think upbeat. Upbeat jingle. Ugh!

Fraser Kelton: for a, uh, a podcast. Focused on AI products and tech startups. the problem is that it's going to be the, the prompts that it generates for these types of things. We we've done this before, man, we did that with the titles. Remember it, it doesn't have taste in the, in the way that we're going to want it to have taste.

Nabeel Hyatt: Oh, but I, I feel like that's a cheap cop out. I feel like this violates the law of good prompting. Do not blame the output. Blame the input, my friend.

Fraser Kelton: Ouch, that hurts. I think you're right. I think you're right. Okay. So, um,, hold on.

Nabeel Hyatt: Okay. I wrote a prompt. That is a terrible song.

Fraser Kelton: How much is that on us?

So talk about polish here, right? It's not just the model quality. It's also the fact that it generated these two in only a couple of seconds, which is an amazing experience.

Nabeel Hyatt: Took a couple of seconds. It also produced a little piece of image art. .

Fraser Kelton: Here's my hypothesis on this, we know these models are quirky and have to be shaped in certain ways. And I think we're still at the earliest stages of figuring out how Suno even wants us to speak to it. And with 10 more iterations, we'll get our head around that.

And then on the 20th time, we're going to get something that, you know, sounds okay for what we want.

Nabeel Hyatt: I think that's a good summary. The interesting thing about music and anything that's creative is that it's so multivariate. Here we go. .

Fraser Kelton: Mm hmm.

Nabeel Hyatt: And so similar to what we saw in image generation, I'm sure there will be syntaxes that will develop to how to speak to this model to get the things you really want out of it.

That will sound very similar to the conversations we've been having about prompt engineering and the conversations that I have with people about image generation. But there's real depth here. I mean, it's quite good for a very fast, quick beta output.

I wish it had negative prompting. I don't really have the musical language to know what more I want, because I don't have a lot of musical language.

At least I can have the things that I'm viscerally negatively reacting to, and you can do less of that.

Fraser Kelton: The thing that makes it great is how fast it is. I can't believe you get this output of two songs, lyrics as well. ready to listen in what feels like less than 20 seconds.

Nabeel Hyatt: That's right.

Fraser Kelton: Listen, this is again a glimpse of the future. My guess is that if you think about infinite music creation combined with an effective filtering, we're very quickly going to have a lot of great music that bubbles up for us that's purely AI generated.

Nabeel Hyatt: I'm going to tell you, somebody's going to make a hit song very soon with this thing. And whether or not it's an artist who shapes it in the sense that we think of artists shaping, or it's just somebody cranking the 100, 000 times until something awesome comes out. This is going to be amazing. It's going to be so crazy in the next couple of months.

I agree. I agree. Well, let's leave it at that. Uh, which one of these, Fraser, should I open the show with?

Fraser Kelton: Your choice. You got to curate that experience. I want it to be a surprise for me.

Nabeel Hyatt: Okay, will do. Uh, thanks everybody for listening. If you got any products that you want us to talk about let us know. And if you have any other comments, um, you know, humans run off of positive energy. We wouldn't mind a little positive energy.

Fraser Kelton: Let us know. See ya.

Nabeel Hyatt: I'll talk to you later, Fraser. Bye bye.