Alan Cline

Thank you so much for joining us here. It's a great pleasure that we get to spend a little time facilitating a conversation. I'll give a little bit of background, I'll brag on him so he doesn't have to.

Raghu is a physicist, an engineer, a business leader who spent 25 years since grad school focusing on different ways of innovation. You spent time early at Netscape, starting in 98 during the Internet boom, which we're all familiar with. Spent time at Bang networks, doing real time network development, really trying to bring the Internet to to a much faster pace use. And then in 2003, started your journey at Vmware, which everyone's generally familiar. Vmware. He joined when there was less than 200 employees and then your journey there over decades started in product with Hypervisor, but really grew the business to, I think, correct me if it's wrong, to the sale of Broadcom,13 billion in revenue, roughly 4 billion in cash flow, 38,000 employees sitting in 60 countries.

An enormous scale. As we talk about the exit, Broadcom, I guess there's things in there. An interesting one is, I think it's signing is just short of 70 billion. And then you had a very long approval process. Which meant that you exited about 85 billion. So that's the one time a long regulatory process paid off by15 billion. I wish we could all say that about long regulatory processes.

So when I look at this. We talk about a bunch of things, but I think there's 2 really big dimensions that I think are useful. One is, you know a quarter century of participating and leading innovation in a few of these changes, particularly with what happened with virtualization that VMware drove. And then what we're seeing with AI today. And then, second, is running an enterprise that was a very big consumer of these things all along the way.

So I think we'll get to that. I do want to start something a little more personal. I'd love to start with just a little bit about something you had, maybe even from your childhood, that really has leveraged who you are today.

Raghu Raguram

Yeah, he warned me that he was gonna ask this question. I still don't have an answer to that. But actually I was reflecting on it. I grew up in the 70s and the 80s in India and India at the time: technologies came late. And in the 70s, if you remember, that was the age of all the lunar missions in space, and all the polar stuff, and space lab, etc, etc.

So very early on that sparked my interest in science and technology. And believe it or not, even TV came to India very late. In the early days, I still vividly remember when the Munich Olympics was going on, I was middle school, or something, primary school. We had to go somewhere to someplace common to go watch it. So this whole fascination with technology for me started there, right? And it's continued on.

And then in the eighties, you, you have the advent of the PCs. And the microprocessors, and so on and so forth. And so that's really the formative experiences of my life that led to technology.

So that still stays. In fact, that a couple of days ago, just before boarding the plane I downloaded the new book by Ashley Vance. I don't know if anybody has read it already, if they've read it don't tell me. It's ‘The Heavens went for Sale’. It's a story of how space is getting commercialized.

Alan Cline

Another frontier. So the other thing I thought would be useful touching on before diving into some specifics would be getting to Netscape early. And then, frankly, even the innovative work you're doing at Bang. What do you look back on that and say, Hey, what's the analogous lessons that you think you're applying today to the AI revolution?

Raghu Raguram

Yeah, I mean, there are some commonalities. But fundamentally these are 2 very different things, in my opinion. Yeah. I mean, the commonality, of course, is a use case that the even a middle schooler anywhere in the world could understand and appreciate and get excited about, right?

So when these things happen, enormous amounts of money pour in, right, and people start using it without regard to things like ROI, etc, etc.

While we are all business people and want to think rationally, the greatest technologies always happen when people ignore ROI, because in the early stages there's so many other things that make it difficult to realize that. I think that property exists in both, right, in the early Internet age as well as this.

And when these things happen, there is usually a lot of overbuilding that happens, and there is a boom and bust cycle before things get to normal, right? A lot of things to get to significant value creation.

So I would say, those are probably the similarities but fundamentally these are different, very different things. The Internet is, as the name suggests. It's a network, right? Whereas the AI phenomenon is a computing phenomenon, right? So if you think about AI, at least the modern excitement, it rests on things: one is Nvidia creating a cloud computer, basically, that's really good at linear algebra, right? If you think about it, what can it do? It can do vector, operations and it can do metric multiplications and matrix operations. So it lends itself to these high dimensionality problems where in order to represent something you have to use a lot of dimensions.

And then the second thing is, there is an algorithm that came about in 2017 which uses that linear algebra processor to say, if you give me a sequence of observations there is a very, very high likelihood that I'll predict the next set of sequences on that, at least for human data. We don't know how it applies to climate data and all those sort of things - natural data.

So fundamentally, this is a computer and a algorithm that uses a computer. Whereas the Internet was a set of network protocols and inherently that the network effect property in it, right? And when you had a network device that was ubiquitous, that got attached to it, which was the mobile, then the whole world took off. So I think the economics of distribution, the economics of access, the economics of value creation, I think, will be very, very different.

So when people get carried away by saying, "hey, this happened in the Internet, this is going to happen with AI", I sort of wince a little bit because, if you look at it from a first principle's point of view, these seem to be very, very different things.

Having said that both are going to be tremendous transformers of productivity at large in the world. So there's no question about that. But for all of us that are in the business of building and operating companies, or in investing companies. I think you had to look at it slightly differently.

Alan Cline

That's super interesting. And then so what has that meant for over the last kind of 12-15 months with what you're you're doing at VMware, both internally and externally. How did you apply that to what you're gonna do for your business and your customers.

Raghu Raguram

Yeah, firstly, because this is a computing phenomenon, right, and we are in the computing business. So we took that very early, very seriously right?

If you think about virtualization, virtualization is a resource management technology at its core and an infrastructure management technology, secondary, right? And thirdly, an enabler of elastic computing.
That's the foundation of cloud, right? So we said, how can we apply all of these to this new form of computing: GPU instead of CPU.

And, by the way, the GPU is super expensive, right? So today, if you wanted to, if you had a hoster and you wanted to start up a GPU and delivered to end users. It cost you about $2 an hour, right? If you take the computing costs and power and cooling and everything etc.

If you are a CPU hoster it probably costs you like between 5 and 10 cents an hour. So there's a massive difference. So there is a significant impetus for us to say, How do we make this a lot more easier and more manageable and so on, and so forth.

To my earlier point, I don't think this is going to become a worldwide technology unless the economics changes and the good news is, it is changing, but it's got to change even faster.

So that was first thing that we did. And because we were serving large regulated enterprises and governments and defence who constituted, 60% of our 70% of our revenue, we said, let's apply it in their domain, because for them 60 to 70% of their data was still inside of their data centers. So that led us to pivot and create a new category of computing that we call Private AI, which is now shipping under Broadcom, and it's off to a really fast start.

So that was one. The second was because this property of being able to predict linear sequences really well, right? We said, Okay, where can we apply it? And the first thing we started was code.

In most companies, software development, that's probably your most expensive resource. Certainly for software companies in the audience. So that's where we applied it. And then the other interesting observation as we applied it was: almost everything in a software company is tied to code. You think about it, all of your marketing data sheets and white papers, and everything is really about what the code is doing, right.

What your professional services organization is going and bidding with the customer's statement of work is really about what the code is doing. What the sales engineer is presenting is all really about what the code is doing, right? So once you add one, we found that you could very easily start generating all of these things.

So we started creating internal enterprise language-model based infrastructures, if you will, that could spit out based on the code, start to say, ‘Okay, here is how we will create content for these sort of things and start to help these various functions’. That was the second thing that we did.

And the third thing that we did was what Microsoft has popularized now as the copilot kind of a model.

When you have a lot of proprietary data yourself, or when your tool controls a lot of proprietary data: in our case our customer was the data center administrator and the security administrator. We had 15 years of data on how they wanted to administer data centers and manage data centers. So we could take all the data that we had in the 10-15 years of help desk requests, and so on, and so forth, that came from customers knowledge bases. And then we downloaded some models from hugging face, and then we constructed a whole new thing that we sold as an adjunct to our management products.

So those are the 3 ways by which we did it.
How could we use this to open up a new market?
How does it make our existing products better?
And how do we transform our internal operations?

Alan Cline

That's great. I think it's great because it's a microcosm of what everyone's wrestling with, and so maybe we hit hit a few of those. So on your internal operations, maybe give a a little bit of a sense of what you started with, and kind of your rough, your rough view we talked about this morning on impact.

Raghu Raguram

A lot of what we did was within the first year of when Chat GPT became public because we had started working on encoding models slightly even before that. Strong summer, like we said, has been around since 2017.

So we did some forward projections. So if you think about VMware as a company, we were 13 and a half billion dollars in revenue, about 10 billion dollars of OPEX, 9 point something. Out of which, roughly, 35% was sales and marketing 25% is R&D, right?

Cost of goods, including support and professional services and the cost of computing cloud computing was about 15 odd percent. And GNA was about 5%.

We figured we could take a sizable chunk out of sales and marketing, reduce it from 35 to around 30. Reduce our R&D from about 20 to maybe 17 or so, improve our gross margin by about 2-3 points, so you will very quickly come to about 10 percentage points of your overall P&L improvement.

And for VMware that would have translated roughly to about a billion dollars. So it was tremendous. And the line of sight to doing that did not involve - obviously we would have had to do some reduction of employment workforce, and so on and so forth - but beyond that it was not super disruptive. So it is very achievable. I don't know what the equation would be for companies that are in this audience. But some model like that you can pretty easily apply, especially if you're based on software.

Alan Cline

Yeah, I thought it was super interesting given how early you started on that. And then it just really just says Broadcom got a screaming deal.

Raghu Raguram

They got a screaming deal in many ways. By the way, I'm still an advisor to Hock, Hock Tan, CEO, he doesn't lead with technology so he waits for the technology approval. So he, this is not the 1st lever he's trying to pull. He's trying to pull other levers.

Alan Cline

And then I do, wanna hit the other two. And so if you talk about potentially your data center product, I think I'd love to hear a little bit about what you're thinking about the commercialization. How do you weave it in? That seems relatively straightforward. And then thoughts on pricing. I mean, I think this kind of gets to, you know, what a lot of people are wrestling with.

Raghu Raguram

Yeah, we had two things in the data center. One was, like I said, our infrastructure foundation. That was. we could apply conventional economic value add to that right. If you'd needed, for want of a better,100 GPUs to do something, now you could do either a lot more with those 100 GPUs, or potentially, we could let you do it with some 75 GPUs or whatever, right? So you can apply a a simple economic reduction out to that.

And, by the way, the other big value proposition for these regulated companies especially, because so much their data was On Prem still. They're moving it, but they're moving it so slowly. They didn't want to wait to move all of the data to the cloud first, put it into a data lake blah blah blah, and then do AI, they said, Hey, can we do some use cases on-prem, right? So a lot of the internal use cases where the users were employees, they get started on. So it was a very clear portion of value creation of which we said, We'll take a portion.

The trickier one was the equivalent of the co-pilot, right. Where the puts and takes are. Do you just use it to get greater stickiness on your existing products and get upsells, and so on and so forth. Or do you distinctly value this?

And we ended up using it primarily, except in one case, to treat it as an upsell on an existing product line.

And in our case the pricing was based on CPUs under management or data center infrastructure under management. So it was independent of the number of users that somebody else brought up in an earlier session.

But I've seen some companies struggle with that, too, because it dramatically cuts down, especially in customer success and all those domains that could be a pretty good big challenge.

I mean the other big thing that I would touch upon in our journey. It was all very early, by the way. In the earlier discussion people are saying, there is an LLM, and then there is application, etc., fundamentally there's a big temptation for all of us, especially those of us that have been in the industry for a while, to jump to a market-tecture or the stack view very, very quickly, right.

I don't know what the end answer is here, but I don't think that stack is the right way to look at it, because if you think about the most interesting applications that are consumer-facing today? They all have proprietary LLMs, right.

And when I say proprietary LLMs, it doesn't mean they've started with the whole foundation, right?

Take one of my favourite examples: Character AI. I don't know if anybody's tried it here. So this is the one that you can actually have conversations with the characters, historical characters. You can go have a chat with Julius Caesar or somebody right? Or you could create your own characters, etc.

They started with foundation models, but they've constructed something that's very unique to them.

Replit, which is the IDE tool, they constructed something that's very unique to them.

So I do believe that what is an application and what is a language model is not gonna be very distinct. And then, if you look at this age of multimodal models, where you're interacting with voice and video, and this and that and the other, conventional applications cannot handle that, right? I mean, our conventional applications are form based, or whatever.

So I do think UI will fundamentally change, applications will fundamentally change, and they will be more intrinsically tied to language models and language models themselves. These days you can have smaller models, right? There are techniques or distillation, and so on and so forth, where you now can execute inference on these models in fairly restricted compute environments.

So I think fundamentally, the application stack will change. And so my view, if I was in any one of your portfolio company's shoes is to not think of this in conventional terms.

The easy part is this quick co-pilot thing which you should do. But really, what you should be thinking about is, what is the overall value creation thesis that I have, and to the first speaker's point, what problems have people not been able to solve in your domain for the last 5 years, 10 years, that you can now take a fresh look at. Sorry, elongated answer on that.

Alan Cline

No, I think it's great. It relates back to a couple of things that we're we're mentioned earlier around: we're not really just talking about software market dollars. You're talking about the services, human capital dollars. And you know, depending on which market you're looking at, that's 5 to 20 times the size of the technology market. And you're solving a broader problem.

I'd love to come back to what you guys are doing on private cloud. I think this gets a lot into what Jae's talking about with Oracle, and what you know frankly, being a little late to the game allows you to see kind of where opportunities are unmet or things have evolved.

It seems like you guys are very focused on, hey, there's an emerging set of needs that are very different for the enterprise. Your private cloud approaches attacking that. I'd love to hear a little more about that.

Raghu Raguram

So just to clarify there we were. So when we were a standalone company, we were operating in what Jae called the multi-cloud. We were operating public and private. Since the Broadcom philosophy is to narrow it down purely to private. I'm still an advisor there and we are seeing a fairly tremendous amount of interest in this Private AI product. The reasons for that are 2 or 3 fold, and some of it was touched upon earlier conversations as well.

One like, I said, in these large regulated industries there is significant amount of data On Prem that people don't want to take over to to the public clouds as yet. Public language model providers has yet to be more specific.

Secondly, a lot of the high value applications are still internal employee facing, right. And thirdly, there is no way to formally verify correctness of a language model, right?

And that is the reason why human in the loop is the only thing that works today. Because if you get an answer, there's no way. I mean, it's software. All of us that have grown up writing software know that there is formal ways of verification and software. That is why the software testing industry exists, right. Whereas here there is no correctness at all. I mean, so no formal way to do software correctness.

So as a result of all of that, there is still a tremendous amount of interest in doing things internally, and the other misperception is that you need massive clusters right to do inference.

What's happening in the enterprise is mostly post-training, right, and inference. And post-training could be simple things like fine tuning. You know. this technology called rag, and so on. Or it could be the newer ones like distillation, and so on and so forth. But those are not super compute intensive.

And inference is not super compute intensive from a needing widely distributed clusters or wide diameter clusters. It could be done within narrow clusters, if you will, right. As a result, these companies are able to deploy infrastructure and the model the business model such that for them. Capex turns out to be a better ROI tool than Opex.

So if you saw some of the recent earnings announced from HPE and Dell, they're all talking about massive pipelines of hardware going into their enterprises, right? I mean, Dell, at point said they have 3 billion dollars worth of pipeline on the earnings call.

So that's what's happening there. So there is the mini regeneration of compute infrastructure going into going into enterprises. So that's one. And then, like I said, the huge knowledge bases that exist in enterprises are still On Prem. So that's why that's working.

Alan Cline

Thank you so much for the time.