Remember those early Infosys days when Narayan Murthy and Co. struggled to import computers? The same is happening all over again in the generative AI space, but with GPUs. 

Ask the folks in the AI space and they will tell you how much they love their GPUs. Earlier, if someone gave a 10 times bigger computer to a company, it wouldn’t know what to do with it. But now, each company just wants to have the biggest cluster possible and maybe later figure out what to do with it. Well, they would even begin to get more money based on the number of computers they have. 

Now instead of just building AI, companies have been using the “hoarded” H100 as currency, to get… guess what? More GPUs! CoreWeave, a cloud computing upstart supported by both NVIDIA and Magnetar Capital, has successfully obtained a substantial $2.3 billion debt arrangement by leveraging the value of NVIDIA’s coveted H100 GPUs. The primary objective behind this financial manoeuvre is to acquire additional premium chips and fund other strategic initiatives.

In simpler terms, NVIDIA’s investment in CoreWeave, a company heavily reliant on NVIDIA’s offerings, has led to the situation where the very NVIDIA products are being utilised as security to facilitate the purchase of more products that are likely to be from NVIDIA’s inventory. Quite remarkable, isn’t it?

CoreWeave was founded to mine Ethereum with the help of NVIDIA GPUs. Now, those same GPUs are being pivoted to work as the base for cloud infrastructure running AI.

Same is the case with AWS Lambda. CEO Stephen Balaban stated, “Lambda has a few thousand more H100s coming online before the end of this year — if you need 64 H100s or more, DM me.” Moreover, Elon Musk also has been hoarding 10,000 GPUs for a long time to build his AI company.

NVIDIA has been funding almost every AI company in the world. Arguably, apart from gaining profit from the AI of the company, it was so that the companies can buy the H100 GPUs from them. On the other side, the companies were happy with the amount of money they got so that they would be able to build massive AI products. Most of the companies wouldn’t put a hard stop on the amount of compute they needed — the more the merrier.

What has your GPU done for you today?

GPU shortage is no joke. According to Andrej Karpathy, “who gets how many GPUs” is the top gossip of the Silicon Valley right now. At the end, it is up to NVIDIA to decide how they would come up with a plan to address this shortage. 

Karpathy had earlier said the same thing, “Today FLOPS is one of the things you can spend $ on. Tomorrow $ is one of the things you can spend FLOPS on.” It is increasingly becoming true. Even Sam Altman posted a few hours later that all he thinks about all day is FLOPs. 

But on the other hand, this bid for debt with GPUs being a collateral is an interesting one. And it gets even more interesting when we think that this shortage is going to be just temporary. NVIDIA has already announced that it is going to release the GH200 GPU that would be almost thrice as powerful as the current H100s. The GH200s will be released to the market in the second quarter of 2024. 

Till then, hoarding of H100s becomes the moat for AI companies.

But just how beneficial that would be, is still a question. Eventually, the companies would want the newer GH200 GPUs and not the H100s. So the companies that have huge amounts of H100s would end up selling them to other companies. This would even mean that bigger companies like OpenAI and Google would be able to establish a monopoly in the market by owning the better GPUs by next year, and moving away from H100s.

But this won’t happen until next year. Even then, people would still be using H100s for building AI models, just like developers are using consumer grade GPUs to make AI models. 

So, it seems like GPUs are like the new dollar bills in the market right now. Since no one can buy GPUs anymore, tech companies are exchanging GPUs amongst each other to build AI products. This is what the GPUs are doing for the tech world — getting money even without using them.