Reader Response to "AI Overinvestment"
A couple of days ago, I wrote a piece titled “What if there’s an overinvestment in AI?”
I received some thoughtful emails in response to that piece. One reader named Craig Melillo took the time to share a more detailed thoughts which he granted me the permission to share with all of you. I am copying his email below which I think adds to my aforementioned piece (slightly edited; emphasis mine):
“This is/was a great exercise. I am sure you are not alone in attempting this (I know that I am and have). I think the four incumbent hyperscale spenders (AMZN, META, GOOG, MSFT) are in somewhat of a “tails I win, heads you lose” situation given their AI spend to date is discretionary and it is not damaging the balance sheet. Some capacity will get utilized immediately, and some capacity will get utilized eventually. The attached revenues might not be what investors hope for, but it doesn’t kill them and IRR is likely >0%. In your scenario, NVDA would be in for a really tough time, which likely means in the short term, at least for stock prices, that everyone would be in for a tough time (especially ORCL, neo clouds, power supply chain). Open AI is its own case, and the $500B post-money from this past month plus a $20-25B exit rate annualized revenue will keep private shareholders trapped. Maybe the bubble popping solves inflation, enables rate cuts, and strengthens the dollar.
I am not exactly going out on a limb with this next statement, but I see three potential “events” that could act as rubber meeting the road event that pours some water on the fire:
Scaling laws asymptote and there’s a realization that digital God isn’t around the corner which removes the biggest totem that public markets and CEO’s both have in terms of giving permission to chase the dream. How does this manifest? More respected skeptics speak on it, and the frontier models level off.
I think Dwarkesh Patel made a really good point a few months ago when he said that the more time that goes by without achieving AGI, the less likely it becomes that we get there at all. If we assume that we do not get AGI within 3 years, then how much incremental value does a general purpose model that “just” provides longer context windows, memory, and faster response times actually provide if tomorrow’s small models are good enough (and what is the willingness to pay?). Training frontier zero-revenue models on chips that become obsolete for training purposes every 18 months is not sustainable. Ben Thompson made the point that early on in his career, he initially underappreciated society’s collective demand for better technology for the sake of better technology (maybe I am making that same mistake), but in this instance the cost of the incremental improvement is quite literally measured in the tens of billions of dollars (and you don’t know the return ahead of time, so it’s another leap of faith after the previous leap of faith did not deliver). Also, the predominant use case of today’s models are “free to me” inference, so the enterprise NEEDS to adopt this stuff, and if you listen to Accenture’s call yesterday, basically the large enterprise is nowhere near ready for it. Obviously GPT5 is much more accurate than GPT4 due to better reasoning, but if enterprise use cases are going to be grounded in domain specific data, then hallucinations should be lower in those cases lessening the demand for improvements in reasoning that are intended to reduce hallucinations. So how many bites at the apple will OAI get to train the next frontier model if they cannot live up to their own promises?
Any one of the cloud companies stops saying “demand exceeds supply” or implies that they have sufficient capacity to meet demand (implies AI demand decel, not growth decline)
If it takes 18-24 months to stand up an AI datacenter, then we really only have like 12-18 months of AI data center investment online since the hyper scalers actively committed to going pedal to the medal on investing in 2024. So, it makes sense that collective demand exceeds supply. But if collective supply goes up by 2-3x over the next 18-36 months, will there be a smooth 1:1 balance with demand? i.e. can AI tokens continue to growth exponentially for the current base? Maybe, but what is the willingness to spend in the face of excess supply? We don’t know, but if MSFT doesn’t want to build training infrastructure, then we know someone will say “we’re good”.
We see one/some of the negative margin AI model wrappers run out of funding and there’s reluctance for VC’s to step in (this is the least likely one given sovereign money will be there as long as the first two events don’t happen).
You point out that if you were in Zuck’s shoes then you would continue to invest too, and I agree that it is the right move. Everyone in the value chain is acting rationally and in their own best interest given everything they currently know: scaling laws have more or less held up keeping the dream for “digital God” alive, demand for tokens/AI inference capacity exceeds CURRENT supply, and all competitors are choosing NOT to cooperate. I have a short take on each, and figured I would share.
META – META is competing for consumer screen time directly with LLM’s as well as traditional competitors YouTube and TikTok. They need to make the product “AI native” to retain/grow mindshare. I don’t know if they will get a model that will act as “the draw” to the app for consumers, but they need a model that will drastically improve the recommendation engine, enable pro-grade multi-media content creation for users and advertisers, and then be good enough to act as an LLM destination for people that don’t want to leave the app. This is putting the augmented reality dream to the side for now, because I’m sure there is a lot synergistic spend for RL in here.
I think Meta will prove to be the most difficult company in terms of disaggregating “AI revenue” versus “non-AI revenue”, as they may already be generating AI revenue via ad creation tools, improved targeting,
GOOG – they are doing as well as you could hope with balancing monetization and disrupting oneself. YouTube will likely only increase in value and perhaps search has a much higher floor than the worst case fears. OpenAI may need to launch an ad supported model to keep the dollars flowing, and this could potentially steal from Google’s ad revenue (and ignite some ‘search is dead’ fears for a little). But Google Cloud, YT, and Gemini are crushing it.
MSFT/ORCL/OpenAI – the only one you could argue who is not acting rational is MSFT, unless they firmly believe we will not see the digital God dream (in which case, then they are acting rational and are optimizing for future inference demand that needs to be available when enterprise clients ultimately need it). MSFT is pretty much telling us that the incremental value of training frontier models will have diminishing returns. We want to applaud Oracle as “the winner” but they’re winning because MSFT is allowing it, and because OpenAI doesn’t have an existing business that prints $100B+ of operating cash flow per year to fund the spend. Maybe MSFT is wrong, and this will be the most obvious fumble ever, but they have skin in the game, but they were early to OpenAI so there’s a track record of seeing around the corner a bit with Satya. I don’t think they have an innovator’s dilemma even as/if OAI says they want to takeover the workplace application market. the G Suite is essentially free, and it MSFT has done quite well with its coexistence. Oracle has no other path to fast track itself into the hyperscaler conversation. Ellison is 81 and in it for glory, so he figures why not attach myself to a potential anchor customer in OAI (and now TikTok) that can maybe do $100B+ per year with me. I suspect there will be a lot of collateralized, non-recourse debt in their Stargate arrangement.
OAI/NVDA - OAI wants to be everything yesterday, but they only have one product (though it’s a general/world model), and they don’t have the organic cash flow to get there. So how do they solve it? They sell the dream to anyone and everyone who will listen to raise the money. The backers (NVDA/ORCL) need to adopt the same approach, and their leaders have no problem doing so. 18 months ago, Altman tried to say he would manufacture chips and TSMC laughed at him because they can’t afford to waste their time. Today, Altman has aligned himself with two owner operators in Huang and Ellison who have long histories of selling the dream. Since there is a hype market for it, it feels dumb for people to push back on it in the moment. NVDA also needs as many customers as possible to apply pressure to MSFT/AMZN/GOOG who are developing their own chips internally. To the extent NVDA can keep the market tight for GPU’s it means competitors are running fast and providing less leash for the ASIC customers to experiment with pushing internal chip efforts, and it forces the HPC’s to keep buying. This is also why Huang is so keen on keeping the chips flowing to China (and Singapore).”
I thought this was very well grounded discussion. The only people I disagree strongly with are people at the both extremes: “AGI is just around the corner” and “this is the most obvious bubble in the history of mankind”. Of course, there are gazillions of scenarios between these two extremes and navigating these scenarios remains the key challenge for the next few years.
Thank you, Craig for sharing your thoughts with me and allowing me to publish it on MBI Deep Dives.
In addition to “Daily Dose” (yes, DAILY) like this, MBI Deep Dives publishes one Deep Dive on a publicly listed company every month. You can find all the 63 Deep Dives here.
Disclaimer: All posts on “MBI Deep Dives” are for informational purposes only. This is NOT a recommendation to buy or sell securities discussed. Please do your own work before investing your money.
I’d be curious about Craig’s opinion on Amazon.
I believe they are playing the right long game, much like Microsoft, by betting on the inference market. That’s the smarter choice: all this training *must* ultimately lead to inference as the fourth building block of cloud, and the bigger market.
Betting on a cost advantage through vertical integration, combined with embedded switching costs and AWS’s wide user base, essentially de-risks both the investment and the demand side (especially if everything else collapses). And it makes sense: inference, like it or not, is here to stay. AGI, on the other hand, remains more of a dream. But I agree it’s one worth betting on, just in case (so Zuck is right).
Also, unlike Google (and, to a lesser extent, Microsoft), Amazon doesn’t really face disruption risk from AI. And, as Craig points out, that risk will eventually resurface for Google. On this point, I’d suggest reading Matthew Prince’s comments about the shift from search engines to answer engines. I think his view that AI chatbots provide a superior value proposition for 95% of users, 95% of the time is directionally correct (even if the numbers feel made up).
And historically, superior value eventually wins out. That’s why I believe Google Search’s decline is inevitable; the real question is when, and by how much.