Welcome to The TechCrunch Exchange, a weekly startups-and-markets newsletter. It’s inspired by the daily TechCrunch+ column where it gets its name. Want it in your inbox every Saturday? Sign up here.
Should early-stage founders ignore the never-ending debate on server infrastructure? Up to a point, yes: Investors we talked to are giving entrepreneurs their blessing not to give too much thought to cloud spend in their early days. But the rise of machine learning makes us suspect that answers might soon change. — Anna
Bare metal, rehashed
If you had a sense of déjà vu this week when David Heinemeier Hansson (DHH) announced that Basecamp’s and Hey’s parent company 37signals was leaving the cloud, you are not alone: The debate on the pros and cons of cloud infrastructure sometimes seems stuck on an infinite loop.
It is certainly not the first time that I heard 37signals’ core argument: That “renting computers is (mostly) a bad deal for medium-sized companies like ours with stable growth.”
In fact, both DHH’s rationale and its detractors strongly reminded me of the years-old discussion that expense management company Expensify ignited when it defended its choice to go bare metal — that is, to run its own servers.
However, it would be wrong to think that the parameters of the cloud versus on-premise debate have remained unchanged.
As Boldstart Ventures partner Shomik Ghosh noted in our cloud investor survey, there’s more to on-prem these days than running your own servers. Debate aside, I think most of us can agree that bare metal is not for everyone, which is why it’s interesting to see a middle ground emerge.
“In terms of terminology,” Ghosh said, “I think on-prem should also be called ‘modern on-prem,’ which Replicated coined, as it addresses not just bare metal self-managed servers but also virtual private clouds, etc.”
Ghosh has a horse in the race with the aforementioned startup Replicated, which “gives software vendors a container-based platform for quickly deploying cloud-native applications inside customers’ environments.” But he also has a point: Some startups deal with clients whose sensitive data makes them squeamish about public cloud environments.
With a nod to regulated sectors such as healthcare and financial services, Ghosh noted that “the scope of what is considered sensitive is growing over time,” adding that “it’s something more and more startups need to be aware of.”
Machine learning versus early stage
Machine learning and artificial startups are dealing with sensitive data more often than their peers in other sectors and have much higher computing needs. Because of this, I was curious to know if the investors we surveyed thought that they should go on-prem in their early days. Well, they mostly don’t think so.
“Going on-prem from a data center perspective — as opposed to cloud on-prem, i.e., virtual private cloud (VPC) — would require a very compelling business reason to justify,” Menlo Ventures partner Tim Tully told TechCrunch. (Note Tully’s caveat with regard to cloud on-prem — which overlaps with the concept of “modern on-prem” mentioned by Ghosh.)
To sum things up, for early-stage ML startups, the “early-stage” part weighs more than the “ML” part when it comes to cloud infrastructure decisions. This commonality also applies to cloud spend optimization, but according to Zetta Ventures managing director Jocelyn Goldfein, it could change.
“A number of us are speculating about the new generation of large language models like GPT-3 and DALL-E — and whether those might change the calculus at earlier stages,” she said.
Zooming out of the early stage, the rise of artificial intelligence is bringing changes to cloud infrastructure that we didn’t necessarily expect.
Commenting on his 2022 State of AI Report, co-author and investor Nathan Benaich highlighted a new trend: “While the largest supercomputers tended to be owned and operated by nation states, private companies are making real progress in building larger and larger clusters.”
The company formerly known as Facebook is at the forefront of this trend. “There’s a global competition to build the biggest, most powerful computers on the planet, and Meta (aka Facebook) is about to jump into the melee with the “AI Research SuperCluster,” or RSC,” TechCrunch’s Devin Coldewey wrote in January, noting that the supercomputer will be used “for the massive number crunching needed for language and computer vision modeling.”
Will the appetite for compute change what cloud means and how it looks? On second thought, we wouldn’t be surprised if it were the case.