Enterprise

Measuring and benchmarking the four vital signs of SaaS

Comment

Electrocardiogram (ECG) trace. An ECG measures the electrical activity of the heart.
Image Credits: Science Photo Library (opens in a new window) / Getty Images

Rory O’Driscoll

Contributor

Enterprise software investor Rory O’Driscoll led Scale Venture Partners’ investments in Box, DocuSign, ExactTarget, and Omniture, and currently invests in AI-powered business applications and automation technologies.

More posts from Rory O’Driscoll

SaaS metrics should be to a management team what patient vital signs are to an emergency room doctor: a simple set of universally understood numbers that allow a doctor to quickly know how ill a patient is and what needs fixing first.

Heart rate, blood pressure, respiratory rate and temperature are the big four vital signs in the ER. Everyone knows what they are, what they mean, and what good and bad looks like. When a patient is wheeled in, the doctor does not start by asking the EMT, “How exactly are we defining heart rate?” This shared understanding allows for rapid evaluation, then fast, focused action.

Not so much in SaaS, where discussions about definitions are all you hear. There are too many metrics, too many things to measure and too many useful but incompatible ways to measure them. This results in a loss of clarity, comprehension, and — most importantly — comparability across different companies.

At Scale we’ve spent the last 20 years evaluating investments in SaaS and other subscription companies. We have built an internal shared belief of what the Four Vital Signs of SaaS are, and how exactly to measure them. We have opted for simplicity over complexity in selecting these metrics. This has allowed us to benchmark accurately across companies and to know what a realistic version of “good” looks like.

Scale recently launched Scale Studio, an open-to-anyone tool that gives cloud and SaaS companies performance benchmarks based on these vital signs and 20 years of data across more than 300 companies.

The four vital signs of SaaS

The vital signs of SaaS are Revenue Growth, Sales Efficiency, Revenue Churn and Cash Burn. Almost everything that matters about the financial performance of a SaaS business is captured in these four metrics.

Revenue Growth matters because growth is the central purpose of a startup, and thus for an investor the most important driver of if value can be created at all. We have found that at each stage of a company’s development there is a minimum required level of growth below which a startup will struggle to attract venture capital. We’ve analyzed this Mendoza Line for SaaS growth previously on TechCrunch.

Sales Efficiency matters because software at scale is all about distribution, and thus the relationship between dollars invested in sales and marketing and dollars back via revenue is the key determinant of how much value is created per dollar invested. In a perfect world I would call this Distribution Efficiency, because calling it Sales Efficiency tends wrongly to narrow the focus on this metric to just sales, but that ship has sailed.

Revenue Churn matters because as growth slows the impact of churn escalates and provides an upper bound on how big a company can become. More fundamentally, high churn is just the financial evidence of a product that is not delivering value to customers. Products that do not deliver value cannot build value for their investors, which is why my former board colleague at Box, Mamoon Hamid, is right in saying that every company needs a non-financial north star.

And of course, Cash Burn matters because, well, duh — try running a company without it.

Measuring and benchmarking the four vital signs of SaaS

Vital Sign No. 1: Revenue Growth

There are multiple ways to measure revenue and thus Revenue Growth. ARR-based metrics are more forward-looking, but GAAP revenue tends to be calculated more accurately and is thus more comparable across companies. It typically lags ARR by about a quarter, and for simplicity we have not excluded services revenue. The simplest measure of Revenue Growth is the quarterly GAAP revenue run rate compared to the same quarter GAAP revenue one year ago (or a year from now for a forward-growth estimate). We also measure revenue growth using the ARR Growth Rate and a forward-looking measure of ARR growth that we call iCAGR.

We benchmark Revenue Growth by looking at companies at a comparable revenue run rate because revenue growth rates decline fairly predictably as absolute revenue scale increases (as is clear in the chart below). This means that the top quartile Revenue Growth rate of 123 percent at a $20 million revenue run rate would represent bottom quartile growth at a $2 million revenue run rate.

The chart and table below show, for various revenue run rates, which revenue growth rates represent top, median and bottom quartile performance. The Scale Studio data set has 300+ public and private SaaS companies, some of which have become public, many of which have not and most of which have raised at least some venture capital.

Using this table, a team can quickly benchmark their company’s performance. For example, a company that grew last year 80 percent from $11 million to $20 million is growing at just above 50th percentile growth for SaaS companies at that stage, which is 78 percent. We can also generate a separate table showing the same data on a forward-revenue basis, to allow a company to answer the related question: If I am at a $20 million run rate now, and I grow next year at 50 percent, how will I be doing relative to other SaaS companies?

It is also interesting to see that the data here broadly agrees with a separate calculation we did recently around the Mendoza Line for SaaS growth that tracks at or just above the bottom quartile of growth rate. The Mendoza Line was derived by math; this table was generated from real data — it is good to see both estimates roughly agree.

Another way to look at the same data is to think about the revenue trajectory over time, or “how many years to $100 million.”  The graph and table below show for a consistent top, median or bottom quartile company (at the cutoff points) how long it takes to grow from $1 million to $100 million. A company that grows consistently, just at the top quartile cutoff growth rate, takes six years to get to a $100 million run rate, a median performer takes eight years and a bottom quartile performer does not yet get there in 10 years. This is a calculation with all sorts of survivorship bias problems, because the slow growth companies tend to get acquired and not make it all the way to $100 million, but the analysis is roughly right.

This data also matches well to various rules of thumb. An example is the T2D3 (triple-triple-double-double-double) rule, which is also shown in the table above. T2D3 matches top quartile performance for the first four years and becomes just a little aspirational in year five. If you fail to double from year four to year five and only grow at 90 percent, we would still be glad to talk to you! (The data also roughly matches the Bessemer State of the Cloud Report, which shows an estimate of times to $100 million for best-in-class companies).

Vital Sign No. 2: Sales Efficiency

Sales Efficiency metrics (and again the reminder to think of this more broadly as Distribution Efficiency!) measure the relationship between dollars in (spent on Sales & Marketing) and dollars out (in the form of new revenue). For a recurring revenue business, by far the most intuitive way to measure this concept is by dividing the Gross or Net New ARR for the quarter by the fully loaded Sales & Marketing spend for the same quarter. The Gross SE metric measures the effectiveness of the company in generating new ARR, and the Net SE metric measures the overall effectiveness of the business in both generating and retaining revenue.

We love the simplicity of this calculation and its direct actionability. It can be explained in 30 seconds at a Sales Kick-Off meeting in a way that no other measure can. “We gave you this money, you gave us this ARR.” We are not a fan of putting lags in this method (comparing this quarter’s Net New ARR with last quarter’s S&M spend). There is some logic to the idea that there is lag between spend and results, but once you start to adjust, you end up with all sorts of special pleading. Keep it simple.

For a vital signs diagnosis, we prefer this metric to the complexity of the LTV/CAC calculation. An LTV to CAC works really well for consumer businesses and for B2B businesses that have fairly consistent deal size and low net churn. A former Scale portfolio company, HubSpot, did a brilliant job of orienting their business around this metric. However, for enterprise businesses with highly variable deal sizes and strong positive net cohort growth over time, the calculation becomes arbitrary. The underlying idea is real, namely that enterprise customers can have lower Gross SE but higher ultimate value as the cohorts grow, but trying to track and explain quarterly fluctuations is hard.

Another complexity we choose to avoid is using Gross Margin instead of Revenue. It is of course more correct to use Gross Margin, but especially at the early stages of a SaaS company, Gross Margin fluctuates based on fixed cost recovery issues that significantly distort the calculation.

The problem with an ARR-based Sales Efficiency metric is it doesn’t allow easy comparison across companies. ARR is not reported by public companies and private company ARR numbers are often suspect. Our workaround was to slightly tweak the calculation for Net SE, replacing the numerator (Net New ARR) with the intra-quarter difference in GAAP revenue multiplied by 4 (annualized). We call this formula using GAAP instead of ARR the Magic Number and it should be equal to Net Sales Efficiency with a one quarter lag (to allow ARR to convert to GAAP).

As a rule of thumb: When you’re talking to your team and want to keep it simple, talk Gross and Net Sales Efficiency; when you want to do benchmarking against other companies, use Magic Number.

The benchmarking results here are very different. Unlike Revenue Growth, which clearly declines as absolute revenue increases, we have found Sales Efficiency to be fairly consistent across the entire SaaS universe. The median Magic Number for our data set is between .8x and .7x, and the range from top quartile to bottom quartile is between .5x and 1.5x. This matches the public company data set where the median is .7x today.

Payback (on a revenue not a gross margin basis) is simply the inverse of this number, which implies that the average SaaS company is earning back in revenue what it spends in sales and marketing in one divided by .7 years — 17 months — with a top/bottom quartile range of eight months to two years.

We have also observed something that does not come in this table, which is that Sales Efficiency tends to be persistent over time for a given company, especially after $10 million. A good go-to-market model at a $10 million run rate tends to still be a good model at $100 million. And bad sales efficiency at $10 million is hard to change later.

The high top-quartile Magic Number for the $1 million revenue run rate represents an anomaly early on, as often founders are doing the selling themselves (and probably not allocating their costs to Sales & Marketing!). Pretty quickly top quartile Magic Number falls to 1.4x and then to 1.0x at scale.

Vital Sign No. 3: Revenue Churn

The simplest way to measure Gross and Net Churn is by taking Churned ARR (Gross) and Churned less Upsell ARR (Net) and dividing it by opening ARR for the period, usually a quarter. In the tables below, we show Gross Churn by quarter and annualized.

We acknowledge that this metric is a horrible oversimplification. For the Sales Efficiency calculation above, the simple method is also, we believe, the best method, but for the churn calculation this simplification comes at a significant cost in terms of being able to diagnosis underlying issues. At high growth rates especially, this measure understates actual churn. However, the vital signs framework calls for simplicity to allow consistent, relevant benchmarking across companies. If this simple benchmarking exercise exposes a churn problem, then a deeper dive using retention analysis and a cohort analysis is an absolutely required next step.

The data above shows Darwinian selection at work. Early on some companies have huge churn but they have to either improve or die. At a $20 million revenue run rate, even bottom quartile companies have annualized Gross Churn hovering around -22 percent.

Vital Sign No. 4: Cash Burn / Operating Income

Internally, we measure cash burn by looking at free cash flow for a quarter (operating cash flow less capex) and compare it to cash on the balance sheet to calculate a cash out date. For confidentiality reasons, we do not ask for cash balances in Scale Studio. A reasonable proxy for cash burn is Operating Income, and the chart and table below show Operating Income as a percent of Revenue at different revenue run rates.

To illustrate what this metric means, at a $20 million revenue run rate the median company in the data set is losing 63 percent of revenue, $12.7 million dollars, or colloquially “burning $1 million a month.” In Scale Studio you can also further benchmark total operating expenses one level down, across each of Gross Margin, Sales/Marketing, R&D and G&A.

The most important point to make about this metric is that in a recurring revenue business, operating income, or “burn,” however calculated is not a measure of efficiency. Instead it is a measure of how aggressively a company is investing. A high operating loss coupled with a high growth rate and high sales efficiency is an aggressive but probably sensible strategy provided of course the company has access to capital. A high burn, low growth company is a disaster in the making.

Exactly how much burn for exactly how much growth will be the subject for another post, but any comment that tries to link burn rate to value creation, without taking growth rate into account, is simply wrong. Many of the most successful SaaS companies were in the bottom quartile on this metric at $100 million in run rate revenue, but were also in the top quartile on revenue growth. Worth highlighting again is the proviso regarding access to capital. If the cash runs, out even the best business dies.

The very best companies are those such as Veeva and Atlassian where a high Sales Efficiency allowed them to simultaneously be top quartile on growth, and top quartile on operating income profitability. It is no accident that companies with these characteristics get premium public valuations.

What are your company’s vital signs?

Getting started with Scale Studio is simple: you enter nine basic data points for each of your trailing eight quarters then generate a benchmark report. The benchmarks use a sample set consisting of companies at the same revenue stage as yours. This allows for much more accurate benchmarking, especially for Revenue Growth and Operating Income which, as we have said, are a direct function of revenue run rate. The benchmarks give you a sense of your performance that is clear, concise, and comparable. Your report might say something like:

“At your current revenue run rate of $5 million, your Y/Y Revenue Growth rate of 150 percent is in the second quartile for companies of your size, your Magic Number of 0.8 is in the second quartile, your Gross Churn of -1 percent is in the top quartile, and your Operating Income of -152 percent of revenue is in the second quartile.”

Vital signs don’t cure patients, doctors do. SaaS vital signs don’t fix companies, management teams do. But realistic benchmarking metrics do what ER vital signs do: pinpoint issues, provide actionable context and allow you to get to work.

Jeremy Kaufmann contributed to this article.

More TechCrunch

Welcome back to TechCrunch’s Week in Review. This week had two major events from OpenAI and Google. OpenAI’s spring update event saw the reveal of its new model, GPT-4o, which…

OpenAI and Google lay out their competing AI visions

Expedia says Rathi Murthy and Sreenivas Rachamadugu, respectively its CTO and senior vice president of core services product & engineering, are no longer employed at the travel booking company. In…

Expedia says two execs dismissed after ‘violation of company policy’

When Jeffrey Wang posted to X asking if anyone wanted to go in on an order of fancy-but-affordable office nap pods, he didn’t expect the post to go viral.

With AI startups booming, nap pods and Silicon Valley hustle culture are back

OpenAI’s Superalignment team, responsible for developing ways to govern and steer “superintelligent” AI systems, was promised 20% of the company’s compute resources, according to a person from that team. But…

OpenAI created a team to control ‘superintelligent’ AI — then let it wither, source says

A new crop of early-stage startups — along with some recent VC investments — illustrates a niche emerging in the autonomous vehicle technology sector. Unlike the companies bringing robotaxis to…

VCs and the military are fueling self-driving startups that don’t need roads

When the founders of Sagetap, Sahil Khanna and Kevin Hughes, started working at early-stage enterprise software startups, they were surprised to find that the companies they worked at were trying…

Deal Dive: Sagetap looks to bring enterprise software sales into the 21st century

Keeping up with an industry as fast-moving as AI is a tall order. So until an AI can do it for you, here’s a handy roundup of recent stories in the world…

This Week in AI: OpenAI moves away from safety

After Apple loosened its App Store guidelines to permit game emulators, the retro game emulator Delta — an app 10 years in the making — hit the top of the…

Adobe comes after indie game emulator Delta for copying its logo

Meta is once again taking on its competitors by developing a feature that borrows concepts from others — in this case, BeReal and Snapchat. The company is developing a feature…

Meta’s latest experiment borrows from BeReal’s and Snapchat’s core ideas

Welcome to Startups Weekly! We’ve been drowning in AI news this week, with Google’s I/O setting the pace. And Elon Musk rages against the machine.

Startups Weekly: It’s the dawning of the age of AI — plus,  Musk is raging against the machine

IndieBio’s Bay Area incubator is about to debut its 15th cohort of biotech startups. We took special note of a few, which were making some major, bordering on ludicrous, claims…

IndieBio’s SF incubator lineup is making some wild biotech promises

YouTube TV has announced that its multiview feature for watching four streams at once is now available on Android phones and tablets. The Android launch comes two months after YouTube…

YouTube TV’s ‘multiview’ feature is now available on Android phones and tablets

Featured Article

Two Santa Cruz students uncover security bug that could let millions do their laundry for free

CSC ServiceWorks provides laundry machines to thousands of residential homes and universities, but the company ignored requests to fix a security bug.

2 days ago
Two Santa Cruz students uncover security bug that could let millions do their laundry for free

TechCrunch Disrupt 2024 is just around the corner, and the buzz is palpable. But what if we told you there’s a chance for you to not just attend, but also…

Harness the TechCrunch Effect: Host a Side Event at Disrupt 2024

Decks are all about telling a compelling story and Goodcarbon does a good job on that front. But there’s important information missing too.

Pitch Deck Teardown: Goodcarbon’s $5.5M seed deck

Slack is making it difficult for its customers if they want the company to stop using its data for model training.

Slack under attack over sneaky AI training policy

A Texas-based company that provides health insurance and benefit plans disclosed a data breach affecting almost 2.5 million people, some of whom had their Social Security number stolen. WebTPA said…

Healthcare company WebTPA discloses breach affecting 2.5 million people

Featured Article

Microsoft dodges UK antitrust scrutiny over its Mistral AI stake

Microsoft won’t be facing antitrust scrutiny in the U.K. over its recent investment into French AI startup Mistral AI.

2 days ago
Microsoft dodges UK antitrust scrutiny over its Mistral AI stake

Ember has partnered with HSBC in the U.K. so that the bank’s business customers can access Ember’s services from their online accounts.

Embedded finance is still trendy as accounting automation startup Ember partners with HSBC UK

Kudos uses AI to figure out consumer spending habits so it can then provide more personalized financial advice, like maximizing rewards and utilizing credit effectively.

Kudos lands $10M for an AI smart wallet that picks the best credit card for purchases

The EU’s warning comes after Microsoft failed to respond to a legally binding request for information that focused on its generative AI tools.

EU warns Microsoft it could be fined billions over missing GenAI risk info

The prospects for troubled banking-as-a-service startup Synapse have gone from bad to worse this week after a United States Trustee filed an emergency motion on Wednesday.  The trustee is asking…

A US Trustee wants troubled fintech Synapse to be liquidated via Chapter 7 bankruptcy, cites ‘gross mismanagement’

U.K.-based Seraphim Space is spinning up its 13th accelerator program, with nine participating companies working on a range of tech from propulsion to in-space manufacturing and space situational awareness. The…

Seraphim’s latest space accelerator welcomes nine companies

OpenAI has reached a deal with Reddit to use the social news site’s data for training AI models. In a blog post on OpenAI’s press relations site, the company said…

OpenAI inks deal to train AI on Reddit data

X users will now be able to discover posts from new Communities that are trending directly from an Explore tab within the section.

X pushes more users to Communities

For Mark Zuckerberg’s 40th birthday, his wife got him a photoshoot. Zuckerberg gives the camera a sly smile as he sits amid a carefully crafted re-creation of his childhood bedroom.…

Mark Zuckerberg’s makeover: Midlife crisis or carefully crafted rebrand?

Strava announced a slew of features, including AI to weed out leaderboard cheats, a new ‘family’ subscription plan, dark mode and more.

Strava taps AI to weed out leaderboard cheats, unveils ‘family’ plan, dark mode and more

We all fall down sometimes. Astronauts are no exception. You need to be in peak physical condition for space travel, but bulky space suits and lower gravity levels can be…

Astronauts fall over. Robotic limbs can help them back up.

Microsoft will launch its custom Cobalt 100 chips to customers as a public preview at its Build conference next week, TechCrunch has learned. In an analyst briefing ahead of Build,…

Microsoft’s custom Cobalt chips will come to Azure next week

What a wild week for transportation news! It was a smorgasbord of news that seemed to touch every sector and theme in transportation.

Tesla keeps cutting jobs and the feds probe Waymo