How Analytics Drives Operational Excellence

The true purpose of analytics

Hello and welcome to the latest issue of Data Patterns. If this is your first one, you’ll find all previous issues in the archive.

Today we get to tackle one of my favorite topics: operational excellence.

I believe the true purpose of analytics is to drive operational excellence. Let’s unpack this statement so I can show you how. First let’s define what I mean by operational excellence.

Defining operational excellence

My friend Cedric Chin defines operational excellence as the “pursuit of knowledge.” By knowledge he means building cause and effect models for your business, so you learn exactly what happens when you move certain levers and can use them to drive the goals you want to achieve.

My own definition focuses on those exact goals.

If you think of any organization (nonprofits included) as a system whose goal is to make money or to serve its customers, then operational excellence is about fine tuning the system in such a way that it achieves goals consistently. To do that, you absolutely have to understand cause-effect relationships in your business.

And how does analytics drive this?

To answer this question we turn to my favorite diagram, the Goal Tree

This tree represents our destination. If you don’t know where you’re going, any road will get you there, said Lewis Carroll. So it’s important to clearly define what the ideal situation looks like.

At the top of the tree lives our overall goal:

• Analytics drives operational excellence.

One level below the goal live the critical success factors (CSFs) without which the goal will be impossible:

• Metrics are well defined, understood and agreed upon

• Metrics are accurate and consistent everywhere

• Metrics are used in key operational processes

Underneath each CSF live the necessary conditions (NCs) which enable the CSFs:

• For metrics to be well defined, understood and agreed upon

  • Metric definitions need to be agreed upon by everyone in the org. This way when the VP of marketing and VP of sales talk about a “customer” they both mean the same thing.

  • For metrics to be properly mapped to key business processes

    • Business processes need to be mapped to key business activities.

      • This means that key business activities are well defined. For example when a customer starts a subscription in a B2B SaaS business, that’s a key business activity that should be mapped to the process of how you onboard new customers

      • SOMA for B2B SaaS defines several hundred key business activities such as “customer starts subscription”

    • Business processes need to be properly instrumented so telemetry data is correctly captured and stored. That way we know exactly when a subscription began or when an email was sent.

• For metrics to be be accurate and consistent everywhere

  • Data quality needs to remain consistent and high. This is one the biggest problems in data these days but it’s not an insurmountable problem. The solution has to come from the front lines.

  • This means that there are policies in place for every data producing team to be responsible for consistent high quality data.

    • So what that means is that data quality is closely monitored on the front lines, not at the point it it consumed. This ensures problems are fixed in context instead of being traced from the data warehouse backwards.

    • Data issues are then easily communicated about as early as possible.

• For metrics to be used in key operational processes

  • Business reviews need to be regularly conducted (weekly or monthly) It’s not enough to conduct a quarterly financial performance review, you need to perform more frequent metric reviews.

    • In order to conduct these metric reviews you need enough of the right metrics, as my SOMA collaborator Abhi Sivasailam discussed in the dbt Roundup newsletter.

    • This means you have to have both input and output metrics defined.

  • Planning activities need to be easy to perform and effective. Well defined metrics enable:

    • effective forecasting which drives these planning activities

    • data science modeling

    • root cause analysis

    • traditional BI reporting and analysis

    • etc

So how do you get there? It’s really simple.

By adopting a metrics standard like SOMA, you get a set of well-worn, industry standard metrics you can use as is. Each metric is either a composite of a few base metrics or directly tied to a business activity.

How analytics unlocked operational excellence - An example

I recently I heard a great example of how analytics unlocked tremendous growth for an e-commerce company.

True Classic was started in 2019. By 2020 they had $9 million in revenue.

When Ryan (the CEO) hired Ben (now company president), they were 13 months into the business. They had already optimized their Facebook ads which were doing really well, but they were missing out on hundreds of millions more and they had no idea.

Within a week of starting, Ben discovered that the email opens rates were in the low 20%. Industry standards are about 40%+. It turns out they had massive deliverability issues with their emails being classified as spam. Now that’s a massive insight.

Notice that the insight has nothing to do with the number itself but rather how the number compares to industry standards. This highlights the importance of domain knowledge for analysts.

Why is email delivery that important?

Customer acquisition cost (CAC) is the biggest driver of profits (EBITDA) in a D2C business (in any business really) Email sending costs are almost $0.

By using clever segmentation and good copy, you can generate revenue without spending additional money on customer acquisition. Once they fixed their deliverability issues and started segmenting their list, it unlocked growth to the tune of $150 million in revenue by 2022.

I love this example because it also highlights the importance of constraints in your business. CAC is a massive constraint in D2C so if you can find ways to drive it down (e.g through email marketing or viral ads) you can generate large amount of profit.

I’ll write more about this topic in upcoming newsletters.

Until next time.