It was written in 1924, by a physicist named Walter Shewhart, working at a telephone equipment factory in Chicago. His boss, George D. Edwards, later recalled:

“Dr. Shewhart prepared a little memorandum only about a page in length. About a third of that page was given over to a simple diagram which we would all recognize today as a schematic control chart. That diagram, and the short text which preceded and followed it, set forth all of the essential principles and considerations which are involved in what we know today as process quality control.”

Dr. Shewhart didn’t set out to revolutionize manufacturing, or give birth to Six-Sigma, TQM, Just In Time, Lean, etc. all of which still drive excellence and quality, but they call can be traced back to his work on the control chart.

He inspired the work of W. Edwards Deming and Joseph Juran, both of whom had tremendous success in post-war Japan and are credited with turning around the phrase “Made in Japan” from the laughingstock of industry to the second most successful economy in the world for a long time.

In many ways, Walter Shewhart can be thought of as one of the early pioneers of applied analytics, a field we consider quite modern but whose roots go all the way back to his invention of “Statistical Process Control” (SPC). Yes, statistics goes back even further, but its application in industrial processes began in a phone manufacturing plant at the start of the 20th century, with an American physicist and self-taught statistician.

Quiet beginnings

In 1918 Shewhart joined the Western Electric Company Inspection Engineering Department at the Hawthorne Works plant in Cicero Illinois to help with their quality issues. Western Electric produced telephone hardware for the Bell Telephone Company, which later became AT&T.

At Hawthorne Works, telephone systems roll off the assembly line. A worker inspects the apparatus testing it carefully. If it falls within tolerance limits, it’s a “go” otherwise it’s straight into the reject pile. We’re talking about a piece of hardware with more than one hundred thousand part here, getting scrapped just like that. This is not a joke!

It’s the early 19th century, and despite the concept of standardization being invented a century earlier, everything is still done by hand operated machines. Imagine making even something as simple as a screw on a lathe by hand.

Its length, diameter, number of threads, depth of the groove, etc. all standardized, and yet one small change in the process, like an extra rotation of a lathe, can cause a few tenths of a millimeter variation in length. Add up a few of these and the entire apparatus is thrown in the scrap pile.

It is precisely in this environment that Walter Shewhart gives birth to the Theory of Variation.

Controlling variation improves quality

The fact that manufacturing processes produce variation was common knowledge at this time. A machine cutting parts to a specified diameter will never cut every single part exactly the same. Temperature, tool wear, raw material variation, the vibration of the floor all of it introduces small differences. This is normal. This is expected.

Shewhart’s insight was that you can improve quality of a part by reducing that variation in its manufacturing process through the use of controllable inputs.

We don’t think about it much these days, but In order for parts to fit perfectly they need to be produced within certain tolerances. This is easy to measure: length, width, height, diameter, curvature, etc. and can be thought of as you “output measurements.”

Despite being thought of as “outputs” it doesn’t mean they can’t be controlled. You can control the diameter of a part by how long it spends in the lathe, how far the cutting tool is set, how many passes you take, how sharp the insert is, etc. By controlling these inputs, you can directly control a good portion of variation in final measurements, which means you can improve the part’s quality.

If everything is working as expected, the variation should be within an expected range (called upper process limit and lower process limit). If something unusual happens (a tool starts to get dull, there’s too much heat produced, or something breaks, you will see the change in variation. Now you can investigate.

Too many changes

Before Shewhart introduced the concept of “routine variation” vs “exceptional variation” managers would react to every fluctuation in outputs. Tighten a screw more, spin the lathe slower, use more coolant, take fewer passes, etc.

This is not much different to how we operate today.

We may not be tightening the screws of a work piece in a milling machine, but we do control how much we spend on an ad campaign, what kind of messaging we use in the ads, how quickly we reach out to customers who sign up for a trial, whether we email first then call, how often we call, etc. These are all controllable inputs for modern digital processes.

Shewhart discovered that if you keep adjusting a system that’s exhibiting only routine variation, you introduce variations of your own. You add noise on top of noise. He called it tampering, and his work showed that tampering with a stable process consistently makes it worse not better.

So he introduced a rule: “If all you see is routine variation, DON’T TOUCH IT!”

Unlike in manufacturing, where quality is visible — you can see if two parts fit together perfectly or not — we cannot see the quality in our digital processes. So we measure things we think make sense (e.g. conversion rates) and go about trying to improve them.

Exactly like the workers before the control chart, fiddling with the machines after every measurement, we tweak everything at once; we measure the impact poorly (by comparing today with yesterday or last week) and wonder why we can’t seem to figure out how to improve things.

But how do you measure routine vs exceptional variation?

Process control charts

Shewhart’s answer was the control chart. The idea is straightforward: calculate the natural limits of variation for a stable process, draw lines at those limits, and then observe where new measurements fall. Points within the limits? Common cause. The process is in control. Don’t touch it. Points outside the limits? Special cause. Something changed. Investigate.

Shewhart’s original hand-drawn control chart from the May 16, 1924 memo. The upper and lower limits define the boundary between normal variation and a signal worth investigating.

For those who haven’t seen such a chart in the past, the upper and lower limits capture Three Sigma (99.73% variation or roughly 2700 defects per million). For comparison, Six Sigma captures 99.99966% variation or 3.4 defects per million. Given the tooling at the time, this was more than adequate for improving quality.

In 1931, Shewhart published Economic Control of Quality of Manufactured Product — the foundational text. It was the first publication on statistical quality control in manufacturing. The core argument: you cannot improve what you do not understand, and you cannot understand a process without distinguishing between its normal behavior and its exceptions.

The problem Shewhart solved in 1924 is the same problem analytics teams face every day. A metric drops 8% week over week. Is it a real problem? Is it seasonality? Is it within the normal range of fluctuation? Should leadership be concerned? Should the data team be investigating? Should the engineering team be checking for a bug?

So why don’t we use the framework of process control charts?

If you don’t have a framework for distinguishing signal from noise, you end up doing exactly what those pre-Shewhart factory managers were doing: reacting to every fluctuation, investing enormous effort in problems that aren’t problems, and missing the genuine signals buried in the noise.

A Preview of what’s coming

Shewhart’s memo in 1924 was the beginning. His ideas stayed largely confined to Western Electric for over a decade until World War II changed everything, and the methods proved themselves at a scale nobody had imagined.

In part two, we’ll look at what happened when Shewhart’s framework met the urgency of wartime manufacturing and how it drove one of the most remarkable leaps in industrial productivity in recorded history.

I’ll write more on this series because it’s of genuinely high interest to me, and I hope to you as well. The history of SPC is really the history of how analytical thinking learned to create accountability at industrial scale. That story is not over. It’s just getting started.

Until next time.

I have built my entire (circa 2 decade) career in data on SQL. Many others even longer. This is not some exaggeration on my part. I only started learning Python for software engineering in the last 2 - 3 years, as surprising as that may sound.

You probably know me as the guy who loves SQL. I even wrote a book on it. I’ve gone viral several times SQL-maxxing, so have I changed my tune?

What’s happening with SQL

Let’s go through this logically and systematically so you fully understand the points I’m trying to make.

SQL is the best interface we have to query databases. It’s the lingua franca of any modern database. Many vendors who have tried to develop alternatives eventually give up and build a SQL engine.

But something very important happened in the last few months. The latest LLM model offerings from the big AI labs (Opus / Sonnet 4.6, GPT 5.2) have gotten very good at writing correct SQL (given the right context as we’ll soon discuss)

This both SQL used to query existing data and SQL used to transform and model new data to prepare it for querying. I have seen this myself in my own daily work, but if you want to read more about it check out this awesome article Jacob Matson wrote.

The shift we’re seeing is that the interface to the data is getting compacted.

Until now, a stakeholder question (in English or any other human language) had to go through you (the data professional who understood both the business context and the data model) before it got translated to SQL, executed on the database, results extracted, polished and then served; the LLM now steps in and handles all that (provided it has the proper context as we’ll see shortly).

You should NOT be discouraged about this.

Yes, a large part of your role was to answer questions like this, but if I know anything about data professionals, one of the most despised elements of the job was exactly that. Instead you should rejoice! The most annoying part of your job is about to be automated!

Why you should still learn SQL

If you read Jacob’s article (and you totally should) you saw that the latest models produced nearly 95% accurate SQL. Given the right context, the latest versions of LLM models can almost entirely replace your “natural-language-to-SQL” translation duties.

So does this mean you should no longer learn SQL? Nope! You totally should, and here’s why:

First of all, while the models can generate very accurate SQL, they still need validation. Known as “model evals” this is how you ensure that LLMs produce accurate SQL. After all, you wouldn’t trust the work of a junior analyst on your team without checking their query, would you?

Second, learning SQL, especially as it relates to your company’s business domain, will teach you far more about the business than you can learn through other means. Repeatedly answering questions using SQL will show you exactly how the data represents the business.

Third, in order for the models to produce such high accuracy SQL they need clean, well-modeled data, but this data isn’t going to clean itself. That’s where you come in. In order for you to know what to model, how to model, what to clean, etc. you need to inspect the data by writing (you guessed it) SQL.

And as the data changes, expands and schemas evolve, someone needs to make sure the agents still produce accurate SQL. This will become a key aspect of the future of data engineering.

While your role as the “human interface to the database” will start to diminish, your role as the data context engineer will start to become even more important. In fact you should want this to be the case and you should start tinkering with it RIGHT NOW. The longer you wait, the more you will be left behind. See my previous post on this.

Also, I don’t know about you, but I’d much rather spend my time cleaning, transforming and modeling data. That’s what brings me joy. I have zero patience to be the “human interface to the database.”

If you don’t believe me, go read this article from OpenAI on how they built their in-house analytics agent and pay attention to this section:

Fin.

That’s it for this issue. If you enjoyed it, hit the like button, leave a comment or hit reply to this email, I read and respond to every one.

Until next time.

I bet many of you follow each new model release with a tinge of anxiety, worrying if your job will be replaced by it soon. I felt the same way at first.

I started using coding agents extensively at work since the beginning of year and since then I’ve kept searching for a sensible way to make sense of what’s going on. Something devoid of hype. Something based on fundamental truths.

That’s the main reason I’ve been so quiet with this newsletter.

I wrote almost 3000 words on the idea that you can use logic to predict the future. Then I scrapped it. Why? Because I had a much better idea!

Alan Kay once said: “The best way to predict the future is to invent it” And so my main message in this post is one of hope:

When a new wave of innovation comes, there are those who sit on the sidelines fearing the worst and those who hope for the best.

Who wins? It’s always those who tinkered with the innovation, built something, learned from it and made it useful for themselves. Everyone else is brought along by the few brave souls who actually did something.

Even the language we use these days when talking about AI (prompt engineering, context engineering, vibe coding, agent orchestration, etc.) was literally invented by someone who dared to mess around with the technology in order to learn.

This post is literally 1 year old!!

How I almost took the wrong turn

If there’s anything you should know about me, I’m never early on new technologies. I’ve successfully avoided most of the fads of the last decade (crypto, blockchain, NFTs, etc).

And so for a long time I stayed on the sidelines when it came to AI. I decided I was skeptical and chose to ignore all the hype. That was until last month.

So what changed?

In January we decided to switch our software engineering to agentic, AI driven at work. Meaning whenever we’re developing a new feature, we first write the spec, develop a plan, develop the test cases for it, review the plan and then let an agent do all the coding.

Another thing you should know about me, I can be impatient. Code has always been too low level for me. I think that’s what drew me to SQL and data. I can operate at a slightly higher level of abstraction.

Hence why AI assisted software development has been a godsend for me.

With AI handling the coding, I can focus on what I do best: figuring out what the problem is, what needs to be built, gathering requirements and writing specs. This by the way coming from someone who used to hate writing specs; (read the code bro)

Worried that AI is a better analyst than you?

Figure out how to use AI to get accurate answers to most analytics questions quickly so you can focus on more interesting pursuits.

Worried that AI can build data pipelines in minutes?

That kind of work was low leverage anyways. Automate it with AI so you can get to the good parts.

Don’t let AI happen to you. Make AI work for you. Be agentic. Do things. Tinker with it, try things out, build something and then showcase it. Most AI projects are failing, but at least people are trying things.

Some things are bound to work. Some people are bound to reap the rewards of this new age. Why can’t it be you?

Until next time.

So the logical next question is, how? The answer is simple but not easy. Use patterns lifted from case studies. In this newsletter we’ll get into two of my favorite patterns and two case studies for each.

• Pattern 1: Lifting demand constraints

• Pattern 2: Plugging revenue leaks

Lifting Demand Constraints

As data professionals you’re probably familiar with things like funnel analysis. It’s an awesome way to find inefficiencies in sales and marketing processes. But you almost never think about demand.

You can often get 2x to 10x improvements in sales or marketing ROI by fixing demand vs doing funnel analysis. Yes funnel analysis can reveal leakage, as we’ll explore later, but before going there, look at demand constraints first Doing that can mean anything from improving the offer, the product, positioning, pricing, etc.

Here are two case studies where improving demand led to massive improvements in profitability. I’ve deliberately chosen non-SaaS examples to show that these patterns apply everywhere.

Case Study 1: Improving restaurant profitability

Restaurants are notorious for being low-margin, high-failure businesses. They’re capital intensive (the kitchen alone can cost $15,000 to $150,000) and have high fixed costs. Rents and operating costs are a constant drain.

And yet profitable restaurants do exist.

In an interview on the Tim Ferriss podcast Nick Kokonas, who is known for his partnership with chef Grant Achatz, owning several high-end restaurants and for founding the reservation business Tock, explains how he completely turned the restaurant model in its head.

Demand for most restaurants is their biggest constraint.

In the US you can walk into a restaurant at almost any time and get served the full menu. This doesn’t come cheap. The kitchen needs to be stocked and operational, which means you have to have someone on staff capable of cooking every dish. The waiters need be ready to serve and the bar is often open and tended.

Meanwhile demand is unpredictable and spiky. You’re either at the whim of customers walking in or them booking a table in advance that can be canceled at any time. Restaurants are not alone in this. Airlines and hotels for example face the unpredictable demand issue.

Unlike restaurants however, hotels and airlines deploy several techniques to handle such demand: they often have dynamic pricing, their reservation systems require upfront payment, a deposit that comes with a cancelation fee.

This is where Nick Kokonas’ ingenuity comes in. While Grant Achatz (who was prominently featured on Netflix’s Chef’s Table Vol. 2 Ep. 1) created amazing dishes and a once-in-a-lifetime experience, Nick decided to treat each table and time slot as bookable and opened up paid reservations far in advance.

If hotels and airlines do it, why not restaurants? You can even go further and implement “surge pricing” which for restaurants can be as simple as increasing prices on peak demand days.

With demand exploding after the Netflix documentary, people booked in droves. Having cash in the bank and demand locked in, Nick challenged another convention in the restaurant business. He paid for food in advance.

So instead of getting net 90, or net 60, which is what every restaurant tries to get, we actually negotiate a discount by prepaying that and then sort of truing up afterwards. So for our big-ticket items, we try to stay a couple months ahead on payments, in order to get more favorable terms and better product.

Case Study 2: Improving profitability at manufacturing plant

This next case study comes from a personal conversation I had with the CEO of a manufacturing plant. This is a small plant that produces a plastics product that retrofits existing infrastructure without needing costly replacements.

When he joined as interim CEO the plant was facing several problems. They had some demand but it was spiky and inconsistent. They were selling both directly to installers (B2C) and to distributors (B2B) and losing money monthly. Weekly sales tracking showed that growth was happening but only in big spikes.

Since we’re both Theory of Constraint fans, we had discussed potential constraints inside the plant. He’d mentioned that the manufacturing process itself was simple and well optimized so the biggest constraint was demand. His key decision was figuring out whether to focus on B2B or B2C.

Interestingly the spikes occurred when distributors bought large quantities. Using simple XMR charts, he decided that the best way to move forward was to focus on B2B and sell only to distributors. But there was a twist.

Rather than waiting for them to place orders, he signed distributors up with minimum monthly deliveries and recurring payments. After signing up the first distributor in the beginning of the year, revenue started to recover. 3 months later, another 4 distributors had onboarded and revenue doubled.

This single lever shifted the constraint from the market back to operations and the supply chain. With predictable revenue came predictable cash flow and he could now shift his focus back to the factory to expand production capacity and explore new products. He mentioned today they have tripled the product line!

Plugging Revenue Leaks

Next we look at two cases where plugging revenue leaks led to massive improvements. What’s cool about this pattern is that you can get wins quckly by just looking at key business processes and doing funnel analysis. You just need industry benchmarks.

Case Study 3: Plugging a massive email deliverability issue

The third case study comes courtesy of True Classic, an e-commerce company who sells t-shirts. They had become somewhat famous from viral TikTok and Instagram videos, so demand wasn’t really an issue. The issue was a leaky bucket.

In an interview for the Limited Supply podcast, Ryan Bartlett (co-founder and chairman) shared a story of finding a huge revenue leak courtesy of a new hire.

When Ryan hired Ben Diamond (current CEO), they were 13 months into the business. Within a week of starting, Ben discovered that the email opens rates were in the low 20%. Industry standards are about 40%+. It turned out they had massive deliverability issues with all their emails being sent straight to spam!

Talk about an insight, right?

Customer acquisition cost (CAC) is the biggest driver of profits in a D2C business (but really in any business) Email sending costs are nearly $0. By using clever segmentation and good copy, you can generate revenue without additional marketing spend. Re-ocurring purchases will help build a lasting business.

You can use all sorts of fancy growth tactics, viral videos, podcast appearances, direct advertising, etc. but if you have a leaky bucket, if you’re losing customers daily, you’re not going to have a business for very long.

Case Study 4: Plugging a huge CAC (customer acquisition cost) issue

Our final case comes courtesy of Alex Hormozi. In this video he’s helping a Ashley Capps (owner of AC Styles) get to $5 million in revenue. AC Styles is a professional wardrobe and personal styling service helping high-end clients improve their confidence by improving their style, wardrobe and fashion sense.

The business is otherwise very healthy (42% net margin) but this slide immediately caught my attention. 8 sales out of 1800 clicks is 0.045% conversion rate. Talk about a leaky bucket! Yes this is basic funnel analysis, a staple of marketing analytics these days, but this is still shocking to see.

The reason for such a low conversion rate was a poorly designed landing page. The page was just a google form where you put in your contact information and the company calls you to schedule a consult. That’s it! No lead magnet, no lead qualification and massive friction.

To fix the issue, Alex recommended designing a good lead magnet, reducing the friction and adding intermediate steps for signups. Now I’m not against friction in the signup flow. It can help reduce low quality leads dramatically, improve your CAC and profitability. But it requires a delicate balance. No friction is just as bad as having too much.

That’s it for this week.

Hope you enjoyed the case studies and they gave you some ideas about implementing something similar in your company. If you have questions or want to let me know you enjoyed this issue, leave a like, a comment or reply to this email.

I read all emails and love getting feedback. It often feels like shouting into the void.

Until next time.

Which brings us to this issue. There are a handful of attitudes (or mindsets if you like) that enable you to start building this skill while still on the job.

Tinkering without permission is by far my favorite.

I’ve used this exact attitude to:

• Learn for free while on the job

• Cultivate more luck in my career

• Look for opportunities to make an impact

I’ll show you exactly how to do all that in this newsletter.

What is “tinkering without permission?”

Tinkering is about being curious and trying things out. As you read books, articles, listen to podcasts, etc. certain ideas pique your interest and you’d want to explore them. The easiest way to explore is while on the job because you’ll be solving concrete problems not academic ones.

All you have to do is find a good reason to explore, ideally one that aligns with your company’s goals or can benefit them somehow. If you can do that, you’ve just given yourself license to pursue your curiosity.

Some of things you try won’t go anywhere, so it’s in your best interest to try things without asking for permission. It requires a little courage, but if you do it in your own time, or you can show how benefits the company, you’ll be fine.

Let me give you some examples.

Learn for free while on the job

I first encountered data science back in 2014. I was a newly minted “data analyst” trying to expand my expertise beyond data engineering and my company had just hired their first data scientist.

I got curious about what she did, so after talking with her for a while, I was peeved to learn I would need to take classes and get a degree in statistics if I wanted to call myself a “data scientist.” While I love learning, I had all but sworn off going back to school after my dismal experience with my MBA degree.

If I was going to learn data science quickly, I was going to do it on my own. I also realized that with the exception of a few select fields, companies don’t care much about your degrees, they care about what you can do for them.

So I found some books that taught the key concepts of data mining (what it was called back then) and statistics without getting too deep into the weeds. I also found YouTube tutorials that had applied examples.

I did all this in my own time. After learning enough theory, I looked for specific projects at work where I could apply it. I landed on the idea of building a model to score leads based on their propensity to convert.

The effort of data preparation, exploratory analysis, modeling and then explaining my work to my manager taught me far more about data science than any graduate program ever could.

Cultivate more luck in your career

In the summer of 2019 I entered the annual 2.5 day TripAdvisor hackathon with a project that had been brewing in the back of my mind for months. It was an application of NLP to extract tags from text descriptions. Amazon uses text tags to categorize and filter their vast number of reviews.

Hackathons are very short, intense, single or multi-day projects where people from various backgrounds collaborate to create new products or try out new things. They’re used in tech companies as a source of new ideas for innovation but they also happen to be excellent sources of serendipity.

That is exactly what happened to me after I presented my project in front of the three judges in the first round. Unexpectedly I made it to the second round, where I presented in front of a much bigger group of colleagues and judges. It turned out that the company was already considering a similar project.

I kept following this thread into other meetings, chats with the data science team (I wasn’t a data scientist) on how to approach it, and even a collaboration with a group of MIT MBAn students as a consultant on their project.

The whole thing was surreal as I never expected or planned for any of it to happen. Serendipity however is not that uncommon. It also led to my next role as a data product manager at a startup a few months later.

Look for opportunities to make an impact

One of the best things I’ve learned is to share your ideas with others even if they’re not fully fleshed out. Tell colleagues you meet randomly what you’re working on. Ask questions in meetings. Use sprint demos to showcase your projects.

You never know what might turn out

Richard Hamming, the Alan Turing award winning mathematician called it the “open door” phenomenon:

…working with one’s door closed lets you get more work done per year than if you had an open door, but I have observed repeatedly later those with the closed doors, while working just as hard as others, seem to work on slightly the wrong problems, while those who have let their door stay open get less work done but tend to work on the right problems! I cannot prove the cause and effect relationship, I only observed the correlation
Richard Hamming - You and Your Research

Having an open door leads to casual collisions, where people are free to stop by, ask you what you’re working on or ask for help on a project. These can dramatically alter the course of your career in beneficial ways. In my case my casual collision was the presentation I did in front of the judges and my peers.

The number of comments I got from that led me in a number of seemingly random but interesting paths which culminated with my involvement with the MIT students’ project.

These days I do a lot of virtual meetings. When I meet someone interesting, I schedule regular monthly meetings with them. I’ve met some of my best friends through this. We always talk about what we’re working on and share ideas.

Many of these ideas can be subtle pings that you have to recognize and follow up on that could turn into opportunities. Not all of them will pay off, but the ones who do could make up for it.

“The great scientists, when an opportunity opens up, get after it and they pursue it. They drop all other things. They get rid of other things and they get after an idea because they had already thought the thing through. Their minds are prepared; they see the opportunity and they go after it.”
Richard Hamming - You and Your Research

When you think of seemingly random occurrences as subtle pings it helps you to recognize their potential as an opportunity to pursue further, but noticing them in the first place requires, as Hamming puts it, an open mind. In fact, psychological research on luck puts a premium on openness.

Richard Wiseman in his book The Luck Factor found that people who consider themselves “lucky” tend to score high on extroversion (an orientation of one’s interests and energies toward the outer world of people and things rather than the inner world of subjective experience) and openness (to new experiences) and low on neuroticism (the tendency to experience negative emotions like anxiety, anger, guilt and depression).

That’s it for this issue. If you enjoyed it, let me know by liking, commenting and sharing it with your friends and colleagues.

Until next time.

More importantly I wanted to build expertise. So I looked everywhere. Career books, videos, articles, podcasts but only found bits and pieces.

The best book on the topic was Cal Newport’s So Good They Can’t Ignore You. It has a simple premise: If you focus on learning “rare and valuable skills” you’ll have a long and successful career, full of opportunities, satisfaction and compensation.

That’s great, but what skills fit those criteria? Cal doesn’t give you a satisfactory answer, but I believe I found it, at least for data professionals:

By the way, when I say “organization” I don’t necessarily mean the entire company. It could be that, of course but it could also just be an org inside of a big company.

In this issue I’ll try to achieve three things:

1. First, I’ll show you why you should focus on finding these leverage points

2. Second, I’ll give you ideas on how to learn them

3. Finally, I’ll provide a couple of examples so you understand what I’m talking about

What are points of leverage?

Organizations are complex systems. The parts are interconnected, full of dependencies and feedback loops. This means that improvements to the parts don’t necessarily add up to overall improvement.

Complexity also makes it so that organizations have points of leverage (also known as constraints) that drive most of the throughput.

Efforts to improve anything other than these points will at best result in incremental improvements and at worst waste time, effort and money with nothing to show for.

Many of these leverage points are inherent to the business model.

For example a subscription business model’s key leverage point is retention. Without it, all you have is a leaky bucket where all your marketing efforts will only delay the demise of the business.

Why focus on finding leverage points?

First of all understand that everyone from the C-suite down to every manager are already looking for them. That’s why you’re getting inundated with data requests. In fact, every request can be seen as an attempt to find leverage points:

“Can you break down conversion rates by channel for Q4?”

Translation: Are there any channels that are better than others where we can focus our efforts and win more deals?

“What’s our win rate for deals over $50k?”

Translation: Is there a segment of our customer base that spends more money, converts better and is easier or faster to close? That way we can focus our efforts and increase sales.

“How many users churned after the pricing change?”

Translation: Is pricing a lever? Is there a segment of our customer base who are price sensitive vs price insensitive? That way we can tailor our offerings accordingly and keep both or choose to focus on one vs the other.

The hope is that you (as a data professional) can find these leverage points in data. And as data professionals we sit in the perfect position to do so. How do I know? I’ve been there!

Being the business analyst gave me access to all the company’s data from sales, marketing, finance, operations, customer service, etc. The only problem is we waste this golden opportunity.

All you need is a little initiative. The key is asking the right questions, analyzing the business properly, coming up with hypotheses for where the leverage points lie, and tinkering with data.

I call this “tinkering without permission” and it’s one of the key attitudes you should turn into second nature. I’ll write more about that in another issue.

Second, this skill is highly valued. If all you did was focus on finding the next “million dollar” leverage point you’d be the most highly valued person at that company. Now tell me what CEO or president of an organization wouldn’t want to have a person like that on their team?

It also compounds over time, meaning the more you learn the more valuable you become, and can easily be transferred to other organizations. You can become an expert doing this, and expertise in data outside of technical skills is rare. That expertise is exactly what I was after.

How do you learn this skill?

Assuming you agree with me, we can now shift our attention to figuring out how to learn this skill. If I could write a letter to my younger self, I would tell him to do everything in his power to learn it. Yes, I’ve only just begun to think this way, but it’s already bearing fruit.

The only reliable way I’ve found to learn this skill myself is through case studies. This means reading books, listening to podcasts that discuss specific businesses, watching YouTube videos, hunting for patterns and collecting examples.

So in future issues I will pick out specific case studies I find in books, articles, podcasts, etc. to help you start to spot patterns and build your intuition as I continue to build mine. We’ll be on this journey together.

Let me illustrate with a couple of examples.

How PayPal boosted revenue by $100 million

In the book Growth Levels and How to Find Them Matt Lerner relays a story of how he found a leverage point, while launching a new product at PayPal, that ultimately resulted in $100 million of additional revenue:

“It was 2005. PayPal had decided to expand and bet the whole company on a new product, which allowed merchants to process transactions on their own sites via card processing APIs. Even back then, it was obvious the world needed this product. I was in charge of launching it. Thousands signed up, but few used it: the activation rate was just 10%.”

They had tried many different ways to fix it and the activation rate did increase to about 30% but that was still low. What Matt needed was a leverage point, so he decided to go to PayPal’s customer center and ask real customers why they had not activated.

He soon discovered that many of these merchants didn’t actually have any customers of their own, so logically they couldn’t activate. He had a hunch. What if he could find merchants who did have customers? How many of them could potentially be activated?

“I realized the problem: many of our customers had no customers of their own. And I couldn’t “growth hack” my way around that. That night, I asked my favorite analyst, Igor (Stanford engineering grad, of course): “How much of our revenue comes from our top 10% of signups?” An hour later, he sent me a spreadsheet with the answer: almost all of it.”

Now that’s what I call a leverage point! 90% of revenue coming from 10% of signups is one of those typical Pareto distribution things that are so typical in the business world. And while you should be careful about what you do with this information—because the answer isn’t always to focus on the 10%—it made sense in this case.

And that’s what Matt and his team did.

“We changed our entire approach. We developed a predictive model to score new signups on their propensity to become valuable customers. We’d try to increase revenue by giving high-scoring signups a white-glove onboarding experience, walking them through setup and teaching them to use the system.”

They tested this approach on a small group at first and when they saw promising results, expanded the treatment to the entire group.

“So we expanded the white-glove treatment to all our promising signups, increasing the revenue from new cohorts by 40% would eventually deliver over $100M in annual recurring revenue.”

One more example to whet your appetite.

Improving retention at HubSpot

Early on in HubSpot’s journey to a CRM behemoth, they were struggling to hold on to customers. There was a concern about whether they’d ever transition from an “inbound app” and into a full-blown CRM.

Mark Znutas was an operations analyst (later VP of GTM Ops and Strategy) looking for insights in the data. The approach he describes is essentially a form of feature engineering and classification.

I was an Operations Analyst at the time - one of dozens of analysts at HubSpot who struggled to stand out in the crowd…[]. Obviously, I knew I should spend my time thinking about improving net dollar retention, but I didn’t know where to start. How does one just fix NDR? Surely someone smarter than me would have already done it if the solution were straightforward, right?

What you can clearly see here is his quest to find and exploit leverage points. If you know anything about the SaaS business model, you’d know that retention (aka LTV) is the lifeblood of the business. NDR is just a proxy for measuring it.

Mark found his insight in a report he was already preparing that he called “The Customer File.” Anyone of you who has data science or machine learning experience will instantly recognize this format where each row represents a customer, each column is a feature and the response variable is whether they churned or not.

The basis of nearly all of our reporting was a large table called The Customer File. It contained Stripe billing data we pulled for all our customers, aggregated by month, and calculated the beginning-of-month ARR, end-of-month ARR, net ARR change, and change category for each customer.

But the secret didn’t lie here. ARR is a lagging indicator only showing the results of actions taken in the past. It doesn’t tell you what signals to pay attention to. So Mark expanded the customer file with firmographic and behavioral data to see if he could find any patterns. Like I said about this is basically feature engineering.

I cut retention by every dimension I could think of to see if I could find any insight into retention. Those dimensions included firmographic data, product usage data, lead source data, Sales & CS ownership data, and many others. I added new columns for every question I wanted to answer.

Eventually he found his answer in two distinct signals: business size and whether they were using the CRM platform HubSpot had just started building:

After producing dozens of these output tables, I finally found the insight I had been searching for. There were two variables with clear and strong correlation to retention:

1. Employee Size → Companies with 10 or fewer employees were rough

2. Whether the CRM customer also used HubSpot Marketing Hub → ”Platform” customers were great.

This is the kind of insight every executive expects to get from their data team! It fundamentally altered HubSpot’s customer acquisition strategy and starting turning their fortunes around.

Now, it was time to put our insight into action. We went all in on focusing on our best-fit customers. We started by focusing on acquiring more “Tier 1” customers (companies with 11+ employees or HubSpot Marketing Hub) and fewer “Tier 2” customers (companies with 1-10 employees and no HubSpot Marketing Hub).

And you know what, it worked! My guess is that it also helped Mark grow in his career from an operations analyst to VP of Ops and Strategy. You can read the full article here:

Wrap Text by Equals

An analyst's quest to improve retention for HubSpot CRM

A few weeks after launching The Ultimate Guide to ARR, I received a note from Mark Znutas, former VP of GTM Operations and Strategy at Hubspot…

Read more

2 years ago · 11 likes · 1 comment · Mark Znutas and Bobby Pinero

If you really liked this article, let me know. I’m planning to focus all my efforts to teach this one single skill along with all the attitudes surrounding it, as I believe it has the highest possible ROI of any technical skill.

Until next time.

I thought the break would be longer, but it turns out I was putting undue pressure on myself for my writing to be perfectly aligned with a singular idea.

What I enjoy instead is exploring ideas related to data engineering, analytics, decision making and business.

I’ve read a lot of books and articles on those topics, learned some useful things and would love to share them.

So you’ll get a slice of that in your inbox. I’ll keep them short and to the point, so you can read it in a few minutes and still get nuggets of insight.

Here we go!

Every organization has a goal. Once understood everything becomes clear. Data work is properly aligned and you’re no longer swimming in unanswered requests and overdue tickets.

More often than not, if you ask stakeholders what the goal is, you’ll get different answers:

• A marketing executive might say “ROAS” or “CAC.”

• A sales executive might say “Close Rate” or “Deal Flow”

• A VP of finance might say “ROI”

• The CEO might say something like “Growth”

It’s really easy to get caught up in these local-optima metrics and lose sight of the overall goal. This lack of clarity makes it difficult to see the impact of your work. Worse, it can get very discouraging.

Usually the goal of any for-profit company is to make money. All the other metrics, like the ones we saw above, support that goal and can be thought of as necessary conditions.

Nonprofit organizations also have goals. A hospital for example has a goal like health outcomes through patient care and research.

The goal should be expressed as a single metric and come from the top.

Until you get the executive board to agree on the goal, you’re doing yourself a disservice. In many cases the goal should come from them and be disseminated through the company.

Every request must have a clear path to the goal

As data professionals we help stakeholders make evidence-based decisions. These decisions are mainly about improving the organization’s system.

Improving a system means to make it achieve more of the goal, achieve the goal faster, better (e.g. higher quality), cheaper or simpler. Therefore whenever you get a request for data—whether it’s a pipeline, a metric or a report—pause and ask:

“How does this impact the goal?”

They should at least have a rough idea or a causal mental model. Here’s an example of a mental model for increasing ROAS (return on ad spend):

Move Lever => Increase ROAS => Reduce CAC => Increase CAC/LTV => Increase profitability

Seasoned readers of my newsletter will immediately see this as a branch of the company’s metrics tree.

I would highly recommend you do this for every request. Start with their request, figure out what outcomes they’re looking for, figure out what actions they are considering, then map out the casual chain.

Now you can suggest different actions or data research projects. This, by the way, is how you discover actionable insights. Doing data spelunking without a clear understanding of the goal and a rough idea of the causal model is a waste of time.

When you map out the entire mental model explicitly, you’ll not only help them get clarity, but also make your work seem more structured and make yourself seem smarter. People will come to you just to clarify their thoughts.

The simple diagram I showed is an example of a CDD (Causal Design Diagram) which is the key element of the field of Decision Intelligence. I’ll write up a more detailed post on CDDs later on but for now, the above should suffice.

Now your work suddenly feels strategic. It has real impact. Isn’t that what you got into data to begin with?

In a separate post Elena Verna (of Elena’s Growth Scoop) writes:

Growth is rooted in data. It’s about running activities that are measurable, trackable, and tied directly to user behavior and monetization outcomes. Marketing, on the other hand, includes plenty of things you _can’t_ (and shouldn’t try to) measure in the same way - brand, comms, PR, community.

It took me a while to figure out why it bothered me.

I don’t consider myself a “growth hacker” and I don’t know much about it, but when an executive is asking you for “actionable insights” aren’t they looking for just that? And if that’s the case, does this mean you now have to learn growth hacking skills?

That’s when I figured it out. My answer now? Not necessarily!

I’m not interested in learning growth hacks necessarily. My main interest is figuring out how to help organizations improve. And that doesn’t necessarily mean growth.

Improvement doesn’t always mean growth

The concept of growth hacking comes from the world of Venture Capital. In that world, exponential growth is necessary in order for a single portfolio company’s “exit event” (acquisition or going public) can make up for the entire portfolio regardless of the other company’s successes.

However there are many businesses for whom improvement might mean “become profitable” or “build an enduring company” or “maximize freedom for the founder.” All these companies can benefit tremendously from using data to improve without necessarily requiring growth hacks.

Take Prefect for example, a startup run by Jeremiah Lowin, (a fellow Postgres and SQL hacker) who recently decided to eschew growth in favor of building a sustainable startup.

He writes:

Becoming a profitable business frees us from that [capital] constraint. A startup can do extraordinary things when it doesn’t operate under the shadow of its next fundraise. Our decisions can now flow purely from what creates value for our customers, not from what extends our timeline.

Does this mean they no longer needs data? Absolutely not! In fact data can help them uncover opportunities for serving their existing customers even better, embedding themselves deeper in the customers businesses and building a more solid foundation.

And that’s why analytics needs to start from the top, from the goal of the organziation, the strategy for achieving that goal, and work downwards to figure our success factors, how to measure them and how to drive them.

Articles I’ve been reading

This week’s articles are about growth of which there’s too many to list, but these are some of my favorites

How Superhuman Built an Engine to Find Product Market Fit
A fantastic article on applying analytics to find PMF. My favorite part is the application of good ol’ fashioned segmentation to find and focus on the right audience for Superhuman.

As an early-stage team, you could just narrow the market with preconceived notions of who yo_u think_ the product is for, but that won’t teach you anything new. If you instead use the “very disappointed” group of survey respondents as a lens to narrow the market, the data can speak for itself — and you may even uncover different markets where your product resonates very strongly.

11 hard truths about working in growth
Elena Verna’s Growth Scoop is a fantastic resource for growth hacking ideas and applications.

Here is an example of how it usually looks (seen first hand at one of the companies I worked at): paid marketing would strut into Quarterly Business Reviews bragging about “crushing it” - lower CAC, more traffic, higher conversions. They’ve hit their goals! Everyone give each other high-fives! But wait… The overall business was struggling and (gasp!) shrinking. Even overall acquisition was down.

A Quantitative Approach to Product Market Fit]
Another fantastic, deep dive article on finding PMF for a SaaS business. I especially liked the explanation of accounting for SaaS.

The first technique we use at Tribe Capital when looking at a company is “growth accounting,” which breaks down overall growth in some activity across specific customer segments. The activity can be anything, though revenue and product engagement are the most common examples of growth accounting.

That’s it for now, until next time

And while the system design won, what this says to me is that I have to continue to straddle the line between analytics and engineering, so don’t fret. I will alternate posts between the two.

This week’s post is about data system design.

I’ve been designing data systems throughout my career. It started out building data warehouses on SQL Server using SSIS and stored procedures and continued onto cloud data platforms (BigQuery then Snowflake). Nowadays it’s mostly PostgreSQL (dbt) and some Python.

Here’s what I’ve learned:

You can’t know all the requirements upfront

My entire career, the one thing that has remained constant is how requirements keep changing as business needs evolve. This has made very apparent for me the need for a modular design.

I covered modularity in my book Minimum Viable SQL Patterns, but that was in the context of SQL queries. Modularity is actually a system design pattern. The key is simple, self-contained components that can be reasoned about, developed and tested independently.

A simple, evolvable architecture is crucial

The traditional way systems are designed is to gather all requirements and then spend time designing the perfect system. This is known as BDUF (big design upfront). I’ve often found that even if I know (or think I know) the requirements, I have a really hard time designing a large system upfront.

Some people have the ability to design the whole thing in their heads. That’s not me. In fact, I prefer to get started with a minimal design, where I outline the various elements of data transformation (simple boxes and arrows design) then I just start coding. This way I can evolve the system as I go.

Design the data model first

When starting to design a data system (like a data warehouse) one of the key elements that MUST be designed upfront is the data model. Like the rest of the system it needs to be easy to evolve and maintain, but once you’ve got something drawn up, it dictates the rest of the elements.

How you model data depends on what you want to do with it. If you want to analyze business processes, model events and activities using facts and dimensions. Kimball is great for that. Start with one-big-table and evolve as needed. If you want to analyze entities (e.g. customers, campaigns) model entities directly and roll up the activities and metrics to the entity level.

I plan to write a much more in-depth article on entity-centric modeling so stay tuned.

Articles I’ve been reading

Going forward, I’ll be including a section with relevant articles based on the theme of each post. This week you get some links on functional data engineering

FITT Data Architecture
Functional, Idempotent, Tested, Two-stage (FITT) data architecture has saved our sanity—no more 3 AM pipeline debugging sessions.

A wonderful article by Chris Bergh of DataKitchen explaining why a functional approach to data engineering can save you a lot of headaches in the long run. My favorite quote:

Yes, storage and compute have costs, but debugging at 2 AM is infinitely more expensive. The math is simple: data engineering time is worth more than compute costs, which are worth more than storage costs.

FITT vs. Fragile: SQL & Orchestration Techniques For FITT Data Architectures
SQL & Orchestration Techniques For Functional, Idempotent, Tested, Two-Stage Data Architectures

Another banger by Chris Bergh, and a follow up to the above article. What I love about this article is that Chris gets into the weeds of how to make SQL — a language decidedly designed for state mutations — be more functional.

Idempotence in SQL requires even more deliberate design. The naive approach of simply re-running transformation queries often fails because SQL operations like INSERT, UPDATE, and DELETE are inherently stateful.

Functional Data Engineering — a modern paradigm for batch data processing

The grandaddy of functional data engineering articles by Max Beauchemin which lays out the critical elements of why you should consider shifting to the functional paradigm and why it’s superior.

Functional programming brings clarity. When functions are “pure” — meaning they do not have side-effects — they can be written, tested, reasoned-about and debugged in isolation, without the need to understand external context or history of events surrounding its execution. As ETL pipelines grow in complexity, and as data teams grow in numbers, using methodologies that provide clarity isn’t a luxury, it’s a necessity.

That’s it for this issue, until next time

I thought I’d become obsolete.

I thought our entire industry would be handed over to AI and I’d have to switch careers.

That’s not exactly what happened but things have definitely changed.

This issue of Data Patterns is sponsored by Sheet Army Knife, a privacy-first Chrome extension that cleans, formats, and anonymizes PII data in spreadsheets before you accidentally load it to an AI tool. It runs entirely on your machine to reduce privacy risks when using AI tools.

Two years later, with AI hype at an all time high, one thing is clear to me.

Data and analytics engineering work is not going anywhere!

Instead, what’s happening is that data roles are becoming more multi-faceted. Data analysts are doing data science; data scientists are doing data engineering; data engineers are doing software engineering.

As a self-procliamed data generalist (I’ve done analytics, data engineering and even some data science) this was more than welcome, but what I didn’t expect was for me to get into software engineering this late in my career.

While my background is in computer science, I never wanted to get into software engineering, so when my manager asked me dive into Python I was a little concerned. I had never written a single line of Python in my life 😁!

Copilot to the rescue!

Once I realized I could use GenAI / LLMs to help me write the code, I noticed something very interesting. Not knowing how to write Python from scratch didn’t hinder me. As long as I could read AI-generated code and understand what it was doing I was fine!

The key skill for me shifted from writing code to writing very clear design docs; and to write very clear design docs, you need to be REALLY good at system design and architecture.

I soon realized data system design was a skill that wasn’t going anywhere. In fact, many of the solutions I’ve built even required embedding LLM capabilities into software.

Sure you can build apps with a single prompt nowadays, but are these apps easy to maintain, debug, update or are they a bunch of AI “slopware?”

Companies are learning that there’s no such thing as a free lunch.

Just when they thought they could replace software and data engineers with AI agents, they are waking to the reality that putting AI generated code in production (without human verification) is a nightmare waiting to happen; especially when that system needs to be well-designed to be maintainable.

In Summary

To summarize my argument in a few points, here’s why I believe data and analytics engineering work is not going anywhere:

1. In an “AI augmented” world data work is becoming multifaceted and encompassing more not less

2. With the advent of AI codegen tools and agents, system design and architecture skills are becoming more important

3. Vibe coded systems are requiring more human intervention and expertise, not less in order to not become a nightmare

Before you leave, I have a small favor to ask.

Could you let me know by voting below what kind of content you’d like to see more from me in the future?

That’s it for this issue.

Until next time

I love nerding out on data topics so the conversations were very enjoyable. One of the requests many of you had was on how to design interactive metric trees that cover the entire business.

That’s where Sergey comes in. We chatted last year while he was still building a metrics tree for the sales team and he was finally ready to reveal what they had built.

Sergey runs the internal analytics team at Brightly Software. The following is an edited version of a conversation we had about his experience designing, implementing and using metric trees at Brightly.

Agenda:

• Why did he choose to build a metrics tree?

• How specifically did he build it and how did he automate it?

• How is it being used and what’s the impact to the business?

• How he’s leveraging the tree to provide on-demand AI powered insights

Interested? Let’s get into it.

Why did they build a metric tree for sales?

Ergest: Hi Sergey. Thank you for taking the time to speak with me. Can you provide some context about Brightly? What does the company do?

Sergey: My pleasure! We're a B2B SaaS company selling a variety of cloud solutions to different verticals. Our largest market is probably education; we sell to schools and universities, and our focus is on asset management software. You know, when you need to submit a work order to get something fixed, that's our software.

Sergey: Before metric trees, we were very reactive. Questions would pile up and we’d scramble to answer them. We had dashboards and reports but it felt more like “choose your own adventure.” We wanted to be seen as a partner rather than somebody who just gets asked a question and provides an answer.

That’s when Sergey decided to build a metrics tree after reading my post and watching Abhi Sivasailam’s Data Council talk on metric trees.

How did they build the tree?

Ergest: I get a lot of questions about how to implement metric trees and people always want to capture the entire business in one tree. That’s not what you did right?

Sergey: Correct. We started with sales, not the full business, just sales and pipeline metrics. I think that's something I'd really recommend as an approach. With a business large enough to span different areas (sales, marketing, finance, operations) I don't know if you can or even want to capture all of it in a single metric tree.

Sergey: We knew for a fact that our sales stakeholders were really craving for that data. We had a lot of questions about how to do root cause analysis and how to figure out why are we up or down? At the same time, the North Star metric for sales (ARR bookings) is quite easy to build a metric tree for.

Ergest: What does the actual implementation look like? Can you walk me through what people see?

Sergey: What you see here just a screenshot, but the real tree is is automated and built in Qlik. You can interact with it, zoom in and out, and drill into specific branches. The colors represent how individual metrics are performing against whatever comparison you choose. It could be an average, last year same period, things like that.

Sergey: This way you can quickly see different areas where things are performing well versus not performing as well. The intensity of the color is where we get into statistical analysis and we do a simplified version of control / XMR charts. This lets us focus on specific areas that need deeper root cause analysis.

Ergest: How did you approach the actual design of the tree?

Sergey: We followed what I think is a pretty common blueprint. Start with a familiar business area, define our North Star (ARR in our case) then break that down into direct inputs, the next lower-level components. Then rinse and repeat recursively until you get to input metrics that you have control over.

Ergest: What about those influence metrics? The ones that aren't direct mathematical relationships? How do you discover and integrate those?

Sergey: That's an interesting area where we probably need to do more research. So far, we've been gathering feedback from conversations and field notes, talking to the reps and managers to understand what they see as causal relationships.

Sergey: For example, sales cycle length. It's commonly known that the shorter your sales cycle, the more likely you are to close the deal, so your win rate should be higher. But there's no direct mathematical relationship between those two metrics.

Ergest: How long did the full tree take to build?

Sergey: For the metric tree itself about three months with just me working on it. If there was a team working on it, it would probably be quicker. But I'd say a quarter of time just to get it to something useful. Then we iterated and improved from there. You don't even need to automate it right away. Start with a diagram. Build it on paper if you need to.

Ergest: Did you interview business users to validate the structure or did your team already understand the business well enough?

Sergey: We had a pretty good understanding since we work with stakeholders day in and day out, but we still interviewed them and checked with stakeholders because we really wanted to get this right.

How did the tree impact the business?

Ergest: Has this reduced the number of questions you get, or changed how you work with stakeholders?

Sergey: The main use case for us as analysts is that it's helped us bring more insights to the table instead of just answers, which really helps with reducing questions and addressing some of them before they even start. We've become much more proactive. Here’s an example of doing root cause analysis using the automated reports:

Sergey: What I'd love is for more users to actually start using it as self-service, not just using it but understanding the concept and getting value so they can answer their own questions. We still get plenty of questions, but we've shifted gears from just sitting and waiting for new questions to becoming proactive and digging into well-known questions every month and quarter to give insights before anyone asks.

Ergest: Any final thoughts on the challenges or what you've learned?

Sergey: The tool is powerful, but the more powerful it is, the more overwhelming it can be for users who aren't familiar with the concept. You want to find that balance between keeping it simple but also keeping it as powerful as it is.

Sergey: When I show this to typical business users and start talking about filters and driver analysis, I can lose them on the first sentence. So you have to be careful about how you introduce it. Don't worry about root cause analysis or forecasting yet. Those are separate use cases.

Extracting automated insights with AI

Sergey: One of the cool things about automating trees and metric definitions is leveraging AI to provide automated insights. What we’ve been able to do leveraging LLMs we can feed that data into an AI model and ask, hey, give me a summary of insights based on that metric tree data, what's driving what, what's the root causes of changes, and it actually does it. If people are interested we can do a deeper dive into that.

Ergest: Thank you so much Sergey. I appreciate you taking the time to explain your experience with metric trees. And I’m sure people would love to hear more about the AI section

Sergey: My pleasure. Maybe next time?

That’s it for this issue. If you have questions for Sergey, please send them to me via email and we’ll do another call.

Until next time

I’ve seen his play out multiple times in my career but it rarely works long-term for two very important reasons:

1. Yes you move quickly but all the tech debt and inadequate architecture design is carried forward into the new system. When you lift and shift you “copy / paste” the old system into a new environment. Eventually cracks will appear.

2. You miss out on the opportunity to redesign the data architecture properly. Do you know the expression “never let a good crisis go to waste?” It applies here. A redesign gives you a chance to clean house and rethink your approach.

Take the time to analyze the existing architecture, figure out what’s working and what needs to change, then come up with a plan where you can both update the design and deliver value incrementally.

My good friend calls this method “steel threads” whereby you take a horizontal slice of your existing architecture, from source system to end product (e.g dashboard) and deliver that in the new platform.

The beauty of this technique is that you not only deliver value incrementally and quickly, but you also validate your work by comparing key metrics in the two systems.

These steel threads make a lot of sense to the business than vertical slices which only make sense to engineers (build the ETL first, the data model next, the metrics next and finally all the dashboards)

Rethinking your data strategy is not for the faint of heart though. You need confidence, political capital and experience to convince stakeholders to give you a big budget with low initial ROI.

I know at least one person who is building a whole new data platform using open source technologies like Iceberg, Airbyte, etc. who pitched the full data strategy to their stakeholders and secured a nice budget. Usually this is a recipe for instant rejection.

The new platform they proposed will set the business up for near infinite data growth and easily swappable components without worrying about investing in proprietary technologies and vendor lock-in. Maybe I can get them to write something about it.

A strategy that builds for the future, especially one utilizing open source technologies will look like a bad investment early on. You’re spending additional capital on top of the current platform.

Hence why “steel threads” methodology is so powerful. You can compare the costs of delivering each dashboard with the new system vs the current one and show the potential for reducing costs or improving efficiency.

Thats it for this issue. I have at least a couple of in depth use cases coming up that show you how different companies are implementing and using metric trees. Both of them will have a visual demo.

Until then.

We looked at the key metrics / KPIs and built a metrics tree using the value chain. There’s a link to that post later one, but it’s not strictly required to understand this one. I’ll provide just enough context here for you to be able to follow along.

In this post we seek to answer the question: “How do we improve this business?”

Improving Systems Through Constraints

To answer that question we’ll look at another powerful framework for analyzing businesses called Theory of Constraints (TOC). I will not get too deep in the weeds of TOC because the knowledge base is vast, I’ll just focus on what you can use today.

TOC takes a wholistic systems approach to improvement. The core tenet is that every system has a goal. A constraint is a limiting factor that prevents the system from reaching that goal. It’s whatever resource you don’t have enough of to get the goal.

TOC also believes that this constraint is the only thing that matters and the only thing you should focus on to improve that system. This is quite a departure from the traditional approach which seeks to improve a business by optimizing every part independently.

The constraint acts as a point of leverage, in the sense that improving just the constraint by 10x gets you 10x improvements in the entire business. Improving anything other than the constraint will get you, at best, incremental improvements.

The five types of constraints

According to Dr Alan Barnard (who worked directly with and was mentored by TOC founder Dr Goldratt) there are only 5 types of constraints:

1. Demand constraint (also known as a market constraint) where you don’t have enough demand (enough market) for your products or services. Unless you have customer orders piling up or a queue of customers outside your establishment, you have a demand constraint. This is the most common type of constraint.

2. Capacity constraint where have enough demand but not enough capacity to handle the demand. This is usually an internal constraint based on your current operations, policies and business processes.

3. Supply constraint where have enough demand and capacity but not enough supply from a third party (hence an external operations constraint). This happens when you’re in the middle of a long supply chain between manufacturing and customers and are reliant on suppliers for raw materials.

4. Cash constraint where you’ve got plenty of demand, not enough capacity but in order to increase capacity you need money to invest. In this case the constraint isn’t capacity but cash.

5. Management attention where your business is doing well but you’re so busy handling daily operations and making every decision yourself that the business cannot grow regardless of demand, capacity or even cash.

As I said earlier, most businesses have a demand or market constraint, so it’s always a good idea to focus there first. That’s why it’s at the top of the list. If on the other hand you have a backlog of orders you haven’t fulfilled yet, or a long queue out the door, you have more than enough demand and you should focus on capacity/operations.

If however you don’t know what to focus on first, the best thing to do is to set an outrageous goal!

But why?

The traditional approach of goal setting in many organizations is to make sure they’re reasonable. For example “How do we grow 10% year over year?” But this type of goal will NOT expose your constraints and thus limit your upside.

When you set an impossible goal (for example how do we 10x this business?) the constraints become very clear. Let’s illustrate using Chiro-1st.

The Goal

In the video, Raymond (the owner of Chiro-1st) wants to achieve $5 million in EBITDA in 3 years and exit. This is definitely in the realm of outrageous goals, but we can go even further and ask “How do we 10x this business?”

TOC has a very simple financial equation to help you think about improvements.

There are only a few KPIs:

1. Throughput (Sales - Variable Costs)

2. Inventory or Investment (e.g. clinic location, equipment, etc)

3. Operating Expense (e.g. staff salaries, rent, etc.)

4. Net Profit = Throughput - Operating Expense

5. ROI = (Throughput - Operating Expense) / Investment

Let’s take a look at the numbers. Chiro-1st is making $5.2 million in top-line revenue and has a 23% net margin. If they wanted to 10x revenue, they need to 10x sales. Could they do it with the five centers they currently have? Each center has limited capacity in terms of a doctor, support staff, equipment and space.

We don’t know exactly but assume each center has one doctor working there. If we assume the duration of an appointment is 45-60 minutes, then a doctor can only see about 6-8 patients per day. We also don’t know whether the centers are operating at optimal capacity.

Decreasing the duration of an appointment can help a little bit, but that will also negatively impact patient health outcomes, which negatively impacts future sales. So the only way to 10x revenue is to open up other centers.

We know that will require upfront investment and potentially decrease the net margin, so we should try and squeeze as much performance as possible out of our current set up.

Ok this is a useful mental simulation, but let’s focus on Raymond’s original goal. Can we get to $5 million in EBITDA using our existing setup? What should we do first?

The business is already doing $5.1 million in revenue, so to get $5 million in EBITDA we’d need a 98% net margin (meaning our costs would need to be non existent) Since this is a physical business vs software business (whereby software has very high margins) we cannot get that level of profit without increasing revenue.

If we keep the 23% net margin the same, we’d need 4x the current revenue (or about $21.25 million) to get there. We could also try to tackle the net margin itself, which would involve reducing costs. Given the existing Investment and Operating Expenses are needed to run the business, even if we reduce some costs, it’s unlikely we can get 4x improvement.

So the only way forward is to increase Throughput (Sales) by 4x. Another way to do that is to increase prices. A 4x price increase is very likely to impact sales negatively but we can still experiment with some increases and get some improvement that way.

If we choose to keep prices the same, we’d need 4x more customers. Can our current capacity handle 4x more customers? We don’t know but let’s assume that we can. Let’s say we can hire another doctor and double the capacity.

The Power of Focus

At this point I hope you can see that by focusing on just improving sales first, more specifically the number of new customers, we can not only achieve our goal of $5 million in EBITDA but also exceed it. The beauty of TOC is that it lets you focus on a single constraint until you break it and only then start thinking about what’s next.

There are surprisingly few levers needed to increase sales. Hormozi focuses on quite a few of them in the video:

1. Ensure there’s a need for your services (which Chiro-1st already handles well)

2. Improve the offer to the market. This can be done for example by changing the ad copy and tracking improvements in the conversion rate of

3. Improve the sales process by reconfiguring how the offer is presented. Hormozi spends most of his time giving recommendations here.

4. Increase prices. This is something else Hormozi mentions given that the sales conversion rate is “too high” and could indicate more room for prices to grow.

If we look at the bottom of the metrics tree from the previous post, we see that by improving the ad messaging and increasing the spend, we can directly increase the Booked Consultations. And by reconfiguring the offer, and standardizing how the consultation is offered we can also impact the close rate which will improve the Booked Treatment metric.

Hormozi gives far more specific advice but for educational purposes, this is plenty.

Other Considerations

How can we apply this approach to a business without physical locations (e.g a SaaS product)? Well when you build the first version of the product, your primary focus should be demand, or as is known in the industry as Product Market Fit (PMF)

Imagine this SaaS company who has 10 paying customers, whose product can handle a max of 100 users simultaneously, but the focus of all their engineering resources is on scaling the product.

As long as your product can handle the initial demand, don’t even think about spending engineering resources on scaling. Focus all your efforts on building features that close deals and get you closer to PMF.

Only when you’re swimming in support requests from users who can’t acccess or use your product should you concern yourself with scaling.

Here’s the link to the previous post and a summary of the metrics below if you choose not to read it.

Metrics / KPI’s Overview

Next let’s go over the metrics / KPIs and see if we can’t get a better idea about this business.

Top line

• Top line revenue per location: $5.2 million

• Net profit per location: $1.2 million which means

• Net margin: 1.2/5.2 = 23%

Acquisition

• Monthly marketing spend: $1,500-$2,000 per month

• Total new leads avg: 35 per month

• Total shows for free consultation: 28 (80% show rate on avg)

• Total sales for full treatment: 20 (71% close rate on avg)

• Customer acquisition cost (CAC): $700 per year

Retention

• Lifetime Value (LTV) per customer: $3,400

• LTV/CAC ratio: 3400/700 = 4.8

ROAS depends on location (this comes into play later)

• Facebook Ads

• Google Ads

That’s it' for this issue. Until next time.

I did my best, but ultimately failed and decided to focus on data engineering instead.

Almost a decade later, I have discovered a number of analytical frameworks and tools that I know for a fact would have helped me succeed. I would have become an expert analyst and a trusted advisor to the executive leadership.

And so I’ve decided to write this post as a message to my younger self.

The most interesting of these tools and frameworks is the Metrics/KPI Tree. It's the best tool I know for understanding and communicating business context.

I’ve written about it in the past and since then, I’ve seen that people are very interested to learn more about it. What I want to do is write a series of posts (directed to my younger self) where I walk through the steps of building it from scratch.

I’ll be using a business case study I saw in a video by Alex Hormozi where he advises a business owner of chiropractor studios on reaching the goal of $5 million in EBITDA in 3 years. Of courtse I’m no Alex Hormozi, I don’t have his experience running and growing businesses, so this is just my take.

The link to Hormozi’s video is at the end of this post.

Business Overview

The case study concerns a business called Chiro 1st which comprises a number of chiropractor studios in and around Washington state.

These are physical locations where doctors perform chiropractic treatments for patients, such as various adjustments, rehabilitation and spinal decompression. The owner states that spinal decompression is their unique value proposition.

Since this is a physical business (as opposed to a digital one) we can immediately draw several conclusions:

• The business is capital heavy and there’s quite a bit of risk involved. In order to open a new location you need a fairly large, upfront investment to buy equipment (X-Ray machines, chiropractic equipment, exercise equipment, computers, etc)

• Operating expenses are also quite high and also necessary upfront (doctor’s salaries, staff salaries, location rent, utilities, etc)

• Just like any other physical business, the location of the facilities matters a lot. It needs to be accessible, convenient, accommodating, well designed, etc. As they say in real estate the key is location, location, location. This means you’ll also need to invest upfront in construction.

Business Model

The business model is quite simple.

Patient comes in for an initial assessment (free consultation, X-Rays, etc.) after which they are presented with a treatment plan consisting of several visits to the studio. Every time they get a treatment, their insurance is charged providing revenue for the business.

While not exactly a recurring revenue model, it’s the next best thing—reoccurring revenue—which means that the same person pays multiple times. This is a huge positive element Hormozi highlights during his analysis. The treatment plan is offered as a package of several visits as opposed to what he terms “one crack at the back at a time.”

Here’s a brief summary of this business:

• There are currently 6 chiropractor offices in WA offering services

• Only 4 of the 6 locations offer the special spinal decompression service

• The “front-end offer” is a free consultation and treatment plan

• The “back-end offer” is a treatment package that ranges from $2,400 to $3,600 over a 60 - 90 day period

• 1 in 7 patients choose a special $10,000 treatment in the case of an injury

• Revenue is about 70% from insurance and 30% payment upfront

Metrics / KPI’s Overview

Next let’s go over the metrics / KPIs and see if we can’t get a better idea about this business.

Top line

• Top line revenue per location: $5.2 million

• Net profit per location: $1.2 million which means

• Net margin: 1.2/5.2 = 23%

Acquisition

• Monthly marketing spend: $1,500-$2,000 per month

• Total new leads avg: 35 per month

• Total shows for free consultation: 28 (80% show rate)

• Total sales for full treatment: 20 (71% close rate)

• Customer acquisition cost (CAC): $700 per year

Retention

• Lifetime Value (LTV) per customer: $3,400

• LTV/CAC ratio: 3400/700 = 4.8

In a previous post I wrote that understanding demand is key to understanding the business context, so let’s do that next.

Understanding Demand

Demand is at the heart of any business. I mean that in the literal sense, without demand the business cannot survive. However demand finds itself ignored by many

Why do people go to a chiropractor?

The obvious answer is of course that they’re in pain (e.g. back pain). But having back pain doesn’t necessarily suggest you’ll choose a chiropractor. You might very well choose over-the-counter pain medication, some kind of therapy (like a hot/cold pack), physical exercise, stretching or even surgery.

So why then choose a chiropractor?

The context in which a chiropractor becomes a viable solution is one where perhaps medication has failed to relieve the pain permanently, simple therapies, stretches and exercise haven’t worked, your doctor is recommending surgery (which carries significant risks) and you’re thinking “there’s got to be another option.”

So you search online, you see an ad or a friend tells you about seeing a chiropractor.

You might have never been to a chiropractor before, you might have heard about them, watched a few videos on YouTube, you might even have anxiety about going there because the pain might get worse, or something else might go wrong. But in the face of surgery, you’re most likely thinking a chiropractor is much better than “going under the knife.”

Why is all of this important?

It’s important because in order to generate or capture demand you’ll need to buy ads and those ads need to mention what JTBD calls “the struggling moment” as well as handle all the fears and anxieties directly.

Their ad actually reads: “Avoid surgery, try spinal decompression”

Next you’ll most likely check out their landing page to make sure they’re reputable, read a few reviews and make a decision whether to book a free consultation or not.

Mapping the Value Chain

These are the first steps in what most people call the “customer journey” which I prefer to call the “value chain” because the customer must receive some value in every step in order for them to buy your solution.

Here’s the value chain of Chiro-1st as best I understand it. Let’s briefly walk through the steps in order to understand how the customer gets value in each step until they receive the ultimate value: pain relief.

The reason I like to call this “value chain” is because the customer receives some value at each and every step, until they receive the ultimate value: pain relief.

Customer sees the ad

It all begins when a potential customer has physical pain they can’t deal with themselves. This could be from an injury, over exertion or something chronic. They either see an ad or they actively search for ”chiropractor in my area”

Viewing the ad gives them hope that there’s relief available for their condition. Assuming the ad or the landing page handles their concerns and fears, they will click or call to book a free consultation.

Booking the free consultation

As you can see there are two paths from the ad impression. They could either click to book a free consultation online or click to call whereby a representative books the consultation for them. Once booked, they’re entered into a reminder system which calls, texts or emails them the due date.

Having a definite date in the future where their problem might be solved provides tremendous emotional relief and hope again creating some modicum of value. We know that about 80% of them show up for the free consultation. According to Hormozi, this is an excellent show rate.

Booking the paid treatment

Once they show up for the free consultation, the staff takes an X-ray, checks their insurance information and provides them with a treatment plan. This is when the offer is presented and the sale occurs. This business has about a 70% close rate.

Armed with a treatment plan the customer’s fears and concerns are reduced even more, while hope of relief increases. Another modicum of value.

Getting the treatment

Now that they’re booked, they need to show up for the treatment in order for the business to make money. Given that the treatment involves multiple appointments, they need to show up for each one.

Positive health outcome

While the customer needs to show up regularly to receive more treatment, having even a tiny bit of pain relief and the promise of a full resolution in the future provides tremendous value.

Assuming a positive health outcome, the customer is satisfied and satisfied customers come back in the future, write good reviews and are much more likely to recommend chiropractic treatment to their friends.

Bulding your first Metrics/KPI Tree

Now that you understand the value chain, all you have to do is put some metrics in each step and build your initial metrics tree. It might not be complete, but it’s far better than having nothing.

Next week I’ll give you another framework for improving your business context understanding and getting a more complete metrics tree.

Here’s my first tree for Chiro-1st. Feel free to make your own or use this and adapt it to your situation.

Here’s the video in case you’re interested.

Until next time.

This means a lot more in depth content on timeless patterns you all love:

• Data Modeling

• Data Architecture

• DaraOps

• MLOps

• etc.

There will be a brief hiatus while I cook up the next post, but in the meantime I would love it if you could reply back and let me know what you would like to learn.

Until next time.

In this issue I will explain a simple framework I have developed and refined over time to get 80% of the way in 20% of the time.

The best part of this framework is its simplicity but don’t let that fool you. There’s a lot more nuance behind it. And the more you use it the better you will get at understanding the business context and the easier it will become to deliver value with data.

Interested? Let’s get to it.

By the way, if you want to get even deeper into the business context, I suggest you learn about the competitive landscape of your company. The best tool for that is the Wardley Map which I’ve written about in a previous issue you can find in the archives.

The framework consists of three pillars:
1. The business model and its unit economics
2. The demand side (aka why are customers searching, buying and using the product)
3. The supply / operations side (aka how the company fulfills this demand)

Business model and unit economics
The business model is usually the first thing you learn. It's already public information so you can learn this even before you join a company.

For example the key metrics of a SaaS business can be learned in a single (amazing) article by David Skok called SaaS Metrics 2.0. VCs are more than happy to teach this stuff because they want more businesses like that to invest in.

The business model describes how the company captures the value it creates or said more simply “how they make money.” There will likely be a lot of nuance in any one particularly business, such as “usage based pricing (e.g. buy 1000 credits for $100]” or “user based pricing (e.g. $30/user/month)” both of which fit into the SaaS model.

The unit economics will play a big role in determining the viability of the business so that’s another area to understand deeply. I’m not going to dive into it here because it’s a vast topic, but suffice it to say that calculating CAC and the payback period are the key metrics that power a SaaS business model.

Demand
This is where you want to understand things from the customer's point of view. What are they looking for? Why are they searching for products like the one your company provides?

I'll give you an example that's not SaaS related just to broaden your horizons a bit.

When I worked for TripAdvisor one of the most popular things we offered were tours / experiences. These ranged from a "2-hour Italian cooking class" to an expansive "3 day wine tour and tasting experience in southern France."

What's the demand here? Obviously when you're traveling you want to experience and do as many interesting things as possible. TripAdvisor relies on local suppliers to provide these experiences because its business model is a marketplace. So any product features that helps customers find and book interesting experiences from trusted suppliers will obviously do well.

One of the key questions to understand in this phase is whether demand is generated or simply captured and amplified.

Generating demand often means creating a new category/niche in the market or repositioning a product into another market, which is really difficult because it requires educating the market about your company’s value. This explains why so many software companies are stuck in finding product market fit.

And that’s why many companies choose the easier path of capturing existing demand. So when someone searches for something on Google (e.g. "what to do in Italy") they are likely highly motivated to buy.

It’s often the job of marketing to figure out how to capture this demand as cheaply as possible and for that they need high quality data. A lot of the questions here revolve around which channel is best for providing highest quality leads at the lowest possible price. Hence the opportunity for you as a data professional to create value.

Supply / Operations
Upon clicking on a paid ad or organic search result leads come to our landing page. How they proceed from here is incredibly important. The goal here is to improve customer experience and help them onboard into the product as smoothly as possible.

Defining what a good customer experience looks like is another huge the opportunity for you as a data professional to make massive impact. Here you can start to draw up the so called customers journey (I prefer to call it “the value chain” but that’s just me)

Being able to trace a customer through the "value factory," figuring out what potential bottlenecks get in the way and how to make the process smoother (and faster) is a high ROI data activity.

Do you want to know what tool I prefer to use for this? That's right.....a metrics tree :D Obviously not a super detailed tree, but enough to provide some guidance on what metrics are missing, what reports are most useful and so on.

Did you find this issue helpful? Should I elaborate more on this topic?

Let me know in the comments or by replying.

Until next time.

There was a company I heard about that was making multiple tens of millions in ARR while all its reporting was still being done in Excel. People manually extracted data from various systems and dumped it in Excel.

They had hired a data scientist but their main problem was not data science. They needed to consolidate multiple scattered data sources into a common warehouse so that reporting could be automated.

It’s quite common for companies to hire “for the future” and get someone with a PhD to do data science while their most pressing problem lies in engineering. I’m sure you know what I’m talking about.

Anyways, the question in the back of my mind was “why didn’t they hire someone earlier?” Then came the shocking realization: “If they were growing regardless of data quality, does this mean that having standardized, well modeled data doesn’t contribute much to growth?”

As data professionals we debate a lot among ourselves about the value of data, but we never ask the crucial question of how stakeholders perceive that value. We act as if this value is self evident: “Of course you’d want clean, structured data!”

But as far as stakeholders are concerned, we have to dig further to figure it out. And the reason why we have to figure it out is that once we understand the driver, it will make our career growth a lot easier. So why was this company hiring the data scientist only now?

Two words: Financial Controls!

As the company was growing, one of the key concerns of the executive board became financial controls. If they wanted to raise another round of financing in the future they needed to get their financial house in order.

A bigger check from investors necessitates a higher level of diligence, which means a deeper dive into not just the company’s financials but also operational metrics. Every investor wants a “rocket ship” type growth for the companies they give money to, but at the same time they want to make sure that the growth is controlled so the company doesn’t spiral out of control.

This meant two very important drivers:

1. Stricter controls on all metrics to ensure any growth leaks get plugged

2. Finding stable growth drivers to ensure growth continues

This is what matters to a stakeholder! Important, sure, but not as flashy as “data driven decisions.” If you understand this, you can now become a trusted advisor to the executive board. All the reports, metrics, and recommendations you make can be focused around theme of “financial controls” to the delight of your stakeholders.

That’s it for this week’s post. If you enjoyed it leave a like or a comment.

Until next time

And to add insult to injury they don’t even know you could do this easily.

Time for a history lesson.

When I was a business analyst one of the things I was often asked to do was to sit in and shadow operations teams (marketing, sales, customer service, etc.) Every now and then I’d notice them doing things that would baffle me.

While on the phone with a customer, a salesperson would jump between several applications (CRM, email, Excel, etc) to answer customer questions. Other times they would be copy/pasting information from these systems into Excel to create reports that were used by the whole team.

Flabbergasted I would ask why they were doing this. To them this was just how things worked, they had no idea I could write a SQL query in 10 minutes and automate away hours of meaningless work. Productivity gains would be tremendous.

On the other hand, the requests I was getting for data and reports were nothing like what I was witnessing. I started calling this “suffering in silence.” They had no idea I could solve their problems with very little effort while I had no idea they needed this in the first place.

How to avoid the data service desk trap

I often joke that the moment you walk into a company as a data professional asking what their data needs are, you’ve fallen into the “data service desk trap.” You think you’re being helpful instead you’re setting yourself up for endless toil.

You see, business users are often unaware of what data and information is available, or could be easily obtained from existing infrastructure. They also don’t know what’s possible to do with data.

This means there's a HUGE opportunity for you as a data professional to work directly with the business, discover exactly what’s needed, what can be solved with data and what the highest ROI requests are.

But you have to be proactive, not reactive. It requires a very particular set of skills; skills that are often not taught in any course and not seen as super valuable.

Rare and valuable skills

In one of my previous newsletter I wrote about the data expertise triad.

One of the legs of the triad is about mapping business systems with data. This is one area that often gets neglected. You can take a course and learn technical skills or math and statistics skills but there’s no training for learning about the business.

When I was sitting and shadowing operations teams, I wasn’t looking to learn how they did things. I was looking for points of struggle, ideally struggle I could alleviate easily with a simple report or a dashboard.

I was doing what’s known as “problem finding.”

Dan Pink in his book "To Sell is Human” writes that the skill of problem finding is far more valuable than the skill of problem solving. That book was published back in 2012.

In today’s age of advanced LLMs and Artificial Intelligence, problem solving is increasingly being performed by machines thus decreasing in value. Meanwhile the skill of problem finding (ie. figuring out what to ask the LLM and how to prompt it) is suddenly becoming very valuable.

So you tell me.

Do you want to sit back, waiting for requests to land on your desk, or would you prefer to take initiative actively seeking out important problems, crafting thoughtful solutions, pitching your ideas, and take on the work to deliver meaningful results?

And what’s even better, this is EASY!

That’s it for this issue, how you enjoyed it. Until next time.

Subscribe now