The hidden role of analytics

How this hidden role causes disillusionment for data analysts and scientists

Welcome to the latest issue of the data patterns newsletter. If this is your first one, you’ll find all previous issues in the substack archive.

In this edition we’ll talk more about the disillusionment of data careers and the hidden role of analytics that has nothing to do with numbers.

Note: I wrote this initially as a LinkedIn post but decided to expand upon it here.

There is a hidden role in analytics that goes completely unnoticed but is the cause of much of the disillusionment amongst analysts and data scientists.

The most puzzling thing? It has nothing to do with numbers!

I was listening to an episode of Catalog & Cocktails with Bethany Lyons as guest and she mentioned something in passing that stopped me dead on my tracks. I scribbled some notes quickly before I forgot so I could write this post.

She mentioned that in many organizations analytics is used as a “glue” to fill in gaps for processes that require collaboration across multiple operational platforms. Some companies have 10,000+ dashboards many of which fill just this role.

I was immediately taken back to my days as a business analyst where a lot of the report requests I was getting had nothing to do with analysis. They were simply “data pulls” in Excel or dashboards that a manager would use to enact a cross-functional process.

These requests combined data from multiple systems. So my job was to pull data from these systems to a data warehouse, apply transformations, build a shared data model to integrated both systems, write a query to build the report and then send it over as a spreadsheet, or a dashboard.

I was working for a tour company where we produced and sold all-expenses-paid tour packages. There was an entire operations team in charge of building the tours (the components, the dates, inventory, etc.)

They had their own system while the sales team worked in Salesforce. There was some integration between these systems, but not enough to handle all situations.

What if you’re an operations manager who wanted to send a message about certain restrictions on a tour to the Vatican to sales? What would you do? That’s where emails and Excel spreadsheets come in. Or you ask analytics.

Analytics miscast?

That cross system collaboration gap ended up falling to the analytics team given that we had access to both systems and could do the integration. Should it be? Is that the purpose of the data team?

In my opinion analytics’s main purpose — the bulk of their work — should be to find ways to measure and improve the effectiveness of business processes. There are a few other applications that relate to generating insights and business strategy but they’re less frequent.

These operational reports however are not about measuring and improving processes but rather about enacting said processes. There’s no math involved, no statistics, just a pure operational report that combines data from multiple systems with some business logic underlying it.

It requires the same processes that analytics uses (extract data from multiple systems, model it, integrate concepts across these systems) so it gets assigned to them. But it’s something completely different.

Imagine being an analyst or data scientist and most of the requests you’re fielding require zero analysis. Often requests for dashboards, even if they involve numbers, are simply operational glue.

Obviously you’ll be frustrated. It seems like the work you do has no impact. You wanted to use math and statistics to improve the business. You wanted to generate insights that had a deep impact in the success of the business.

Instead you’re pulling data to enable collaboration across teams working in separate systems. Do this long enough and soon disillusionment will set in.

So what’s the solution?

Like in many such situations the first step is to recognize the problem. The core tension here is between process enactment vs process improvement. There’s a need for new roles. Many data analysts and scientists don’t want to do operational reporting so you’ll have to appoint someone else more willing to do the work.

Having a separate team that handles operational reporting with different objectives would go a long way towards alleviating the disillusionment. Beth actually suggests a data product manager who can act as a filter between operational reporting and actual data analysis and partition work accordingly.

But many teams can’t afford to hire for this role. Is there an opportunity for a technical solution here? A tool that allows a line-of-business manager to connect two disparate systems and build a report based on what they need?

Maybe. I don’t know yet.