The reason: Most companies lack a clear strategy for how to extract value from their data.

Claims that “data is the new oil” or “data is the new gold” were everywhere over the last decade, and they made it sound like all you had to do to participate in this gold rush was to track anything and everything and pipe all of it into a data warehouse.

But this perspective is missing an important element: In contrast to oil or gold, data is not inherently valuable in and of itself. The value of data depends entirely on what you do with it (or others can do with it). And even if companies are leveraging their data, they’re often thinking too small. Creating internal dashboards and reports is only scratching the surface.

Curating and surfacing data in the right context and format to customers and users can massively increase the usefulness of your core product, improve engagement and retention, differentiate you from the competition, or even create upsell opportunities and open up incremental revenue streams.

But you can’t just dump raw data on your target audience; you have to be thoughtful to make sure it clearly communicates insights relevant to that particular user segment. After all, the real value of data is what people can do with it.

Static dashboards, even if they’re well-designed, often aren’t enough to achieve this because they put the mental load of interpretation on the user. AI can solve this, but comes with its own challenges: How can you be sure it gives accurate answers and recommendations every time, no matter what users want to know?

In this article, we’ll cover all of this in detail:

• First, we’ll assess the data your business generates and discuss its potential value

• Then, we’ll cover how you can identify opportunities for impactful data products

• Next, we’ll walk through how to actually build your data product

• Finally, we’ll talk about what you can do to make your data product successful

We’ve got a lot to cover, so let’s dive in!

How to identify valuable data in your company

Before you can think about extracting value from your data by turning it into a valuable product, you first need to assess what you have at your disposal.

Most businesses create a lot of data as a by-product of operating their core product or business. What that looks like varies from company to company, but here are a few common categories that might apply to your situation:

User preference data; for example:

• Engagement (e.g. clicks & likes) on a social media platform

• Searches, bookmarks & alerts on a business travel booking portal

Transaction data; for example:

• Supply & demand by time and place on a ridesharing or food delivery platform

• Sales, pricing information etc. on a second-hand marketplace

Most of this data has the potential to be valuable, but not all of it is (at least not to the same degree).

Below, we’ll go through some of the key dimensions to consider as you’re trying to find the gold nuggets in your data mine, in descending order of importance:

Criterion #1: Is your data useful?

This is the most important factor by far. If your data isn’t useful, nothing else matters. But how do you define “useful” in this context?

Data is useful for an audience if it helps them make decisions or solve problems.

There are two important considerations embedded in the definition above:

1. Whether data is useful, and how useful it is, depends on the audience. E.g. what might be extremely helpful for one segment of your users might be irrelevant for others.

→ This means you have to identify potential target audiences among your customers, users, and even prospects.

2. Data should ideally address an existing question or problem. The reaction shouldn’t be “Interesting – let’s see what we can do with this” but rather “This makes my life so much easier!”

→ This means you need to understand each target audience’s jobs-to-be-done. What pain point can your data address in an actionable way?

We’ll dive much deeper into both of these considerations in the next section on evaluating the opportunity for a data product.

Criterion #2: Is your data proprietary and unique?

Ideally, you have exclusive control over the data in question and there is no other place to get it. Or rather, there is no other place people can get the same insights that your data provides.

For example:

• If the data in question is easy to scrape from your website, it’s not really proprietary. If it’s valuable, it’ll quickly appear in other places and users are free to choose where to consume it.

• Similarly, even if your specific dataset cannot be accessed elsewhere, it might not be unique. For example, if your product is an HR platform (HRIS), you’ll likely track basic data points like the number of new hires. That’s generally valuable — but chances are, your customers can also get similar (or better) hiring data from their Applicant Tracking System (ATS).

So ask yourself: What’s the type of data that only you have, and what are the kinds of insights that it can enable for your customers and users?

However, there is one caveat with regards to the above; even if data is not unique by itself, you can sometimes make it unique (and useful) by combining it with other data in ways only you are able to.

Sticking with the HR data example above:

While hiring data by itself might not be unique, the HRIS is likely the only place where it can be combined with the overall org chart, performance data, attrition stats etc. This holistic view provides a better picture of organizational health and gives the user a much better understanding of where they might need to take action.

For example, some teams might be hiring fast and seem like they’re growing successfully if you’re only tracking one metric. But, if they’re also seeing a disproportionately high share of performance management issues and early terminations, then you have indicators of fundamental flaws in the hiring process.

Finding these opportunities can turn several separate, commoditized datasets into the foundation for a data product that can become a differentiator for your business.

Criterion #3: Is your data dynamic?

The most valuable data continually grows and changes. Static snapshots can quickly lose their usefulness as users get relevant insights from them and move on. Dynamically evolving data, on the other hand, constantly generates new, potentially valuable signals that can draw users back in and help them take action.

For example:

Let’s say your company provides a platform that lets people organize and manage events. You should definitely show historical stats like how many events someone has organized, or how many people attended. Without this data, your product would feel incomplete.

But it’s not going to be a big reason for users to regularly log in, as those numbers don’t change much. On the other hand, you may have access to additional data that is much more dynamic and valuable. For example, you can help users understand how signups are trending for their upcoming event, how that compares to past events at the same stage, how likely they are to sell out, and what the attendee breakdown is to help finalize any content and messaging.

This is relevant information that users will want to keep a close eye on (hello DAU 👋), and you can even link it to other features you might offer (e.g. paid event promotions, partner sponsorships). And it’s exactly the kind of data that AI can turn into a concise, actionable briefing with clear recommendations, but I’ll share more on that later.

How to evaluate an opportunity for a data product

Now that you have a framework for thinking about the potential value of your data, it’s time to get specific. To get started, you’ll need to pick an audience and decide what you want to build.

At a high level, you have two options:

1. Data products focused on attracting new customers or users: Standalone, data-powered tools or resources that attract traffic & convert it to new customers or users

• Location: Typically on dedicated landing pages on the company website

• Example: A mortgage rate calculator on a mortgage fintech website

2. Data products for existing customers & users: Embedded analytics for existing users inside your product that improve the value of your core offering (either offered for free to improve engagement and retention, or packaged as a premium add-on)

• Location: Within the core product

• Example: In-app dashboards and reports in a fitness app that help you optimize your workouts

Note: Data products can also be a hybrid; they can target both existing as well as prospective customers and achieve multiple goals at once.

For example:

Spotify’s famous year-end “Wrapped” report is generated in-app for existing users to remind them of how much they used (and enjoyed) the app. But at the same time, it’s designed to be shared on social media and draw in new users.

The above tells us who we can build for; but it doesn’t tell us what to build yet.

This part might seem overwhelming at first glance, but the solution is somewhat simple: A data product is — surprise! — a product. And as with all product development, it helps to start by understanding your target audience and their needs, and then work backwards from there.

That’s what we’ll cover next.

Use case #1: Designing an external data product for potential new customers / users

The goal of a data product for prospective customers or users is to 1) be useful so it attracts potential users (often through Search Engine Optimization (SEO) or Answer Engine Optimization (AEO)) and 2) draw them into your main offering (e.g. by demonstrating its value, or by presenting your core product as the obvious next step).

Ask yourself: What pain points do prospective customers or users have that your product can help solve?

Common Google or LLM search queries can give you inspiration here. In addition to showing you what problems users are facing, they also give you a clear path to organic distribution. After all, people can’t find your product useful if they can’t find it at all.

In addition to solving a real problem and ideally addressing common search queries, an effective data product for prospects also:

• Is easy to use: Requiring complex user inputs or skills creates friction, which will cause people to abandon it. This usually means simplifying the product and focusing on a particular use case

• Creates delight: If users are genuinely excited about the data product, it has the potential to go viral

• Converts: Ideally, the logical next step after using the data product is to use the core product

Let’s look at a few best-in-class examples of what this can look like in real life:

Example deep-dive: Zillow Zestimate

Zillow is a real estate website, and its “Zestimate” tool lets you view an estimated market price for any real estate property in the US.

Why does this work?

1. Traffic acquisition: The tool is available on the website — no login required. And it directly answers common search questions like “How much is my home worth?”, generating plenty of free traffic from Google (and potentially LLMs in the future).

2. Ease of use: All you have to do is enter an address and click a button, and voilà — the tool pulls up an estimate from the database.

3. Conversion: Depending on the status and location of the house, you’ll be presented with one of these four Calls-to-Action (CTAs), asking you to either 1) list your home, 2) take out a loan, 3) request a cash offer for your house, 4) or pay for a featured listing. Each CTA directly converts you into a user of one of Zillow’s core product lines.

Example deep-dive: Going.com

Going.com is a flight booking website that lets you set alerts for routes you’re interested in and then notifies you about relevant offers and cheap flights. On its website, it features data products like the one below that tap into the data of its core product.

This works for the same reasons as Zillow’s Zestimate above; 1) this data-backed overview is the direct answer to common search queries like “best times to fly to mexico”, 2) it doesn’t require any complex inputs or configuration, and 3) it converts visitors to registered users as the next logical step is to sign up for a Going account and set price alerts.

Use case #2: Designing an embedded data product for existing customers / users

The purpose of a data product for prospects was clear: Get more customers or users.

Embedded analytics within your core product can help with that, too: Most customers expect extensive analytics features, so building them will make it much easier to close deals (talk to your Sales team and dive into the notes of lost deals in your CRM if you want to validate this).

But the value goes far beyond this. Regardless of what your product does, users will inevitably reach a point where they need to dig into the data. Without built-in analytics, they’ll export data into Excel and/or use third-party tools that have the capabilities you’re missing. Either way, they’ll spend less time in your product. And over time, missing features and workarounds start to impact retention.

If you’re “lucky”, customers will let you know about their frustrations and, instead of churning, request data from your Customer Success or Support teams or submit requests for custom features to Product. However, fulfilling these requests only acts as a band-aid for individual customers and can lead to a massive drain on your resources.

Intentionally designed in-product analytics, on the other hand, give you a more scalable solution; instead of doing ad-hoc data pulls or creating custom reports, you invest resources once and get something that adds value for all customers and gives them a reason to spend more time in your product.

In-app analytics can also be a powerful way to remind users of the value they’re getting from your product; how much they’re using it, how many problems they’ve solved, or even how much money they’ve made. Which provides a great foundation for the next use case: Helping your customers get even more value.

• How can users reach their goals faster or more efficiently? Ideally, your in-product analytics tell users what’s working and what isn’t by identifying where they need to take action and where they should double down. For example, Uscreen provides creator analytics on their streaming platform to make it clear what content is the most popular with the creator’s audience and how they can grow their community faster.

• What useful features aren’t they using today? Most users aren’t taking advantage of everything a product has to offer. For example, you could surface that users haven’t set up alerts, automated workflows, or used other features that similar users are finding valuable.

Combined with push notification and/or emails, you can also create a powerful re-engagement loop that leverages analytics signals to draw users back into the product and drive sessions.

Scoping your data product

Now, this only tells us why we might want to embed analytics within our product. It still doesn’t tell us what exactly we should build.

To understand that, we’ll need to dig into the data and talk to users:

1. How are your users using the product? Aggregate data on usage patterns, session replays, and user research sessions can give you insights here

2. What ad-hoc data analytics requests do they send to the Support team or their CSM? If your product doesn’t have robust analytics capabilities, chances are, your customer-facing teams are currently filling the gap at least to some degree

3. What feature requests are you getting? Customers, especially your power users, will often vocally tell you what they’re missing

4. What complementary analytics tools exist for your product? Lack of in-product analytics is often seen as an opportunity for tools (or entire startups) adjacent to your business. For example, limited pricing analytics and guidance for hosts in Airbnb’s early product spawned a whole ecosystem of tools doing exactly that

This exercise should give you a pretty good idea of what type of analytics you could build out. But remember, not all data is equally useful, and the same applies to the analytics and data-powered features you build on top.

Here’s a simple hierarchical framework to help you think through this:

Tier 1 analytics are table stakes

This tier includes foundational data such as standardized stats about users’ past activity within your product. Customers expect you to offer these reports, but they’re not going to be a differentiator, and you shouldn’t expect a huge lift in engagement metrics.

• Example: Sales and revenue by product in an e-commerce tool

Tier 2 metrics go one step further to give users additional context for interpreting data

These stats include aggregate trends or benchmarks that put an individual user’s metrics into context. However, it’s still a “one size fits all” experience where your product team needs to anticipate what metrics users will be interested in.

• Example: An online course builder benchmarking stats like completion rate against other similar courses on the platform

Tier 3 & 4 provide customization options and AI to deliver personalized insights

At these levels, you’re making sure users get exactly what they need to solve their most pressing problems instead of just drowning them in more charts.

Customization: Customer- or user-specific data cuts tailored to a particular business or use case (e.g. customized reports or custom fields)

• Example: A creator analytics report with audience insights based on custom segmentation

Insights and recommendations via AI: Timely, personalized, and actionable insights based on users’ own data as well as broader trends delivered through AI

• Example: A content creation and Answer Engine Optimization (AEO) tool that recommends focus areas based on past performance data and overall trends

Finally, there’s one more important consideration that can make or break the success of your initiative: Many products have different distinct user groups, and these user groups often have very different jobs-to-be-done. And as a result, they’d benefit from very different data products.

To better understand this, let’s look at an example.

Example deep-dive: TikTok

I’ve never spent a single moment on TikTok, but from what I’ve been told, there are at least three different stakeholder groups with different needs.

1. Regular users who are looking for interesting content

2. Creators trying to grow their accounts

3. Advertisers trying to promote products or services

Since each group 1) has different goals and 2) does different things in the product, both the data they generate as well as the metrics and insights they care about are very different. Each group has specific needs for tailored in-app analytics for their particular use case rather than generic reports.

Regular users don’t need any analytics on a daily basis as they’re not trying to run a business or optimize anything…and they may not even want to know just how much time they spend in the product each day.

Creators, on the other hand, need to understand how their content is performing and what they can change to increase their follower count. TikTok’s answer: “TikTok Studio”.

Applying our framework from earlier: Tier 1 metrics for creators include stats like views and watch time or visualizations showing where most users drop off during a video’s play time.

In addition, TikTok lets creators understand overall search trends and trending videos (Tier 2), and provides an AI assistant that helps with anything from identifying new video ideas to optimizing performance (Tier 4).

Lastly, advertisers need detailed reporting on ad groups and individual campaigns to understand both how much reach their ads are generating as well as how this translates to down-funnel outcomes like clicks and ultimate conversions. While TikTok offers fairly comprehensive standardized reporting in this area, custom fields would make the reports even more useful (e.g. allowing advertisers to define and track custom ROI metrics).

The million $ question: How should you price and package your data product?

There are two questions you need to answer here:

1. Do you charge for your analytics features?

2. If you do, how do you decide which features should be part of what pricing tier?

There are a lot of considerations factoring into these decisions, but here are a few key dimensions that can get you started.

• Competitive situation: Are competitors offering analytics features, and if so, are all of them available to all users? If that’s the case, you might want to consider focusing on closing this gap and giving broad access (at least initially)

• Importance: It’s easier to justify premium pricing for your analytics features if they’re essential and — ideally — generate measurable value for your users (e.g. help them grow revenue or decrease costs) rather than just being a collection of “interesting” charts that are nice to look at

• Cost: If you invest a lot of resources into developing complex analytics features, you might want to amortize that investment

Next, assuming you want to limit some features to premium tiers, you need to figure out which features to offer in which tier.

In most cases, it makes sense to offer basic analytics for every customer and user, regardless of tier. First of all, customers expect this, and being overly restrictive in this regard can lead to increased churn and difficulties acquiring new customers. Not to mention, the more users can access your data product, the bigger the potential for increased engagement.

Features that are good candidates for premium tiers, on the other hand, are:

• Benchmarks and other insights leveraging data across the wider user base

• Customization (e.g. letting users create their own metrics or reports, or tweak existing ones)

• AI-powered chat or personalized recommendations

In other words, anything that is a true value-add for users and makes their lives much easier (e.g. by eliminating an additional tool or step from their workflow).

Going back to our hierarchy from earlier, this means that it usually makes sense to offer Tier 1 and potentially Tier 2 insights to everyone while reserving Tier 3 and/or Tier 4 for premium tiers.

The problem is that even if these features are great, it can be difficult to demonstrate their value to users that haven’t tried them yet. There’s only so much you can do with marketing copy or video demonstrations.

As a result, you might want to consider a reverse trial. In a reverse trial, new customers get complimentary access to premium features in the beginning; then, after a certain amount of time (e.g. one month), they’ll lose access if they don’t permanently upgrade.

Let’s go through an example to see what this looks like in real life:

Example deep-dive: Parenting apps

Any parent will know this: There is a product — and app — for everything now, including apps to help you track daily activities like feedings, diaper changes, and sleeping.

Many of these apps offer free versions for basic tracking with simple summary reports to show you how much (or little) your child did something. However, it’s increasingly common to offer more detailed reporting and suggestions, such as ideal nap times, at a premium subscription level.

To give parents an opportunity to try these premium analytics features and see if they’re helpful, you can often get complimentary access for a short period of time (the reverse trial discussed above).

As a result, instead of deciding whether to try the advanced analytics features that might be useful, at the end of the trial period users have to decide if they want to give up the feature they’ve already tested extensively (and in many cases found extremely valuable and built habits around).

Creating a flywheel

As mentioned previously, when we talked about Spotify Wrapped, a data product can target both existing and prospective users or customers at the same time.

The holy grail is to design your data product around a network effect so that when you use it to attract new users, the usefulness of the data product for both existing and prospective users increases.

What does this mean? In the Spotify case, when users share their “Wrapped” on social media, they might draw others back into the app (or prompt those without an account to check it out). But those new users don’t make Spotify more useful — or next year’s Wrapped more fun — for everyone else. There are no network effects, and it doesn’t create a flywheel.

In contrast, let’s look at the following scenario:

• Step 1: You run a marketplace for secondhand fashion. To help users price their products, you create an embedded data product for sellers that shows pricing trends over time and suggests a price range for items during the listing process to maximize conversion. This is based on the data you collected from previous transactions through the platform

• Step 2: Then, to acquire more users, you’re adding a simplified version of this data product to your website that allows users to quickly look up the current market price for any designer handbag

• Step 3: This data product is effective because 1) it’s useful and solves a real problem, 2) answers common search queries, 3) is easy to use, 4) and it converts; the logical next step after learning the price of your bag is to list it on the marketplace

• Step 4: And that’s where the network effect comes in: The more users you acquire, and the more bags are being sold via your platform, the better your data gets. This makes your pricing tool more useful for both existing users as well as prospects

How to build your data product

Once you’ve figured out what data product could be valuable, you need to create it. When it comes to a tool on your website focused on acquiring new users, your requirements are likely modest given the limited scope and functionality.

However, if you’re building something customer-facing that’s embedded into your core product (or a flywheel like described above), things get more complex. For example, customers want interactivity and the ability to customize. At the same time, you have to make sure the data they look at is correct at all times.

And things get even more challenging once AI is in the mix. Users increasingly expect interactive AI features, and in-product analytics are no exception. But while AI has the potential to be a massive value-add, it amplifies all of the challenges you’re facing when building a data product: In addition to “just” making sure pre-built views are correct, all of a sudden you need to deal with non-deterministic responses and the potential for hallucinations.

So, what can you do? As always, you have two choices: Build or buy.

“Build” means you’re going to build the data product from scratch. Not just the user-facing frontend components, but the entire foundation as well such as security, user management, metric governance, and performance controls.

“Buy” means you’re choosing an embedded analytics provider and then customizing the solution to fit your needs and product design. You can think of this as a variant of the BI tools you’re likely familiar with; but instead of internal dashboards and reports they power customer-facing data apps.

Typically, this means a trade-off. To decide what’s best for your situation, it’s helpful to think through what building and maintaining an embedded analytics product requires, and then pick the option that most closely aligns with your needs and priorities:

TL;DR

As with most build-or-buy decisions, the biggest draw of building the solution in-house would be the ability to design every detail just the way you want it. If you have extremely specific requirements, know exactly what you want, and an initial evaluation finds that existing solutions cannot deliver on everything, building can be a viable path.

However, in most other scenarios, you’d pull away valuable engineering and product resources to build something that you could get launched to market much faster with a vendor.

Control and customization: Providing a seamless user experience

As mentioned, if you want to be able to tweak how every little thing works and looks, building in-house gives you full control. Embedded analytics solutions allow varying degrees of customization; some are somewhat restrictive and the end result feels “bolted on”, while others let you seamlessly blend each part of the experience — from the look and feel to analytical flexibility — into your core product.

What to look for in an embedded analytics solution:

• Extensive self-serve and customization features that let you tweak the functionality and content of dashboards, reports, and even metrics precisely to customer needs (including the ability to build custom charts and fields)

• The ability to control the entire look and feel of the application, incl. CSS and markdown support

• Proof; marketing promises are one thing, but case studies of customers that have successfully adapted the embedded analytics solution to match their product UI and brand guidelines show you that it actually works

Feature breadth: What capabilities do you need on day 1?

Building your own analytics product from scratch will take a long time; as a result, you’ll likely start with an MVP with limited functionality, and it will take months or years to develop, test, and ship all the features customers ask for.

Depending on your intended use cases, that might be acceptable; but if you need a comprehensive set of features on day 1, buying can massively speed up your time to market.

What to look for in an embedded analytics solution:

• Multiple intuitive interfaces that cater to the preferences and skills of different audiences (e.g. dashboards, spreadsheets, querying, natural language)

• Advanced analytics features like AI and predictive analytics (making sure they’re available when the product is embedded)

Data governance: Accuracy and permissions aren’t optional

As discussed, giving customers access to data comes with many opportunities — but it also presents a risk. If you show data that’s wrong, and users make decisions based on it that end up backfiring, it can have severe consequences.

This means you either have to 1) build a semantic layer to standardize the data model and core metrics for your product, 2) choose an embedded analytics solution that has this functionality, or 3) limit users to a few vetted reports that you can guarantee are accurate.

Security and permissions are a similarly sensitive topic: It’s not enough that data is correct, you also need to ensure that the customer or user in question is supposed to see it.

What to look for in an embedded analytics solution:

• A semantic layer that makes it easy to define standardized metrics across all customers, as well as has the ability to support customer-specific metrics and definitions

• Robust security and permissions, including attribute-based access controls and row-level security that extend to AI and MCP (see below)

• Software development lifecycle controls (SDLC) that allow you to safely test changes and updates before rolling them out to customers.

AI features: A non-negotiable in 2026

User expectations have changed. Where AI features were a nice-to-have only a few months ago, they’re more or less expected at this point. It can be tempting to just point an LLM at your data and let users chat with it, but that’s going to cause more problems than it solves. You need the right foundations in place to give users a frictionless experience while ensuring accuracy at all times.

Among other things, you need to make sure the AI has the right context, learns and improves over time, respects permissions, and responds quickly and accurately in a format that works in the context of your product; otherwise, it’s a risky gamble that can quickly go wrong.

What to look for in an embedded analytics solution:

• A comprehensive set of AI features. Natural language chat is just one option, but there are more ways to bring AI into your customers’ workflows with helpful summaries & recommendations, and dashboard helpers to answer follow-up questions

• A semantic layer that allows you to curate context and define workflows for AI to ensure reliability and help AI understand what matters to your customers

• An MCP server so you can bring analytics functionality into existing AI experiences you have (e.g. a general-purpose chatbot in your product, your AI support agent etc.)

Scalability: Don’t let engineering become a bottleneck

Most of the real challenges in maintaining an analytics product pop up over time as you scale the number of users while continuing to make changes based on learnings and customer feedback.

With a homegrown solution, engineering will quickly become a bottleneck for any changes. And, if you haven’t built a robust shared data model as a foundation, the complexity of keeping every customer’s reports accurate will increase exponentially.

What to look for in an embedded analytics solution:

• Low-code or no-code options for developing data products so Product and Customer Success teams can ship independently of engineering constraints

• Robust Git integration, version control, and testing environments so you don’t risk breaking things (and showing customers broken things!) as you continue to build out your data product

Performance: Users want answers NOW

Finally, it doesn’t matter how helpful and intuitive your data product is. If it doesn’t load quickly, users will run out of patience and stop using it. If you’re a data person reading this, you may know what it’s like to get complaints about slow dashboards from internal stakeholders. But for in-app analytics, we’re now talking about your customer experience.

Customers will have much higher expectations and want answers fast. If you plan to serve up mostly simple reports of pre-aggregated metrics, this will be less of a concern regardless of whether you build or buy. But if you want to let users explore and drill into granular data, you’ll quickly run into latency issues and skyrocketing data warehouse bills.

What to look for in an embedded analytics solution:

• Modern architecture that integrates with your data stack & storage (Snowflake, Databricks, AWS Redshift, Google BigQuery, Clickhouse, etc.)

• Sophisticated caching that guarantees fast load times and limits unnecessary queries to the data warehouse

Making your embedded data product successful

Creating an embedded data product requires an investment, whether you choose to build or buy. As a result, it’s key to treat it like any other product and make a holistic and continued effort to make it successful.

In addition to what we discussed above, there are two key considerations:

Distribution

As discussed above, data products for attracting new customers ideally come with built-in distribution as they address common search queries or LLM questions. However, for embedded analytics, it’s not that simple. If you just build these features and move on, you’ll likely be disappointed by the adoption and engagement rates you see. You don’t want to exclusively rely on users reading the announcement email or clicking through the menus to find it.

At the same time, growth product teams are constantly looking for new “signals” or “triggers” they can use for emails and push notifications to draw users back into the product and grow sessions.

The best data products solve both of these problems by serving as a “signal generation engine” for user-facing comms. AI-generated summaries and takeaways provide ideal copy for push notifications or emails to draw users back into your product and get them to explore the analytics features specifically. And after a while, users will build habits and come back even without any active reminder.

Iteration

Lastly, the first version of your data product that you ship shouldn’t be the final one. Initially, you have to work from first principles and guess what will be useful to your users. However, once they start using the in-product analytics, you’ll get valuable data to refine your offering.

• What types of pre-built reports get used the most?

• Which reports get shared with other users?

• What types of customizations are users making to get insights that aren’t available out of the box?

• Do users still contact Customer Success or Support because they cannot get certain answers from the analytics features? If so, what are the most common requests?

• What questions are users asking AI?

This last bucket is likely the biggest data treasure trove of all. Instead of trying to infer what users find helpful from proxies like usage patterns, they are literally telling you what they’re trying to accomplish via their natural language queries.

In addition to informing your feature roadmap, you can also use this data to decide what defaults to show new users who haven’t customized their analytics yet. Nobody wants to click through five filters and dropdown menus before they see something valuable, so anticipating users’ needs goes a long way.

However, while you should be iterating on and improving your analytics features over time, you don’t want to risk breaking things in production and showing customers wrong data. As a result, it’s key you’re leveraging SDLC workflows. For internal analytics it’s an extremely helpful best practice; but for customer-facing analytics, it’s a non-negotiable requirement.

“As CTO, it’s crucial to me that we can test changes before deploying to our customers.

Omni was the first tool we saw that truly struck the right balance of giving our customers more flexibility, while keeping our team in control.“ — Arsh Mand, CTO & Co-founder at Workramp

Made with 🩷 by the folks at Omni

Most content focuses on what AI might do in the future; and while these predictions are often interesting, they provide little value for data teams looking to leverage AI right now.

This article takes a different approach. Instead of looking into our crystal ball, we’ll discuss concrete ways your data team can amplify its impact by using the current generation of AI models in creative ways, including real-world examples and prompts you can try out yourself.

Because even with its current capabilities, AI can fundamentally transform how your data team operates:

“We use AI to accelerate most of our data workflows — from generating models and documentation to running analyses. If all LLMs were down tomorrow, it would take me ten hours what I’m used to doing in ten minutes.” – Juliette Duizabo, Head of Data at Photoroom

Specifically, we will cover how to use AI to help:

1. Build (or update) your data stack

2. Clean and transform data

3. Scope and ship complex strategic data projects

4. Deliver analyses that drive business impact

5. Enable business users to move faster

One big caveat before we dive in: I strongly recommend never just copy-pasting AI-generated code without fully understanding how it works, at least for any high-stakes (production) use case. AI can accelerate your work by acting as a thought partner and automating manual grunt work, but you still need to use your judgment and are on the hook for everything you ship.

1. Using AI to help build out (or update) your data stack

The landscape of available tools, and what a “best in class” data stack looks like, changes every day. As a data leader, that can be overwhelming; you feel constant pressure to stay in the loop, and “wrong” choices can lead to a setup that will create real headaches as you scale.

Since AI is strong at researching and synthesizing information, it can be a great partner in building your list and staying up-to-date. The ways AI can help here include 1) researching what tools exist and what they excel at, 2) comparing tools side-by-side, and 3) picking ones that work well together and fit your particular situation.

Note: The most important thing is to provide detailed context on your situation. There is no overall “best data stack” or “best tool”; what makes sense very much depends on the type of company you’re at as well as your priorities.

This includes:

• 🏢 What your company does, how big it is, how fast it’s growing, where it’s based etc.

• 🛠️ What other tools you’re using across the company

• 💰 Your budget, and how much headcount you have to support the implementation and ongoing maintenance

• 🙋‍♂️ Key workflows and use cases you want to support, e.g. self-service by business users, or customer-facing in-app analytics

If you’re starting more or less from scratch, you can kick things off by asking for an overview of the main approaches to structuring a data stack and a detailed comparison of the tools available.

However, most likely you already have a set of solutions in place instead of starting with a blank canvas. In this case, you can also use AI as a thought partner to evaluate your setup and suggest opportunities for improvement.

This prompt can get you started; just replace what’s in the [square brackets]:

<goal> I’m a [Head of Data] at a startup, and am trying to update our data stack. Please review our current set-up and the additional context below and highlight opportunities for simplification and improvement. Our goal is [add your goals here; e.g. reducing cost, simplifying maintenance, enabling self-service etc.]. For each potential change, outline a few alternatives with their pros and cons. </goal>

<context> [Add context on your company and data stack here] </context>

<structure> Follow the Pyramid Principle: State key takeaways or recommendations first, then add supporting arguments and data where appropriate. Use bullet points, overview tables and other formatting to make the report easy to parse </structure>

<instructions> Pick recommendations that fit our situation, but can also scale with us for the next few years. Explain your reasoning in detail, and provide a decision framework (i.e. explain how our decision should change as a function of what we care about most). Include data (both statistics and anecdotal) on what other startups are doing wherever available. Don’t (just) focus on established legacy players; include newer tools and startups as well, especially if they align with our goals </instructions>

If you’re already considering specific tools, just include those in the prompt. Then, once you decide on a tool you want to introduce or replace, you can work with AI to draft the migration plan.

AI likely won’t be able to get you something that covers every nuance of what you need, and you still have to think through this and make the ultimate decisions yourself. Plus, you should definitely consult with the vendor you’re planning to switch to as well as data communities, like dbt and Locally Optimistic; not all information is public and thus accessible by AI, and nothing beats first-hand information from people who have “been there, done that”.

But AI can quickly help you get up to speed and put an initial framework in place that you can iterate on, which will make these conversations much more productive.

2. Using AI to clean, model, and visualize data

Everyone who has worked in data for a while knows that a lot of the hard work happens under the hood, invisible to business users, which can feel thankless at times.

To get a simple, clean dashboard showing e-commerce orders or B2B SaaS sales opportunities, you need to wrangle dozens of data sources, clean the data, join it together, and model it in a way that reflects how the business operates (and makes sense to business users).

These tedious tasks used to take up most of the time for data teams, leaving little space for the really interesting and rewarding tasks and projects. But now, it’s finally possible to pass a lot of the grunt work over to AI, so you can focus on guiding, reviewing, and working closely with business partners to drive impact.

To give you some inspiration, here are some example use cases that general-purpose LLMs can already handle well today:

Using AI to clean data

Here are two things that are simultaneously true:

1. Every person working in data will tell you how important data quality is (we love saying “garbage in / garbage out”)

2. Nobody likes actually cleaning data

So what do we do with that? We outsource the work we don’t want to do, of course.

If you give some context and a few examples, current AI models are remarkably good at cleaning data at scale. The key is to give it things that are:

1. High-volume (otherwise you could just do it faster yourself),

2. Straightforward enough (otherwise it’ll struggle), and

3. Easy to QA (so you don’t mess things up by accident).

Let’s take CRM data as an example.

As your company scales, the data quality in the CRM is bound to deteriorate. Plus, as more people use the data, issues that used to fly under the radar and cause few problems suddenly become major pain points.

Whether it’s sales reps and automated workflows creating duplicate accounts, or leads mis-typing their emails when filling out forms: If you’re not careful, you’ll quickly reach a point where issues that started as exceptions now make up a meaningful share of your total records.

Integrated duplicate management features etc. in the tools you’re using can catch some of these issues, but they’re often rather simple and use a set of incomplete rules. LLMs excel at these kinds of tasks, though; just give it access to your CRM data (or upload a list of accounts) and ask it to flag potential duplicates or typos with a confidence level.

Please review the attached csv of CRM data and flag any accounts that might be duplicates. Please 1) provide a confidence level and 2) explain your reasoning.

That way, you can afterwards do an efficient review focusing on the ones most likely to be problematic, and you still have full control to make adjustments as needed.

You can do the same for plenty of other similar use cases; for example:

• Filtering out personal emails from B2B data

• Standardizing values like industries, geographies etc.

Using AI to design taxonomies (e.g. UTM tagging or campaign naming conventions)

A common reason why you end up with messy data like in the examples above is because there is no clear taxonomy to follow. For example, without guidelines in place that standardize the values, UTM data often ends up looking like this:

If you try to build anything (e.g. attribution) on top of this, you’re going to have a hard time. And even if you do a cleanup: Without a taxonomy that will prevent this going forward, you’ll have to repeat that work over and over again.

The same goes for campaign names, be it in LinkedIn, Salesforce, or your email tools. When you set up your first campaigns, you rarely think about all the metadata you’d like to have down the road. Then, when you’re trying to translate the raw data into meaningful insights, you have a rude awakening.

To protect your data team’s sanity, you definitely need to implement some best practices. But at the same time, you don’t want to make things overly complex and slow the business users down or make it impossible for them to actually follow the process.

AI can be a good thought partner in thinking through your options, as long as you provide sufficient context about your situation. For example:

• What marketing programs and types of campaigns you’re running

• What reports users typically run and what dimensions they’re trying to filter on

• How you’re capturing metadata today, and what the key pain points are

Based on this information, you can get a customized overview of your options, including a recommendation for how to move forward:

Using AI to structure and document dbt projects

When you’re kicking off a dbt project, you usually start with an idea of what your final data is supposed to look like. Then, you work backwards and decide how you’re going to structure the series of joins and transformations that will get you there.

There are plenty of ways to do this that lead to the same outcome, but they are not all created equal; some of them will make it easier to understand and debug the pipeline, or re-use certain components for other intermediate and mart models.

Going through all of the possible permutations and their pros and cons can take a while, and having a thought partner makes it infinitely easier. AI can help you think through:

• How to split things up into staging, intermediate, and mart models

• Where it makes sense to leverage seeds or macros

• How to materialize the different models (e.g. view vs. table vs. materialized view)

• What to model in dbt, or in the semantic layer of your BI tool

Instead of asking the AI to develop a proposal from scratch, I typically approach this by writing a rough draft SQL query that establishes the desired output format as well as key fields and joins, and then working with AI to split it up into dbt models, seeds, and macros:

I have the following SQL code that I want to translate into dbt models.

Please review it and then suggest how to split it up into staging, intermediate and mart models, as well as seeds or macros where it makes sense. Please provide detailed reasoning for your recommendations, and alternative approaches where there are multiple reasonable ways to implement something.

Provide 1) an overview of the proposed dbt project structure as well as 2) the code and a naming suggestion for each proposed model.

One common pain point here is that if you’re not starting with a blank slate, the proposed models you’ll get back often don’t fit neatly into your existing project. This is where dbt’s MCP server comes in handy: By giving the AI full visibility into everything that’s already in place, you’ll get a much more customized proposal, without having to type up any more manual context.

A few things to keep in mind:

• Note that I said “help you think through”, not “tell you”; I recommend using AI as a sounding board and brainstorming partner, not to outsource the ultimate judgment of what should be implemented. At least for now, I always recommend having a human in the loop

• Understanding best practices is important, but there are lots of reasons why you sometimes want to deviate from the “ideal” textbook way of doing something (especially in a startup). LLMs tend to be pretty dogmatic with regards to dbt, so I recommend you treat the proposals with a grain of salt and decide where you want to cut corners

Finally, Future You will thank you if you properly document any new models you create. Otherwise, it’ll get harder and harder to properly leverage them downstream, whether by humans or AI.

AI has massively reduced the time this takes, so what used to be a major pain point for most data teams should now just be a no-brainer final step of the development workflow:

“Every time we’re going to deploy, we run the table through dbt Copilot to get a high-level summary and documentation, which automatically gets ingested by Omni’s AI. If we need to tweak or add to the metadata, it’s quick. We’re able to get reliable AI responses without requiring tons of effort from my team.” – Austin Aranda, Director of Data Solutions at SWBC

Using AI to optimize SQL code

If you paid close attention to the example screenshot above, you might have seen that ChatGPT took a jab at my SQL code (“400-line case statement”). While it’s a slight exaggeration, there is a kernel of truth to it: When I write a SQL query, I first write it in the most straightforward, brute-force way imaginable, even if it’s inefficient computationally or results in way too many lines of code.

The reason is simple: I first want to see if it works and produces the right outputs before I spend any time optimizing.

Then, I go back and refactor the query to be more efficient, handle all edge cases correctly, etc. Doing this can feel rewarding, and it’s good to practice it occasionally to hone your SQL skills, but I often get an itch to move on to the next 0 -> 1 task instead.

So I’ve started turning to AI to help speed things up.

You can either share your query and ask generally for opportunities for improvement, or you can pick specific aspects you want feedback on. For example:

1. Query structure / joins:

   1. Is there a more efficient way to structure CTEs and joins?

   2. Are you potentially creating duplicates or dropping rows unintentionally?

2. Edge cases: Are you anticipating and handling all possible edge cases in the data you’re processing?

If you’re using an AI-assisted IDE like Cursor, you can get that feedback without switching tools. Otherwise, copy-pasting code into one of the major chatbots also works pretty well and only adds minimal friction.

“Whenever I write code, I always use AI to debug because it makes it so much quicker. For example, something that I really like about Cursor is that when anyone raises a PR, it does the first code review. We’re a small team and we work really fast, and it’s really helpful.” – Edward Mancey, GtM & Ops Data Lead at Synthesia

Using AI to create custom visualizations

Most BI tools offer a wide range of common visualization types out of the box. But sooner or later, there will be a use case that will motivate you to create a custom visualization beyond what’s readily available.

I ran into this when I wanted to create a visual timeline of how prospects and customers interact with us at conferences or events we host: Who registered, who showed up, who stopped by our booth etc..

• 😊 The good news: Omni lets you create any visualization you want using VegaLite

• 🥺 The bad news: I had never used VegaLite before and was unfamiliar with the syntax; and I didn’t want to spend hours on a simple visualization for a single dashboard

So I turned to AI; since I was creating this visualization for an internal report, there was limited risk of breaking anything important, and speed was more important than writing the most elegant code.

Within a few minutes, I had a working prototype. And after 10 more minutes of formatting tweaks, it looked pretty neat:

And the best part: All it took was a brief natural language conversation instead of writing 100+ lines of code by hand.

3. Leveraging AI to scope complex analytics projects

Using AI to write specs and project plans

Many projects fail before they even start.

There are many reasons for this, but one common one I’ve seen over and over is that the scope of complex 0 -> 1 projects is so overwhelming that the team spends weeks (or even months) in analysis paralysis and never actually develops any momentum.

For example, let’s say the marketing and sales teams asked you to develop an account score. Where do you even start?

• Should you buy software or build this in-house? If you buy, which vendor is the best?

• If you build it in-house, what type of model do you need? What should the output look like? What features are relevant? How do you operationalize it once it’s built?

Individually, none of these questions are particularly difficult, but together they can seem insurmountable.

AI, specifically Deep Research, can make this initial phase much easier. I recommend a two-step approach:

1. Step 1: Ask a reasoning model to help you think through everything that’s relevant at a high level

2. Step 2: Use the information from step 1 to generate a prompt for Deep Research

This will get you an in-depth, step-by-step guide for the topic of your choice. I don’t recommend completely outsourcing the key decisions to AI (e.g. build vs. buy, or the exact approach you’re taking), but it’s much easier to iterate on a concrete proposal than to start from scratch with the input of a dozen stakeholders.

Step 1: Using a reasoning model to brainstorm key considerations

There’s no need to over-engineer the prompt here. Something simple like this will get you a good overview of the key considerations:

I am planning to build an account score, and I plan to use Deep Research to generate a detailed guide. How can I make sure I get a customized, actionable result? Please touch on

1) things I need to decide before I proceed
2) context I need to provide to get a customized output, and
3) sections I should ask for in the report (these should cover all important aspects of building and operationalizing an account score).

That’s a pretty solid starting point, I’d say. You don’t have to answer all of the questions the AI raises before you proceed, but you’ll get a much more useful playbook if you at least get clarity on a few central ones (like what your ultimate objective / use case is).

Step 2: Generating a Deep Research report

Assuming you have clarity on your goal, the type of output you want etc., it’s time to generate the detailed guide. I recommend picking the most helpful suggestions from the AI and complementing them with your own thoughts.

💡Note: You can also ask AI to generate the prompt (in fact, GPT-5 proactively offered this), but I recommend writing it yourself. AI-generated prompts often contain a lot of stuff you don’t actually need, and that will dilute the report.

For the account scoring example from above, an effective Deep Research prompt looks like this:1

<goal> We want to build an account score to [your goal (e.g. account allocation to SDRs + prioritization]. The score should [any requirements w.r.t. the output, e.g. separate firmographic fit and intent scores, or a high degree of explainability] </goal>

<context> [Add context regarding your company; e.g. your industry, product, business model, ICP, marketing & sales motions, marketing & data stack, envisioned timeline & deadlines, budget, etc.] </context>

<structure> Provide an overview of the key decisions that need to be made and the major work streams. For each body of work, provide an overview of different relevant approaches we could choose from, and then provide a detailed step-by-step guide for your recommended one </structure>

<content> Please cover, at a minimum, the following topics:

- Build vs. buy, including a detailed comparison of available product solutions
- A comparison of various approaches to building this in-house (e.g. across criteria like effort to build and maintain, explainability, sophistication etc.)
- An overview of the key firmographic characteristics and intent data points we should use, and the best tools / vendors to source this data from
- How we backtest and evaluate accuracy
- How we operationalize the score
- Which team should own this during 1) development and 2) on an ongoing basis
</content>

<instructions> Whenever there are multiple valid approaches, please compare their pros and cons and give a decision framework rather than a simple recommendation, so we can work through what works best for us. Support any claim you make with in-text citations. If you need additional context, please ask for it </instructions>

Based on this, after 10 - 20 minutes, you get back an extremely detailed guide (in this case, over 14,000 words):

Even if the report you get back looks solid at first glance, I recommend treating it as a first draft instead of a final deliverable. Ask follow-up questions and push back on anything that doesn’t make sense to you. For example, one key weakness in the report above is that it relies heavily on a few sources, so you could ask for a more balanced assessment and/or specify which sources to use.

Then, once you’re happy with the general approach, you can ask for additional implementation guidance. Sticking with the account scoring example from above, that could be:

• An exhaustive list of features to consider

• An overview of the proposed architecture of the model

• Code snippets in the language of your choice to start building a prototype

Also: If you have strong opinions on how you want to do things and don’t feel comfortable leaning on AI this heavily, there is still value in using AI as a thought partner.

Instead of asking for a guide, you share your plan and then ask for feedback:

• Did you miss anything important?

• Are there any edge cases or downstream consequences you’re not thinking of?

• Is there a way to simplify or speed things up?

Using AI to create mock-ups and ensure you’re building the right thing

There’s nothing more frustrating than spending weeks on a project only to realize you built the wrong thing and nobody is using it.

At that point, it doesn’t matter who is “at fault”; whether business stakeholders didn’t communicate their needs clearly, or the data team didn’t describe the envisioned solution in an intuitive way, the work went to waste and you have to start over.

By far, the easiest way to prevent this from happening is to create a mock-up of what you plan to build before you write any code. That way, you can spot — and clear up — any disconnect immediately. Before AI, that meant spending hours in Google Sheets or Figma; but now, you can create a mock-up with a series of targeted prompts.

I generally recommend starting with a static UI, for two reasons:

1. A static UI mock-up is much easier to build and is often enough to get the feedback you need

2. And if it isn’t, you can turn it into an interactive prototype; and that prototype will turn out much better with this reference as opposed to starting from scratch

Let’s say we want to build an internal tool that visualizes experiment results for business teams. Our key idea is that instead of just displaying a list of metrics like in most tools, we want to display them in a driver tree whenever they’re interconnected.

Before we actually build a production-ready version of this, though, we want to make sure users find it as intuitive in practice as we think it will be.

There are a few different ways we can go about the mockup(s):

Option 1: Start with a hand-drawn sketch

If you have a pretty clear image in mind of what you want, it’s often fastest to just put pen to paper and sketch it out. Or, if you’re not artistically inclined, spend a few minutes in Google Slides or a similar tool creating the rough outline.

I have zero (I repeat, zero) drawing skills, but I was able to get something close to what I had in mind with ~1 minute of sketching, a single-line prompt, and another minute of wait time while Claude turned my terrible mess of a sketch into an HTML-based mock-up:

Option 2: Describe in detail what you want and iterate your way towards a mock-up

Alternatively, if you don’t feel like drawing, you can also just briefly describe what you want. This is what I got from Claude with a ~150 word prompt and a bit of feedback on the initial output:

It’s HTML-based and already comes with some basic interactivity; for example, you can scroll through the metrics table.

At this point, you could either keep providing feedback and ask the AI to add more features (e.g. tooltips when you hover over certain parts of the dashboard), or you can switch to a proper vibe-coding / prototyping tool like Lovable or Vercel V0 and ask it to turn this mock-up into a fully-functioning prototype app.

Alternatively, if we have multiple possible layouts in mind and are trying to decide, we can also mock up a few variants and then ask stakeholders which one they prefer.

Then, once you get feedback from stakeholders and finalize your spec, you can either improve your prototype until it’s production-ready, or you can rebuild it in a more robust fashion from scratch.

Either way, you know you’re building the right thing.

Bonus: Incorporating real data into your mock-up using an MCP server

One downside of the approach discussed above is that it will create a mock-up or prototype full of “fake” data; while this isn’t a huge issue, it will make the result feel less real to stakeholders.

If you want to go the extra mile and show them what the dashboard or tool you plan to build will actually look like in production, you can use an MCP server to give the AI access to your data. When your mock-up shows the actual list of experiments you ran last month, or top accounts by revenue, or familiar month-over-month growth rates, it’s much easier to get people comfortable with it.

4. Using AI to level up written and verbal presentations

Improving delivery of data-based findings & recommendations

The number one most common reason why I’ve seen dashboards, models, or data tools fail to gain adoption was discussed above: There was misalignment (or miscommunication) with regards to what was truly needed to solve a business problem, so the analytics solution that was built didn’t add value.

The second most common failure mode I’ve observed happens on the opposite end of the process: Even if the problem was well understood, potentially impactful recommendations or solutions don’t get implemented because the data team often struggles to effectively influence business stakeholders and clearly articulate the business value of their work.

Over the years, I’ve spent countless hours working with my teams to make their communication more concise and impactful. Unfortunately, doing this 1:1 as a manager is not particularly scalable.

That’s where AI comes in.

Even with default settings, current generation models from OpenAI or Anthropic can provide robust feedback on presentations or documents. If you want exceptional results, though, you have to provide custom instructions.

You can either share these with members of your team so they can add them to their settings or Projects as desired; or, if you want central control, you can create a custom GPT that’s accessible to everyone on your team.

I recommend using instructions along these lines (you can customize these based on your and your organization’s preferences):

When you’re reviewing presentations, documents etc., please make sure they adhere to the following principles and best practices. Where they don’t, provide detailed, actionable feedback.

- Problems should be structured in a MECE way; i.e. the problem should be decomposed into mutually exclusive and collectively exhaustive components

- Communication should follow the pyramid principle: Starting with the most important insight, recommendation, or takeaway, and then providing supporting arguments and data as necessary

- Details that are not essential to the core understanding, or all stakeholders in the audience, should be moved into the appendix to be referenced as needed; nothing should distract from the core message

- Communication should be tailored to the audience and focus on what’s relevant for them (e.g., what should they focus on or what should they be doing differently?) instead of recapping what was done, how the model or analysis works in detail etc.

- Encourage use of frameworks such as issue trees, 2x2 matrices etc. and visuals such as flow charts to facilitate a joint understanding of complex subject matter

- Make sure slides tell a story; it should be somewhat clear from reading just the slide titles what the key takeaways are (i.e. titles should state the main point of a slide instead of just saying “Overview” etc.)

- Make sure technical jargon gets minimized and replaced with plain English wherever possible

- Simple charts (line chart, bar chart with single Y axis) are preferable over complex ones; it’s better to split things into multiple individual visualizations (or scrap some of them) instead of creating complex charts that take people a few minutes to understand

- Make sure chart commentary actually adds helpful context or insights instead of just restating what’s visible on the chart

With these instructions in place, all you or your team members have to do is share a slide deck or document and a bit of context (what the purpose is, who the audience is etc.), and you’ll get detailed feedback on how to make it more impactful in return.

If you’re a team lead, you might still want to review important presentations or documents before they go out. However, the GPT can take a first stab at feedbacking your team’s deliverables and save you a lot of time since you’ll review a version that’s already substantially more polished.

Using AI to anticipate stakeholder questions

Besides making sure written deliverables get to the point and tell a clear story, you can also leverage AI to anticipate questions you’ll likely get from stakeholders.

Nothing is more frustrating than getting a question five minutes into a presentation that derails the entire thing. And if you are not prepared to answer it well, it can destroy trust in the entire analysis (even if it’s fundamentally robust).

An easy way to prevent this is to generate a list of likely questions in advance and prepare answers to them; you can then drop supporting data or arguments in the appendix and pull them up when needed.

You can obviously do this manually, but it can take a while to get an exhaustive list; plus, if you’re deep in an analysis, you often miss obvious questions that others that are less familiar with the topic might ask.

That’s why I recommend brainstorming this with AI as a thought partner. As always, the key is to provide as much context as possible. In this case, that means:

1. Attaching whatever you plan to present

2. Listing the most important audience members (e.g. Head of Finance, VP of Sales)

3. Explaining the purpose of the meeting (e.g. making a particular decision)

I’m meeting with [list of key audience members] to [purpose of meeting].

Please review the attached materials I plan to present and compile a list of likely questions each stakeholder might ask. If you need more context to customize your response (e.g. their current roadmap, goal metrics etc.), please ask for it.

As always: The more context you provide, the more customized and helpful the response will be. But even the generic ones AI comes up with are often pretty helpful to prepare.

By the way; you might get some fundamental feedback this way and, as a result, have to re-work your slides or document extensively.

To minimize this risk, you can share the “raw” data you plan to present (e.g. a forecast, experiment read-out etc.) and the context described above, and then ask for 1) a list of likely stakeholder questions and 2) where you should focus your presentation before you start putting it together. Then, after you create your draft, you can ask for a final review to optimize for flow and key takeaways.

5. Using AI to enable business users

Arguably one of the best aspects of AI is that it’s always available and endlessly patient. That means it can provide a level of support and handholding to business users that data teams would never be able (or want) to offer.

• Five people from different teams are asking the same question? No problem.

• Someone needs a 200 line query reviewed, right now? Sure, why not.

AI can guide users across the entire process from finding and pulling data to visualizing it and distilling insights. In many cases, you just need to give stakeholders a few examples of frequent use cases that AI can handle well to get them started, and they’ll quickly come up with other ways to use it themselves.

To help with that, here are three concrete ideas to get you (and your business stakeholders) started:

1. Using AI to structure analyses and investigations

One of the most common analytics use cases for business users is investigating metric movements; for example, rapid spikes or drops, or gradual declines, in metrics the business cares about.

In my experience, these investigations rarely fail due to execution mistakes. Instead, the issue is typically that people don’t know how to approach them so that they systematically and reliably rule out hypotheses and close in on the actual root cause.

Luckily, AI can act as a coach that helps put a structure in place. This can be either through general-purpose AI chatbots, or via AI features in your BI tool. Either way, users can simply share some context on what they’re seeing and ask for advice on how to tackle the investigation:

The best part: After getting an initial structure and starting the investigation, users can report on their findings and get updated guidance, iterating their way towards a solution:

2. Using AI to explain data and models

As a data team, a lot of the value is not created by building things, but by explaining how things work and especially what they mean to business stakeholders. For example:

• How should they interpret the experiment results they see in the dashboard or experimentation tool?

• How can the SHAP values of your ML model help them understand how the model arrived at its predictions?

• What’s the right way to think about accuracy, precision, and recall of a model?

LLMs are great at giving these explanations in a way that’s tailored to an audience’s level of technical understanding. Ideally, you integrate an AI assistant into whatever surface business users are consuming this data at; but even just taking screenshots of the relevant data works surprisingly well in most cases (especially if you’re using a custom GPT that has context on your organization and product).

3. Using AI to pressure-test takeaways

It’s easy to get fooled by data, especially if you don’t have formal training or work analytically on a daily basis. This is one of the big risks when business users self-serve their data questions without any involvement from the data team. At the same time, the data team doesn’t have time to sign off on every conclusion people draw from dashboards or ad-hoc data pulls.

AI can act as a safeguard and prevent the most common issues, though. While it won’t catch every last thing, it is fairly reliable in pointing out any leaps in logic and potential confounding factors.

All you have to do is encourage your business stakeholders to habitually run their conclusions by a chatbot before proceeding; while custom instructions can help make the AI more critical, even the default settings work well.

TL;DR

All of the hot takes and speculative predictions about the future of AI in online discussions or at industry events can make it difficult to find practical, proven use cases among the noise.

That’s unfortunate, because even with today’s models and tools, there are plenty of tedious, time-intensive tasks that data teams can streamline with the help of AI. From building out your data stack to cleaning datasets, guiding metrics investigations, or making stakeholder presentations more concise — there’s almost no part of a data team’s work that doesn’t benefit from AI in one way or another.

Hopefully, this article gives you some inspiration to get you started.

Made with 🩷 by the folks at Omni

The hope is that by empowering stakeholders across the company to access data more directly in a self-service manner, they’ll be able to get insights faster and move more quickly.

At the same time, the data team frees up capacity to focus on more strategic projects instead of mostly fielding ad-hoc requests.

But while this sounds great in theory, most data leaders have yet to see it done successfully in practice.

Biz users either get stuck, unable to get the answer they need (and then still need to escalate to the data team for help), or they make mistakes and draw the wrong conclusions from the data. In the end, instead of making things more efficient, it can often feel like you just traded one set of problems for another.

But that doesn’t mean you should give up on self-service analytics altogether. As always, reality is nuanced; self-service is neither the solution to all of your problems, nor an empty hype you should ignore.

Over the years, I’ve seen data democratization efforts at companies of all sizes, from early-stage startups to big tech companies. And I’ve come to the realization that there are concrete reasons why most companies’ attempts fail. The good news: Most of them can be addressed with a thoughtful approach.

This article will walk through how; specifically, we will cover:

• Why most data democratization efforts fail — and how to avoid these mistakes

• How to define an approach that works for your organization

• The concrete steps you can take to successfully roll out a self-service effort in your company

Why most data democratization efforts fail

While every company is unique, many attempts at data democratization get hamstrung by similar problems.

Let’s look at an example of how this often plays out in practice:

• Step 1: Leadership decides to move to a self-service model

• Step 2: The data team creates dashboards for common use cases and provides a few sample queries for additional deep dives

• Step 3: An announcement is made that business stakeholders are now expected to handle their own ad-hoc data needs, and requests to the data team will be declined. Expecting their bandwidth to be freed up by this change, the data team reallocates its resources

• Step 4: Business stakeholders are still submitting questions and requests. Instead of asking for data pulls, they’re now asking for support with queries and help with metrics discrepancies

• End result: The data team is spending as much time supporting business stakeholders as before and strategic projects are pushed to “later”. The data democratization effort is seen as unsuccessful and teams move on to other priorities

The issues described here can be traced back to two common themes:

1. Ill-defined or unrealistic expectations

2. Insufficient investment into the ongoing data democratization effort

Let’s look at these in more detail so you can avoid falling into the same traps.

1. Ill-defined or unrealistic expectations

Many companies don’t clearly define what they actually mean by “enabling self-service” or “democratizing data” at the beginning of the work stream.

As a result, different stakeholders develop their own interpretation of these vague terms; and when that happens, the effort is doomed from the start. Regardless of what actually gets implemented: Since everyone has different expectations, someone will be dissatisfied with the outcome.

And even when there are explicit expectations and goals, they’re often overly optimistic. As a result, disappointment is guaranteed even if you’re making meaningful progress towards self-service.

First of all, data democratization is a spectrum rather than a binary choice. This means it’s unrealistic to expect all analytics tasks to be handled by stakeholders completely autonomously without any data team involvement.

Similar to how AI coding tools don’t remove the need for oversight from engineers, a push towards self-service analytics doesn’t completely remove the data team from the equation. Instead, the role evolves from executing every single analytics task end-to-end to enabling business stakeholders to get the insights they need and handling follow-up questions.

The benefit of this transition is that the focus shifts towards high-leverage instead of low-leverage tasks:

• Doing data pulls and creating ad-hoc visualizations for stakeholders is low-leverage: Each request typically only benefits one stakeholder, and there are few long-term benefits (solving one doesn’t make the next one more efficient)

• Building self-service foundations is high-leverage: You do the work once, and 1) it affects many stakeholders 2) over an extended period of time. This amplified impact means that some bandwidth will be freed up to work on other projects

Secondly, it’s equally unlikely that this transition will happen overnight. I often see hard “cutover dates” for these types of efforts; i.e. after a certain date, teams are expected to adjust to the new state of the world and any requests to the data team will be rejected.

However, that’s not how things work in practice. Stakeholders need time to build their analytics muscles (and confidence), and they’ll inevitably run into many challenges as they handle these new tasks for the first time.

As data experts, it’s our job to guide them on this journey — not leave them to fend for themselves.

2. Insufficient investment into the ongoing data democratization effort

As a direct result of these unrealistic expectations, there is often an insufficient investment into the necessary education, tooling, and processes to enable successful data democratization.

First, the initial resource investment that’s required is often underestimated. Business users won’t be able to self-serve using the same processes and tools that the data team uses, so this means data workflows have to be reimagined based on the capabilities of the new user base.

That’s why there’s so much excitement around AI right now; in theory, it has the potential to remove a lot of the friction around data access and insights generation for business users. We’ll get more into this potential (and caveats) below.

In addition, the resource investment shouldn’t be viewed as one-off. As mentioned above, data teams should expect to remain involved by creating self-serve resources, fielding follow-up questions, and putting guardrails in place, so you need to budget bandwidth for that.

So, what can you do to avoid the failure modes discussed here? In the next section, we’ll get into how you can define a more realistic self-service strategy and the concrete steps you can take to successfully implement it.

4 steps for successful data democratization

A successful data democratization effort has several key requirements. The most important steps are:

Below, we’ll double-click on each step and talk about how you can customize this for your particular company.

1️⃣ Defining your vision for data democratization

Without a clear vision, you 1) can’t get stakeholders bought into what you’re trying to do, and 2) won’t know what you need to put in place for a successful transition.

At a minimum, you need to define:

1. Why you’re implementing self-service and what impact you hope to achieve

2. What analytics tasks should move towards a self-service model, and

3. How the degree of self-service varies by business area or team

Defining the motivation for and desired impact of self-service

The clearer you are about why you’re pursuing self-service, the easier it will be to know if you’ve been successful. For example:

• Do you hope to free up bandwidth to tackle specific projects from your backlog?

• Is the team burnt out from ad-hoc requests and you want to minimize the number of tickets they handle?

• Do business teams see the data team as a bottleneck, and you want to accelerate time to insight?

• Are you trying to reduce “key person risk” or data bottlenecks where only a single analyst or small group knows how to pull certain data?

Depending on what the driving factors are in your case, you can define clear success criteria and then track against those. And you’ll avoid scenarios like we discussed above where you’ll end up with disappointed stakeholders due to misaligned expectations.

Deciding what tasks should be self-service

Business stakeholders have a few core “Jobs To Be Done” that they traditionally need the data team’s help with; these map to the four types of analytics that people commonly distinguish:

Some of these “jobs” are easier for stakeholders to handle fully autonomously than others.

For example, with the right resources in place, doing a simple investigation to understand the root cause of a metric movement is realistic. Expecting business stakeholders to create their own sophisticated forecasts or design experiments — not so much.

At the same time, the benefits you can expect differ by analytics “job”. The more frequent an analytics task is, the larger the benefit from opening up self-service: You resolve more bottlenecks, and the data team wins back more time.

Combining these two dimensions, we get the sweet spot for self-service analytics: Tasks that need to be handled frequently but have limited complexity.

Deciding the degree of self-service appropriate for each business area or team

Not all teams benefit from data democratization equally. Ultimately, it comes down to a trade-off between the expected benefits on one hand and risks on the other hand.

Whether the benefits outweigh the potential adverse consequences depends on a few factors:

🏎️ Speed vs. accuracy

Self-service is a better fit for teams or business areas where speed is more important than perfect accuracy.

Case in point: During my time at Uber, operations teams would handle all of their day-to-day analytics themselves. They needed to react to changes in the business in real-time, and directional accuracy was usually sufficient. The policy team, on the other hand, relied exclusively on a team of dedicated data scientists since all of their analyses went to regulators or the press

🤯 Data complexity

Not all data are created equal; for example, datasets can vary meaningfully in size (e.g. billions of rows in B2C ad impressions data vs. thousands in B2B customer data) or complexity (e.g. complex geospatial data in ridesharing vs. e-commerce orders).

While you can simplify things in many cases, there is a limit to what users without formal data training can reliably handle

🔐 Data sensitivity

The biggest advantage of data democratization can also become the biggest potential risk, if not addressed proactively: The number of people with access to data increases, resulting in a higher risk of leaks or misuse.

It’s key to manage this risk by setting the right permissions; for example, access to employee data should likely be limited to select teams or even individuals, and this should be strictly enforced and monitored through processes and tooling

📊 Analytics / technical skills

Some non-data teams have higher levels of pre-existing analytical and technical skills, making it easier for them to adjust to self-service analytics.

For example, engineering teams are familiar with database schemas and writing code, making it easy to get used to SQL.And teams like finance or growth marketing already work with data daily (e.g. in spreadsheets), giving them a solid foundation to build on

🙋‍♀️ Push vs. pull

Self-service is much more likely to succeed if teams are asking for increased data access rather than leadership or the data team deciding they should be handling more analytics work themselves.

Defining the overall self-service approach

Now, let’s put this together into a unified plan. When we combine our vision for 1) which tasks and 2) which business areas or teams we want to prioritize for self-service, we get a matrix like this that lays out our approach:

Being explicit about the scope of your self-service effort like this has a few key benefits:

1. Clear expectations: Roles and responsibilities are agreed-upon upfront, and it’s easy to track whether you’re making progress towards your goal

2. Planning & focus: You know what level of resourcing you’ll need to support the roll-out and you can focus on the right enablement efforts to make it successful (we’ll dig into that in the next sections)

2️⃣ Providing safe, frictionless data access

If you want people to actually take advantage of self-service, you need to make it easy for business stakeholders to 1) find data and 2) access it.

Making it easy to find data

At a small 5-person startup, finding data is as easy as walking over to your co-worker’s desk and asking them. But as your company grows and the number of “BI creators” increases due to data democratization, you need a systematic way to create visibility into analytics resources.

If users are unaware and recreate something that already exists, it doesn’t just waste valuable time, but also increases confusion — now there’s another dashboard to choose from.

This problem can be addressed in three layers:

1. Reducing data fragmentation: If data lives in fewer places, it’s easier to find (and keep consistent). Streamline your data stack and encourage users to stick to one tool for a particular use case, if possible. For example, except for ad-hoc reports, keep all major GTM dashboards in the BI tool instead of Salesforce; this also makes it easier to look at your pipeline data alongside things like web traffic or customer support requests that likely live outside of your CRM but are relevant to the full picture

2. Making data discoverable: As a startup, it’s usually unrealistic to maintain a comprehensive and perfectly up-to-date data catalog. However, even a simple repository with links to the most important dashboards is a helpful start. Over time, this can then evolve into a more robust resource with detailed metadata etc., either homegrown (e.g. Airbnb and Uber built their own catalogs) or by leveraging a dedicated tool

3. Publishing / highlighting vetted resources: Users need to know which resources are the most reliable as they have been created or vetted by the data team. You can implement this through a formal mechanism in your BI tool, or in a hacky fashion (e.g. color-coding in a spreadsheet)

Making it easy to access data

Once biz stakeholders know where to find data, the next step is making it easy to actually access it and derive insights. The most important thing to keep in mind: Changing human behavior is hard.

That means if you want to maximize self-service participation, you need to meet business users where they are instead of forcing them to use completely new tools or workflows.

Every bit of friction you add to the process, or new skill you require users to learn, increases the risk that they will give up and your data democratization effort fails.

That doesn’t mean that you should just look at your least technical stakeholder and tailor your approach to them. Instead, offer multiple ways to access data so that each group of stakeholders can pick the interface that works best for them. Depending on your data stack, these interfaces could be part of one or multiple tools.

A word of caution, though: Frictionless data access is great for speeding up time to insight, but without a thoughtful set-up and oversight, it will quickly lead to chaos and confusion. People will use the wrong fields or different metric definitions, and you’ll start drowning in Slack pings:

The only way to prevent this is to create resources and guardrails that remove ambiguity and reduce the mental load for users trying to get insights. As we’ll go through the different interfaces below, we’ll dive deeper into the specific things you can do.

Natural language (AI)

For many stakeholders, the most familiar way to request data and insights is natural language as that’s how they’re currently operating (creating tickets, sending Slack messages, or talking to data team members in-person).

Of course, in a self-service model, we need another entity that plays the data team’s former role: Translating natural language requests into SQL to pull data, and then translating the query output into insights for stakeholders.

LLMs can play this role well. They allow users to ask questions the way they would normally pose them to the data team, get answers almost instantly, and then ask follow-up questions or request summaries and visualizations.

However, for this to work reliably, the AI needs business context. Without it, similar to a human user, it doesn’t understand all of the company-specific nuances that weren’t in its training data.

For example:

• Which custom Salesforce fields to use under what circumstances

• Whether to distinguish countries based on the location of the customer, the sales rep, or the contract currency

• How to define an “active” user for a particular use case

This is why semantic layers have gotten a lot of attention in recent months: They provide a central source of truth that defines business logic, which can create a shared vocabulary for humans and AI. Semantic layers can either be part of your BI tool, or sit as a separate layer between the data warehouse and any front-end tools and applications that consume the data.

In the absence of a fully-fledged semantic layer, you should still document key metrics definitions and make this context accessible to the LLM. The more context you provide (e.g. concrete examples of when to use one metric over another), the better.

Unfortunately, even if you provide comprehensive context, LLMs can still be unpredictable. For example, most LLMs are not in the habit of asking clarifying questions by default, so if a user query is ambiguous, they might misinterpret it.

You can mitigate some of these issues by 1) providing custom instructions that promote desired behaviors and 2) closely monitoring response quality.

However, it's impossible to guarantee the exact answer a user will get, so it's important to remind your stakeholders to critically evaluate and sanity check outputs using the nuanced business context they have.

We'll dive more into this in the next section (“Educating business stakeholders”) below.

Drag & drop BI tools / dashboards

While natural language provides easy access, it doesn’t make other interfaces obsolete.

Many users prefer more direct control over the data they’re analyzing without necessarily wanting to write SQL, and sometimes, selecting a few filters or toggles in a graphical user interface is simply more efficient than describing exactly what you want.

For these users, a BI tool with a drag & drop interface is a good fit. However, there are a few key considerations:

• If users build dashboards from scratch and you don’t have a means of ensuring everyone is using the same “correct” metric definitions, you’ll likely end up with regular requests to the data team to reconcile inconsistent data between reports

• And even if definitions are correct initially: Once they change over time, they’ll have to be manually updated in every dashboard, which is difficult to stay on top of in a democratized data set-up with dozens of dashboard owners

Some of this can be addressed by building “vetted” dashboards for the most common use cases that allow users to explore the data via drilling and filters. However, this partially defeats the purpose of data democratization (removing the data team as the bottleneck) and you likely won’t be able to keep up with demand, so sooner or later you’ll find yourself looking for a more scalable solution.

Another issue to get ahead of: Many users will export data into spreadsheets at some point in their analysis since that’s the environment they’re most familiar with.

This is understandable, but contributes to more data fragmentation. Luckily, modern BI tools have responded by incorporating spreadsheet functionality and Excel-like formulas to provide a familiar environment for these users, reducing the incentive to move data out of the tool.

Keep this in mind as you’re choosing the tools for your self-service effort: The lower the barrier to entry, the more likely it is that users will actually use them; and the fewer tools are being used, the easier it is for you to put guardrails in place and monitor how things are going.

SQL

Lastly, some of your more technical users will want to write raw SQL.

Among the different ways to access data, this one has the largest room for user error and metrics inconsistencies, so addressing these risks proactively is especially important. But, for many power users it’s their preferred way to interact with data, so you’re best off accounting for it upfront.

You can do this in a number of ways, from hacky to more robust:

• Documenting data tables: Well-documented tables make it easier for users to understand which fields to use for what. However, they still need to figure out large parts for themselves (e.g. which filters to apply), leaving room for error

• Providing customizable best practice queries: Vetted queries can be a great starting point for SQL users of all skill levels (e.g. even SQL novices can get custom results by adjusting date ranges and commenting-in/commenting-out certain filters). However, these repositories often become difficult to maintain as definitions change over time

As your organization and the number of data users grow, this can lead to chaos and confusion. As a result, many data teams eventually decide to put more governance in place and create a metric source of truth — such as dbt, a semantic layer, or both.

Finally, opening up SQL to a broader user base comes with a risk of performance degradation and increased data warehouse bills. During my time at Uber, most SQL users didn’t have extensive formal data training. As a result, we’d regularly have instances where massive queries on raw data tables were running for hours before timing out while other ad-hoc queries were sitting in the queue.

For easier reference, here is an overview of the different ways we discussed that you can:

1. Help users discover data

2. Help users access data and make sense of it, and

3. Standardize definitions and provide context to ensure accurate and consistent results, both for AI and human users

This might seem daunting, but the good news is: You don’t have to document or model every metric in your business to start your democratization effort. In fact, you shouldn’t; otherwise you’d be spending weeks or even months just preparing without getting any business impact. Also, your data will constantly evolve — you’ll add new data sources, and metric definitions will change.

If you try to anticipate all of this and put a perfect foundation in place, you’ll never start.

Instead, based on your assessment of where each business unit falls on the Speed vs. Accuracy trade-off discussed above, you can start by focusing on the most critical use cases. Then, you can iteratively build out your repository of vetted resources or your metrics layer over time. Based on the feedback and data you collect from your early self-service users (e.g. the most common questions you get, fields that are queried the most etc.), you’ll be able to do a much better job than if you tried to tackle this all in advance.

3️⃣ Educating business stakeholders

Above, we covered the things that you — the data team — can do to facilitate frictionless and safe data access. However, the other piece of the equation are the users.

Now that they’re self-serving many of their data needs, your stakeholders need to know:

• How to structure an analysis or investigation

• How to get the most out of data tools

• How to QA and critically evaluate data

Structuring an analysis or investigation

If business stakeholders can’t get the answers they need, they’ll eventually lose motivation for self-service and start directing all of their questions at the data team again.

In my experience, the main reason this happens is when investigations lack structure and go in circles instead of producing actionable insights. Business users often didn’t have a chance to develop this skill-set since they used to outsource this work to the data team, so it’s up to you to provide guidance and share best practices.

The framework I’ve found most helpful for this is the Hypothesis Tree. It gives visibility into the various areas that need to be investigated and makes it easy to both track progress and ensure you don’t forget anything.

How this works is best illustrated with an example: Let’s say a business user looks at a dashboard and notices that ad revenue dropped by 9% week-over-week. To figure out what happened, we need to decompose our target metric and rule out potential root causes one-by-one

1. Decomposing the metric of interest: Ad revenue can be split into Ad impressions and CPM (price x volume); as a next step, Impressions can then be decomposed into Time spent and Ad impressions / hour, and so on

2. Segmenting the data: Cutting the data by various dimensions like geography, product lines, user demographics etc. allows us to see if the decrease is isolated to a particular part of the business

Once this structure is laid out, it’s easy for the user to translate it into the toggles and filters they need to apply to a dashboard, or the questions they need to ask the AI assistant in the BI tool.

Note: Many people take a while to really internalize this type of approach. LLMs are a great way to speed things up and help users practice hypothesis generation. Here is an example using ChatGPT:

Lastly, many investigations repeat over and over again. To avoid reinventing the wheel every single time, it’s helpful in my experience to keep a repository of past investigations that people can reference to quickly identify “the usual suspects”.

To start, this can be as simple as a spreadsheet that links out to old investigation docs. Then, if a certain type of investigation keeps happening and you think it’s worth the effort, you can flesh this out into more robust investigation playbooks.

How to get the most out of data tools

Most of us know the data tools we work with like the back of our hands because we work with them on a daily basis. Most of your self-service users will have much less experience, though.

Office hours or Slack channels can be helpful to handle any questions that come up as users are digging in. However, this likely isn’t enough. It’s difficult to know what you don’t know, so users are almost guaranteed to miss out on some features simply because they didn’t even realize they existed.

This is where it pays off to be proactive. First, conducting hands-on live training sessions (and recording them for later reference) goes a long way. Documentation often focuses on functionality and users tend to skim over it; but during a training session, you can show a real self-serve use case end-to-end which makes things much more tangible and shows the value they can get out of the tool.

And if your BI tool has user analytics, it can help you judge how the self-service roll-out is going: You can see who’s logging in and who isn’t, what features are being used, etc.

Based on this data, there are several things you can do. For example:

• Identify power users and leverage them as champions (e.g. document and share their best practices, have them join office hours or training sessions and share their insights, etc.)

• Flag users that aren’t using the tool, or where usage is dropping off, so you can reach out and help them get over any hurdles to adoption

• Understand which features users aren’t taking advantage of, and conducting training sessions on those specifically

How to critically evaluate data

Despite your best efforts in terms of documentation and data modeling, you won’t be able to make things foolproof. Users might still make mistakes, and AI assistants can give inaccurate answers — especially when they don’t have the full context of your business or the question being asked.

As a result, it’s important for business users to know how to critically evaluate the data they’re working with.

At a (very) high level, this involves two steps:

Step 1: Assessing the inputs

It’s unrealistic to expect everyone to apply the same rigor as data engineers or scientists when it comes to QA of a dataset. However, it still makes sense to encourage them to understand what the source data looks like before diving into the analysis.

For example: Let’s say a user wants to do an analysis on funnel metrics for different marketing channels. Before creating the desired summaries and visualizations, they should, at a minimum, do basic checks like:

• Is there an unexpectedly high number of NULL values in the data? This might suggest a data issue that needs to be investigated

• Is there a small set of standardized values for lead sources, or will they need to clean and group them first?

• What are the different opportunity stages, and which ones are relevant for the analysis?

This is a great use case for AI if your BI tool has this capability:

Step 2: Sanity checking outputs

Checking inputs to an analysis is important (we all know: “garbage in -> garbage out”), but it won’t guarantee that your users are getting the “right” result — especially if AI is involved.

Before they share the data more broadly or make decisions based on it, they need to pressure test the output. Here are two best practices that are easy to apply and can catch the majority of issues:

Check #1: Compare results to known sources of truth

Even if there is no vetted resource showing the exact data cut a user is working with, it’s often still possible to make a comparison. For example, if you’re analyzing specific groups of leads, you can check if the total number of leads matches existing dashboards and reports. If it doesn’t, it’s a strong signal that the more granular data you’re looking at can’t be trusted either.

Check #2: Benchmarking / gut checking

Sometimes, there isn’t a source of truth you can leverage at all. However, you can still figure out if you’re in the right ballpark.

For example, you can apply gut checks:

• Is the number of Daily Active Users you pulled at least two to three times smaller than the number of Monthly Active Users you looked at last week?

• Does your user base by country reflect the relative size of those markets?

In addition, you can check with subject matter experts on your team. Nobody ever has all the relevant context; and sometimes, when you’re in the weeds of an analysis, it’s easy to miss the forest for the trees. Teammates will often be able to provide a sanity check with a fresh set of eyes or give additional context that might explain a surprising metric movement.

Alternatively, you can use external benchmarks; a quick Google Search (or ChatGPT query) will provide averages and ranges for the most common metrics like conversion rates, margins, or efficiency ratios.

4️⃣ Expanding data democratization iteratively

With all your pieces in place, you’re ready to get going. But: Regardless of how much excitement there is for self-service, it’s usually not a good idea to roll it out broadly across the company at the same time.

There will be friction points and challenges to work through, and a large-scale roll-out will make it difficult to tackle these before they spiral out of control. It’s easier to ask a team to wait a few more weeks for access than to take it away after they got used to it:

So instead of an instant company-wide roll-out, pick one or two teams for a pilot. In addition to being good candidates for data democratization based on the criteria covered earlier in this post, these teams should check a few other boxes as well:

1. They have at least one analyst profile on their team: Having a decentralized champion that can answer some of the team’s questions and promote adherence to best practices will make your life much easier

2. It’s a small team: Interfacing with a limited number of stakeholders means you can give personalized attention to everyone and can respond to people’s issues more quickly

3. They have a high volume of ad-hoc requests / questions: The goal of the pilot is to get a lot of learnings in a short amount of time. The more the teams (want to) use data, and the more varied their use cases are, the more valuable insights you’ll generate

Speaking of monitoring: The pilot is your opportunity to monitor how users engage with data and the issues they encounter, and then iteratively improve the experience. For example:

• Monitor AI interactions and the quality of responses and add missing context for common issues

• Track the most common questions posed to AI and the most common query use cases so that you can create a source-of-truth dashboard for them down the road

• Observe which fields are queried most frequently and prioritize these for metrics standardization (e.g. via your semantic layer)

Then, once you feel like you have enough data on usage patterns and the biggest issues are ironed out, you can slowly open up access to more teams.

TL;DR

Self-service analytics can be a win-win for the data team and business stakeholders.

But for it to work, you need to implement it thoughtfully and treat it as an evolution of the cross-functional partnership rather than an opportunity to remove the data team from the equation.

At a minimum, you should:

• Clearly define the vision for self-service: What is your motivation and ultimate goal? For what use cases across which teams do you plan to implement self-service access?

• Offer easy data access: Don’t forget that your users come from different backgrounds and skillsets; it’s your job to meet them where they are by offering various ways for everyone to get the answers they need

• Put governance in place to prevent metrics chaos: To enable your stakeholders to get answers and minimize confusion, you need to create data foundations that make standardized metrics available to everyone (did I mention semantic layers? 😉)

• Educate your stakeholders: Business users need support to work through analytics questions end-to-end, from structuring analyses and sanity-checking data to navigating the various BI tools

This might seem like a lot of upfront effort at first glance, but remember: You don’t have to launch self-service for all teams and use cases at once, or have perfect governance in place to get going. Start with a small pilot, test what works, and then gradually roll things out more broadly.

Made with 🩷 by the folks at Omni

Data either wasn’t available, or it was impossible to surface on time for most use cases.

Nowadays, many teams face the opposite problem: They have data on everything — but they still lack clarity. With dozens of dashboards and hundreds of metrics, it’s hard to tell what truly matters, and important insights get lost in the noise.

Teams don’t know what to optimize for, and leaders don’t know which metrics to trust.

Metrics are only valuable if they give new insights or allow people to make better decisions. As data teams, it’s our job to enable this and help the business separate signal from noise.

In this context, less is often more.

But what metrics should you be tracking at the team or leadership level? And how do you design them in a way that makes them actionable and trustworthy?

In this post, we’ll walk through how to:

• Avoid the most common mistakes companies make

• Select the right metrics to track — and make them actionable

• Help your biz stakeholders choose the right targets for their teams

In addition, we’ll also discuss what you can do if you’re already much further along in the metrics journey and want to clean up rather than build a metrics framework from scratch.

The most common mistakes companies make in tracking metrics

Before we talk about how to set up your metrics the right way, let’s talk about what not to do. This will help us choose a nuanced approach that avoids the biggest issues with popular metrics frameworks.

Failure mode #1: Lack of prioritization

TL;DR: If you track everything, you focus on nothing.

Many companies and teams measure everything simply because they can. The data is available, so why not put it in a dashboard? It might come in handy at some point 🤷

This creates noise and distracts teams from their core priorities.

It’s also approaching the problem the wrong way around: You shouldn’t start by tracking everything and then try to figure out how to derive value from it. Instead, try to understand how your business works, what data business users need to get actionable insights and make decisions, and track that.

“Real estate” in dashboards is limited, and your stakeholders will only be able to stay on top of a limited number of metrics. In other words, metrics are competing for attention, and you should focus on the most relevant ones.

We'll cover how to identify these below.

Failure mode #2: Oversimplification

TL;DR: One single metric never paints the full picture.

As a response to failure mode #1, a lot of companies drastically simplify things to focus on one single North Star metric. The North Star metric (or NSM) is, per Sean Ellis, the “single metric that best captures the core value you deliver to your customers” (we’ll talk more about this concept below).

While this can be helpful to focus teams across the company, optimizing for one single target metric while “ignoring” everything else can create massive problems.

Let’s take a music streaming platform like Spotify as an example: The NSM “Time spent listening to music” might tell us whether users are finding value in the app; but are they converting to premium users and creating value for the business as a result?

The same applies to every other company as well. Businesses are simply too complex to capture all relevant dynamics in one metric.

We’ll dig into how to address this challenge in the next section.

Failure mode #3: Siloed metrics

TL;DR: Individuals’ and teams’ goals should ladder up to company priorities.

In many organizations, managers and teams decide what to track in a purely bottoms-up fashion. They brainstorm initiatives to prioritize, and then pick metrics to measure the impact of those initiatives.

Alternatively, the metrics aren’t directly tied to the work at all — they’re just what the team has always been monitoring.

This creates a massive problem: Even if the team hits its targets, it’s not clear whether that will help the company achieve its overall goals. And in more extreme cases, teams might even be working “against” each other, optimizing for competing priorities.

If two teams are pulling in opposite directions, the company loses.

How to select the metrics your company should be tracking

Now, armed with an understanding of what not to do, let’s design a more thoughtful approach to metrics.

To make this tangible, we’ll walk through an in-depth example for a video platform like YouTube 👇

To start, despite the drawbacks we discussed above, we still need a clear goal to rally teams around, and the North Star metric is a helpful tool for that. We just need to make sure we embed it into a framework that addresses its shortcomings.

🌟 Step 1: Choosing your North Star metric

Goal: Select one customer-centric metric to ensure all teams are focusing on a common goal.

Let’s talk about how to choose a good North Star metric. A NSM should:

• Measure value created for customers

• Correlate with business outcomes (e.g. revenue)

• Remain constant over long periods of time (to provide stable long-term guidance for the company to organize around)

So, in the case of our streaming platform, what could a good NSM look like? To figure this out, let’s choose a few candidates and rule out the ones that don’t work:

Option 1: # of videos on site

• ✅️ Pros: Content is at the core of a video platform. The more videos there are, the more likely users are to find something of interest and stay engaged

• ❌ Cons: This metric is too far removed from the outcome we’re trying to measure. You could have a site with millions of videos but zero users

Option 2: # of video views

• ✅️ Pros: This addresses the shortcoming of metric #1: By measuring how often videos are being viewed, we’re tracking users’ activity levels, not just available content

• ❌ Cons: We’re still not truly measuring if users find the content valuable; they could click on videos based on the thumbnail and then abandon them after a few seconds. In addition, long-form content would make this number look artificially small compared to short-form videos

Option 3: Video watch time (hours)

• ✅️ Pros: Addresses both issues with metric #2; in addition, video watch time is correlated with revenue (the more users watch, the more ads we can insert)

• ❌ Cons: The connection to revenue is still indirect at best, and the metric doesn’t tell us whether users are finding long-term value (e.g. even with terrible retention, we could grow this metric by acquiring lots of new users)

While not perfect, option #3 (Video watch time) is good enough to proceed. In the next steps, we’ll build a metrics framework around it to address its remaining shortcomings.

Note: No NSM is perfect. No matter what metric you choose, there will be blind spots. The most important thing is to be aware of them so you can put countermeasures in place.

Here are some examples from well-known companies 👇

🚧 Step 2: Pairing the North Star metric with a guardrail metric

Goal: Select a metric to pair with the NSM to address the biggest blind spot.

Now, with our NSM in place, we need to ensure that we don’t optimize it in an unsustainable way (failure mode #2 discussed above). The most straightforward way to do this goes back all the way to management guru Andy Grove, who allegedly said1:

“For every metric, there should be another ‘paired’ metric that addresses adverse consequences of the first metric.”

The biggest blind spot of our chosen NSM (Video watch time) is the lack of a direct link to monetization. For example, we might find ourselves in a situation where the easiest way to grow watch time is to lean into short-form video (like YouTube’s Shorts product).

However, these clips are too short for most ad types and may negatively impact monetization. Or, we might find that most of the upside for user growth is in APAC; but CPMs for ads are 10x lower there than in the US, diluting our overall monetization efficiency.

So to ensure we’re growing sustainably, we should pair our NSM with a monetization metric like “Revenue per hour watch time”:

At this point, even if the example makes sense to you, you might wonder: How do you generalize this and apply it to your own company?

To help you with this and pick the right guardrail metric for your NSM, we’ve compiled a few common combinations:

⚙️ Step 3: Derive input metrics

Goal: Understand the drivers of the NSM to inform team goals.

The North Star metric represents an output. This is important as it tells us whether what we did had the desired impact. However, it doesn’t tell us what we need to do.

As a result, the NSM itself isn’t a particularly helpful goal for teams. Similar to the north star in the sky, it provides a high-level direction for teams; but in order for it to be actionable, we need to create a more detailed “map” teams can navigate — in other words, an overview of the levers each team can pull to improve it.

This means we need to decompose the NSM into its drivers:

As a general rule: The higher you are in your driver tree (i.e. closer to the NSM), the more likely it is that a metric is affected by multiple teams. The lower you go (i.e. further away from the NSM), the more likely it is that an individual team — or even person — is responsible for driving it.

For example, the metric “# of sessions” is influenced by many different things. But if you continue decomposing it even further, you’ll eventually reach metrics such as the number of sessions generated from certain levers like push notifications, which will likely have a clear owner.

Note: The quantitative decomposition of the NSM is just one dimension. In addition, you should outline what product features or work streams affect a certain metric.

The relationship will not always be strictly mathematical, but it can help you (and your business stakeholders) understand what additional inputs should be measured:

For example, while the driver tree tells you that things like notifications are important for driving incremental sessions, there are other important opportunities that you might miss if you don’t take a more holistic view (e.g. bringing popular creators onto the platform and improving content variety are key to growing sessions in the long term).

❤️ Step 4: Defining business health KPIs

Goal: Select a few topline KPIs that measure business health holistically.

Even with the NSM, its drivers, and the guardrail, we still don’t have a comprehensive view of how our business is doing.

As a result, you’ll want to define a handful of additional KPIs that tell you how efficiently your business is operating, whether users are sticking around, etc.. In the case of our video platform, these metrics could plug that gap:

• Gross profit margin: In order to attract high-quality content to the platform, we need to share revenue with creators. Monitoring gross profit margins ensures we strike a healthy balance between investing in our content ecosystem and operating profitably

• CAC payback period: A user acquisition efficiency metric like our CAC payback period (“How long does it take us to earn back the customer acquisition cost?”) will tell us whether our acquisition strategies are sustainable

• Retention: We can’t run a sustainable business if all our engagement growth comes from costly initiatives to acquire new users instead of retaining existing ones. Cohort-based retention curves will tell us whether we’re doing a good job creating continued value for users

To be clear: We still want every initiative in the company to be aimed at improving the North Star metric — that’s the whole point of having one in the first place.

The additional metrics (the guardrail metric and the business health KPIs) are there to ensure we don’t have tunnel vision and are growing the business in a sustainable, profitable way.

How to help your business stakeholders select the right metrics for their teams

Laying out the relevant metrics is the first step; next, as a thought partner to your business stakeholders, you need to help them figure out which metrics should be used as targets for their teams.

This is an iterative process: As you’re working with your stakeholders to select the right metrics, you might notice that there are challenges with the set-up of your driver tree or the metric definitions you chose, and you might have to go back and revise them.

In the next sections, we’re laying out a few guiding principles to help you with this:

Connecting metrics to the North Star

Team-level target metrics should ladder up to the NSM; this ensures that everyone is working towards a shared goal.

As discussed before, that link doesn’t always have to be so direct that you can quantify exactly how an improvement in your metric will improve the NSM. But at the very least, there should be a clear logical connection.

For example, in the case of an engineering team working on an internal tool that recommends user signals for push notifications to the Growth team, you could set a target with regards to the number of signals surfaced. While not a direct part of the NSM driver tree, there is a concrete logical link:

More signals surfaced → more relevant push notifications → more sessions

If that’s not possible because a team is too far removed from these metrics, it can be a good option for this team to share the target metric of a downstream internal customer.

For example, a BizOps team helping Marketing to leverage AI to speed up the creation of new ad campaigns could take on a shared goal regarding the number of campaigns launched or signups generated.

It can also be a good idea to combine options 1 and 2; we’ll dive deeper into this in the next section below.

Combining inputs and outputs

Similar to how we set a guardrail metric for our NSM to get a more holistic picture and avoid potential adverse consequences, it’s often a good idea to goal teams on both the inputs and outputs of their work.

• Output metrics help ensure that the team’s work drives business impact

  • 🚨 However, outputs can be affected by factors outside an individual team’s control and usually take a while to react to new initiatives

• Input metrics are more responsive and under the immediate control of the team. Setting targets on these ensures teams feel empowered and have a proxy to monitor whether they’re on the right track

  • 🚨 However, this can lead to teams overoptimizing for work that appears productive but doesn’t actually move the needle for the business

Setting a goal for both ensures we drive the outcomes we actually care about while giving the teams a sense of control and a leading indicator.

For example, in the case of our video platform, the Creator Partnerships team focuses on convincing creators to move their content and audiences to us.

The inputs we could measure them on include:

• How many creator deals they sign (potentially weighted by the size of their audience)

• How many of those deals include exclusivity clauses (so we get unique content)

And so on.

However, ultimately we care that this work moves the needle on business goals. So in addition, we should set target metrics that measure outputs; for example:

• How many subscribers (new and existing) these creators attract on our platform, or

• How much watch time or advertising revenue the creators’ content generates

Considering cross-team interactions

Another key consideration is that target metrics for teams should not be chosen in isolation; otherwise, you risk friction and confusion.

There are two “flavors” of this:

1. Teams accidentally work “against” each other simply by trying to hit their respective goals

2. One team’s target is a direct input to another team’s goal, making it difficult to assess performance

Cross-team interaction #1: Teams accidentally work against each other

Let’s go back to our example from the beginning. We set the guardrail metric for the NSM (“Revenue per hour watch time”) to avoid scenarios where watch time is grown at the expense of monetization efficiency.

The challenge is that in practice, these metrics would likely be owned by different teams: One team focuses on driving more sessions and watch time, another handles the ads. In addition to having the right metrics in place on the company level, we need to make sure they are operationalized effectively on the team level.

Here are two options to address this:

• Give the first team a “budget” with regards to how much their efforts can impact monetization (and track the actual impact via a holdout group)

• Pick target metrics that inherently account for this tension. For example, the engagement-focused team could take on goals by region so that they’re incentivized to grow more mature — but profitable — markets like the US as well instead of focusing purely on the regions with the largest upside for user growth

Cross-team interaction #2: One team’s target is a direct input to another team’s goal

Imagine you have two teams in your company working on related areas:

• The first team is working on initiatives to get users to come to the platform more often and is goaled on the “number of sessions”

• The second team is trying to grow the time users spend watching videos and is goaled on “total watch time”

Now, let’s say Team #2 ends the quarter with a 12% increase in “total watch time”, compared to their goal of 10%. They knocked it out of the park, right?

Unfortunately, things are not that simple. The metric “Total watch time” is driven by 1) the number of sessions and 2) the time users spend per session.

In other words, the work of Team #1 heavily affects the target of Team #2, and it’s even possible that they only hit their goal because of this (while their own initiatives didn’t drive any watch time increases at all).

A general rule that can help you avoid this: Whenever possible, different teams that work towards a common goal should have target metrics on the same level of the driver tree instead of being upstream or downstream from each other.

So in this case, Team #2 should have been goaled on “Watch time per session”; that way, any growth in total watch time can be attributed to a specific team’s work.

Keeping metrics simple and transparent

No matter which metrics you choose: They need to be easy to calculate and understand, or you’re bound to run into issues sooner or later.

If teams don’t understand how a metric is calculated, they won’t be able to manage it well.

• How do you respond to metric movements if you’re not sure what might have caused them?

• How can you plan impactful initiatives that improve your target metric if you don’t understand the math behind it?

This happens a lot with complex compound metrics like LTV : CAC (where the numerator and denominator both consist of other metrics with their own opaque definitions) or internally-developed metrics that nobody understands besides the analyst who invented them.

In addition, metrics often become more complex over time as teams try to address perceived shortcomings of the initial definition.

For example, if you try to calculate win rates for B2B deals or retention rates for users, you’ll have to wait a long time until your cohorts have fully matured. But if you build a forecast of cohort performance into the metrics instead, wouldn’t you be able to get answers immediately?

Unfortunately, in practice, this often causes more issues than it solves. Simplicity beats (false) accuracy almost every time.

Data teams don’t show their value by acting as gatekeepers for complicated, opaque metrics — they do it by making it as easy as possible for the business to extract insights from data.

Taking a step back: How do you clean up your metrics if you’re already far along in the journey?

As you’re reading this post, you might be thinking:

That’s a fair point — the tips above are easiest to implement if you don’t have a messy status quo to wrangle with. So what can you do in that case?

Under these circumstances, the goal should be to identify the set of metrics that truly matter. Not all metrics that you’re tracking will be direct inputs to decision-making or even be regularly reviewed. In other words: You need to separate signal from noise.

If your company has an official North Star, you can work backwards from there and, similar to what we covered above, see which metrics are direct inputs to it.

In the absence of a North Star, I recommend starting with the metrics that are tied to official commitments:

• 💼 Board metrics: What metrics are reported to the Board of Directors and investors? What are their direct inputs and drivers?

• 🎯 Team goals: What metrics are teams goaled on? Which metrics tie directly to performance ratings or monetary incentives of individuals (e.g. quota-carrying sales reps, customer success managers, support reps etc.)?

These metrics are more difficult to change in the short term, so they represent the “core” of your set of metrics for the time being. In addition, you’ll want to understand the operational side of things:

• 📊 Data usage: Who are the main data consumers across the company? Which dashboards and metrics are they accessing the most?

• ⚙️ Management: What metrics are team leads looking at to make decisions and run their part of the business? What is their mental model of how these metrics connect to company goals?

This gives you a good initial understanding of which metrics are likely essential, and which ones might be expendable:

The eventual goal should be to integrate the metrics into a holistic framework like we covered earlier. But that’s a lofty goal; to make it more manageable, here are some concrete things you can do to start moving in that direction:

• North Star: Even if your company doesn’t have an official North Star, it might be clear from your assessment and conversations above that there is an implicit one. Creating visibility around this shared goal is a good next step

• Team goals: Are there any improvements to goal metrics you can propose during the next planning cycle based on the best practices discussed above?

• Phase out less essential metrics: The best way to see if something adds value is to see if people miss it when it’s gone. For this reason, Shopify deletes all recurring group meetings once a year. You don’t have to go quite that far; however, I’ve found it helpful to move metrics that you suspect are less essential out of the main tabs of dashboards or into the appendix of slide decks (and let your stakeholders know). More often than not, nobody will complain

• Plugging gaps: Even if you’re trying to reduce the number of metrics you’re tracking: If there are obvious gaps (e.g. no guardrail for the North Star), it’s important to propose a metric to plug them. The goal isn’t to have fewer metrics per se; it’s to remove those that distract from the essential ones

Recap: Creating a holistic metrics framework for your organization

Phew, that was a lot 😮‍💨.

Building something that doesn’t just sound nice but actually works in practice requires a deep understanding of how the business operates and… lots of hard work.

But developing an intentional, holistic approach to metrics is worth it. It’s the foundation for everything:

• How your stakeholders look at the business, and the decisions they make

• Whether teams are working towards company priorities or are incentivized to do other things

• Whether dashboards, models, and experiments create clarity or just add more noise

Hopefully, the framework in this article gives you a helpful starting point.

Made with 🩷 by the folks at Omni

But over time, this often leads to data teams becoming “ticket takers”. Slowly, but surely, they get further and further removed from the business, and tickets become the main — and eventually the only — way they interact with stakeholders.

Instead of partnering with business teams to solve problems together, they are now fully reactive. With this, the goal shifts from driving business impact to reducing JIRA ticket backlog.

The good news is, this can be fixed — and in this article, we'll walk through how.

When the data team works as ticket takers, it creates several issues:

• Issue #1: The data team’s impact isn’t maximized:

  • 🔍 First, the data team is unable to help biz stakeholders proactively identify opportunities they might not have thought of

  • 🎯 Second, data leads are unable to prioritize and focus their resources on the highest-impact opportunities

• Issue #2: The business gets lower-quality analytics solutions since the data team is tackling individual tickets instead of solving a problem end-to-end. Business stakeholders start viewing the data team as a service desk for data pulls and are less likely to involve it when it comes to solving more strategic problems

• Issue #3: Data team members become demotivated. Handling a never-ending stream of tickets with no common strategic theme and unclear impact feels like Sisyphus pushing a boulder uphill

These issues all stem from the fact that in a ticket-taker model, business stakeholders assign tasks to the data team instead of asking them to solve problems.

In other words, the problem solving is done by the business teams in isolation, and they then approach the data team with what they think the solution should be.

This set-up is flawed: The biz team typically lacks technical understanding and the data team is missing business context, so neither is in a position to define the ideal solution to the problem.

To address these issues, a data team must evolve from a reactive ticket taker to a proactive thought partner.

But how do you get there?

1. Data team members need to have deep business context and empathy for the pain points of their stakeholders

2. There needs to be mutual trust and respect between data and business teams

3. The team needs to learn how to analyze and solve complex business problems

4. The data team needs to have a seat at the table

In this article, we will walk through how to succeed at each one of these points.

Requirement #1: Getting business context

In order to be a thought partner to the business, you need to understand:

1. The company’s business model and strategic priorities, as well as

2. What your business stakeholders do — and care about — on a daily basis

Understanding the business model & company goals

The company’s business model and goals provide important context for evaluating any opportunity that comes your way.

Example: Are more “supply hours” the solution?

For example, when I worked at Uber, there were teams in charge of growing the number of drivers on the platform, as well as the time they spend on the app (a metric called “supply hours”). Any analytics project that helps grow these “supply hours” might seem impactful at first glance. But once you consider how Uber’s marketplace works and what Uber is optimizing for as a company, things are a bit more nuanced.

As mentioned in our prior post on “How to make an impact in your first 90 days as a data leader”, a great way to understand the business model is to dig into the revenue equation and key drivers of the business. Let’s look at what that looks like in practice, and how it can help you become a better thought partner.

At a high level, Uber’s goal is to make money, and it does that by 1) growing the number of trips on the platform and 2) increasing the fare per trip:

The number of trips is affected by demand (how many people are requesting a ride) and by how many of these requested rides Uber can fulfill. That’s where the supply comes into play: More drivers spending more time on the platform = higher likelihood any given request can be matched.

But here’s the catch: In order for a match to happen, the driver needs to be 1) online at the same time as the request, and 2) roughly in the same area (nobody wants a 20 minute pickup).

That puts things into perspective. The business doesn’t simply need more supply hours; what it really needs is more supply in times and places where there isn’t enough.1

There are a couple of things you can do with this new context:

1. When evaluating opportunities, you know now what metrics to look at

2. You can potentially help the team build reports and predictive analytics that help them balance supply and demand better across time and space

Without digging into the mechanics of the business, you wouldn’t have been able to do either.

Now, let’s look at the second part of the revenue equation. The higher the fare per ride, the more money Uber will make.

This sounds obvious, but it has some profound implications. For example, this means that when prioritizing initiatives, you should pay special attention to high-fare products like airport or premium rides (e.g. Uber Black) even when growing the high-volume “mass market” products might be easier.

Getting domain knowledge & day-to-day business context

Even if you know what is impactful for the company, you can’t be a good thought partner to your stakeholders if you don’t understand what their teams do or the challenges they face.

Once you understand the status quo, you can:

1. Help improve it (make data more easily accessible, generate automated reports before important meetings, etc.)

2. Identify net new opportunities that can be unlocked through analytics and design custom solutions based on how the team operates

For example, you can only help the marketing team think through how an attribution model should work if you deeply understand the marketing funnel, the different types of campaigns, the tool stack, and the metrics the team is measured on.

You can get a good foundation by reviewing documentation. Internal wikis can give you an overview of how the team operates and what was done historically, while documents like roadmaps, RFCs (Request for Comment), PRDs (Product Requirements Documents) etc. are a great way to get up to speed on what the team is currently working on.

Once you have enough context, you can add additional questions and comments to these documents to start establishing yourself as a thought partner who is invested in learning more and providing proactive ideas.

However, if you want to be able to debate solutions as an equal, you need to integrate yourself and become an (informal) part of the team. That doesn’t mean that you have to officially report into that organization, but it means that you need to be in the flow of information and dedicate substantial time to learning the ins and outs.

Here are some concrete things you and your team can do:

• Sit close to the team: Even just spending one or a few days a week in their area of the office will expose you to a lot of valuable information through ad-hoc conversations etc. It will also simply help build stronger relationships with your business partners

• Ask to join select meetings: A lot of important discussions happen in meetings and never get documented. Joining weekly team meetings, standups, problem-solving sessions etc. can give you invaluable context you otherwise wouldn’t have access to.

  • Note: It’s hard to know in advance just how valuable a meeting will be; to avoid wasting time, pick the most promising one or two to start, join them for a few weeks, and then decide if you should 1) continue to attend, 2) try another meeting, or 3) claim your time back.

  • Also, if your company uses an AI note taker, you can ask to get access to a summary instead of attending, which can save you a lot of time

• Ask to be added to Slack channels & distribution lists: Most teams have Slack channels for group discussions and email distros for sharing updates and announcements. Make sure that you’re added to the ones of the teams you work most closely with so that you can stay in the loop

• Encourage your team to do coffee chats with their stakeholders: Most business teams will be hesitant to have standing 1:1s with their data stakeholders because their calendars are already packed. But almost nobody will say no to a coffee chat or lunch. Besides strengthening relationships, they are a great opportunity to learn about what’s on people’s minds

Requirement #2: Building trust

Becoming a thought partner to your business stakeholders is an evolution. In order for them to consider involving you in more strategic problem solving, you need to fulfill the basic level of your mandate first. For example, this means:

• Being highly responsive and flexible in responding to the needs of the business

• Doing a good job fielding their ad-hoc requests

• Reliably maintaining dashboards & data pipelines

In other words, becoming a strategic thought partner doesn’t mean that you’ll stop working on ad-hoc requests or recurring tasks; it just means that you start engaging with your stakeholders earlier in the process so that you can together scope better analytics solutions (and eliminate some of the low-impact tickets).

Trust is best earned in small increments. So instead of directly asking to be involved in planning discussions and the scoping of all new projects, I’ve found it more effective to gradually evolve the partnership into a more strategic one.

Concretely, that means going from purely executing what you’re asked to recognizing patterns and proactively offering solutions.

For example, if you are getting repeated tickets about reconciling the number of leads between Marketing and Sales queries, you could propose a standardized leads metric to your stakeholders.

If your stakeholders are on board, you can then offer to create a source-of-truth dashboard based on this metric that both teams can reference. That way, you:

1. Proactively identified a recurring problem

2. Found a quick fix to address it, and

3. Provided a self-service solution so that the teams can dig deeper into metrics changes on their own.

If you do this consistently, it shows your stakeholders that you are taking ownership of the problems you’re exposed to. As a result, they’ll be more likely to think of you when new problems come up and pull you in sooner.

Requirement #3: Developing problem-solving skills

For business teams, pulling the data team into planning and problem-solving sessions adds additional friction (since there are more cooks in the kitchen). So to justify that, your team needs to add value without slowing things down.

How do you do that? By analyzing problems and identifying pragmatic analytics solutions in a structured way.

To help you get started, we’ll walk through a framework that you can use to practice this with your team:

1️⃣ What is the true underlying problem?

If your team members have been operating as ticket takers, they will have had little direct exposure to business problems. As mentioned above, the requests stakeholders put into tickets tend to be tasks with limited business context.

Acting as a thought partner means that instead of following detailed instructions, your team is analyzing problems and tracing them back to the root cause to identify suitable analytics solutions. This is a (new) skill set your team needs to master.

The easiest way to do this is to keep asking “Why?”.

Let’s look at an example:

Let’s say your stakeholder asks you to build a customer health score…

• If you ask them why they need it, they might explain to you that they are trying to figure out which customers Customer Success Managers should schedule monthly check-ins with

• If you ask what the goal of these check-ins is, you might learn that the team is concerned about an increase in churn over the last few months and wants to take steps to address that trend

Armed with that understanding of the underlying business problem, you can now work with your stakeholders to define what analytics solution would be most impactful.

For example, before building any type of health score or predictive model, you could start with an investigation into the drivers of the recent churn increase to then come up with more targeted solutions based on what you find.

A customer health score and early warning system for churn might still be the right thing to build in the long term, but your investigation could reveal more immediate things you can take action on (and potentially with less resource investment).

2️⃣ How often does this problem occur?

When deciding on an analytics solution for a problem, there is usually a wide spectrum of things you could do.

By identifying how often the problem actually occurs, you can start to determine which solution may make the most sense. If you’re dealing with a one-off question from leadership, a simple ad-hoc data pull (and a slide summarizing the findings) might be sufficient.

If it’s an investigation the team is doing on a weekly basis, however, it makes sense to think about creating an automated and/or self-service solution so that you don’t have to repeat this task all the time.

The key is to not spend more time automating the task than you save from that automation. This xkcd cartoon is a good place to start when thinking about this:

3️⃣ How large is the expected business impact?

In addition to finding better solutions for analytics problems, your team can also help prioritize which problems would be the most impactful to tackle.

This is where opportunity sizing comes in. With this kind of exercise there’s no need for perfect accuracy, so a rough back-of-the-envelope estimation is usually sufficient. You just need to know if one opportunity is bigger or smaller than others, not exactly how big it is.

Besides keeping it simple, the most important thing is that you size all of them in a common “currency” so that you can compare them (e.g. “$ revenue growth” and “$ cost savings”).

Granted, this doesn’t work for every project, but many other impact metrics (e.g. conversion rate improvements, time savings etc.) can be translated into $.

So, how do you actually size an opportunity?

There isn’t a single “correct” way to do it, but here’s an example to give you an idea of what it could look like:

Let’s say you work in a B2B SaaS company and are considering doing work on the data pipelines that power automated email campaigns. The goal is to increase the daily volume of emails you can send to prospective customers by 30% to book more meetings for the sales team.

You might not have experimental data for this particular change, but chances are that email volume fluctuated substantially in the past. To get a rough idea of how incremental volume might convert to meetings booked, you can plot these data points and see if there is a meaningful relationship. If it’s linear enough, you can keep things simple and assume it’s linear for the purpose of this directional exercise:

Based on this simple approximation, you can estimate how your conversion rate will change and how many incremental meetings you can expect to book. You then just need to apply a few assumptions (win rate, average contract size) to get a revenue estimate and compare it against other initiatives.

4️⃣ What are the minimum requirements and nice-to-haves?

TL;DR: Done is often better than perfect.

A simple solution can often deliver 80% of the value to your stakeholders, and tackling the last 20% is not worth it due to diminishing returns. Your time is better spent bringing an additional project from 0 to 1.

What’s more, I have seen over and over again that stakeholders ask for a solution to a problem that’s pressing at the time, but when you deliver it, it gets very little usage because priorities have shifted.

One of the best ways to address this risk is to build a minimum viable product (MVP) to see if stakeholders will actually use it and find value in it, and then add more functionality or improve performance over time.

To do this, you need to understand what the non-negotiables are.

• What functionality do your stakeholders need at a minimum?

• Are there any thresholds in terms of accuracy/performance below which your solution won’t be useful for business teams?

5️⃣ How will this evolve as the company grows?

In fast-growing companies, most of what you build will need to be overhauled regularly or it will become obsolete.

For example, attribution models will break if you don’t account for new marketing programs and channels that are being launched, or dashboards will show wrong data when metric definitions change.

Acting as a thought partner to the business means anticipating this and building your analytics solutions so that it’s easy to adjust them to changing business realities.

Note: This is not about anticipating every future requirement and building those features. Instead, it’s about building a “good enough” solution for the moment and making it easy to go back and iterate on it.

For example:

• Using a data model instead of hard-coding metric definitions in dashboards allows you to define (and update) metrics once, so they populate consistently for everyone else

• Assuming any set of entities (countries you’re active in, currencies, marketing channels, cost centers etc.) will grow by up to 10x over time, and structuring your code accordingly

• Applying software development best practices like the use of Git / version control so it’s easy to evolve your data products over time and collaborate on them

6️⃣ Developing with your stakeholders

Even if you analyze the problem thoroughly, there’s a good chance you’ll build the wrong thing.

For example, your stakeholders might have communicated their requirements poorly, or you might have misunderstood what they want.

The best way to avoid this: Involve your biz stakeholders from the beginning. Here are concrete steps you can take:

• Start by shadowing your stakeholders’ workflows to understand the use case in detail. Ask them about their current pain points, data blind spots, and inefficiencies

• For larger projects (e.g. building a model), write a spec and get feedback before you proceed

• Mock up the user-facing components (e.g. a dashboard layout, or the output of a model that teams will receive) before you build the fully functional version. This way, you can test if it’s intuitive for non-technical users, and whether anything fundamental is missing or needs to be changed

  • Note: Current-generation GenAI models are good at mocking up UI components and can save you a ton of time — just make sure it’s actually possible to build what you mock up in the BI tool of your choice so you don’t overpromise to your stakeholders!

• As mentioned above, build a minimum viable product (MVP) and see if it gets used before you make it fancy

Requirement #4: Getting a seat at the table

Building mutual trust with your stakeholders and developing strong problem-solving skills is a requirement for being able to act as a thought partner, but it is not enough.

You also need to make tactical changes to how teams operate; in other words, you need to get a seat at the table for important work streams. If you’re not there, you can’t get context or provide input.

Let’s look at quarterly planning as an example, as that’s where a lot of analytics requests come up.

Most companies’ planning processes are dysfunctional. I have seen them at startups, growth stage companies, and FAANG, and there is always a lot of friction. But even though it will never be perfect, you need to make sure you’re not setting yourself up to fail from the start.

Let’s say your company is a B2B SaaS business. Here’s what planning looks like for a “ticket-taker” vs. “thought-partner” team:

Planning as a ticket-taker team

If your team is perceived as a “ticket-taker” team, here is how planning typically plays out:

• Step 1: The business teams brainstorm analytics requests and assign them a priority

• Step 2: They shoot that list over to you, the data team

• Step 3: Based on the limited information you have, you do your best to estimate the effort for each request and indicate which ones you can resource

The result looks something like this:

In this example, you committed to resourcing the two projects with priority “P0” (both of them are estimated to be high effort). This means:

1. For the Customer Support team, you’ll build a model that predicts resolution time for incoming support tickets

2. For Sales, you’ll build a model that produces a running forecast of the number of leads and opportunities by the end of the current month

Based on your estimate, you didn’t have enough bandwidth to tackle any P1 projects.

The problem: You weren’t involved in coming up with the projects. You only reacted to the list and assessed whether you can resource the requests, not whether they are worth doing.

There is a high risk that the resulting analytics roadmap is not the ideal way to spend the team’s time (and isn’t maximizing business impact).

Now, let’s look at the same process with a data team acting as a thought partner.

Planning as a thought-partner team

A team acting as a strategic thought partner needs to be pulled in earlier. If that doesn’t happen automatically, you need to push for a change in the process. The more you have already shown the value you can add, the easier this should be.

Armed with your deep understanding of the business, you should dig into the pain points the teams have before discussing any potential projects.

Sticking with the same examples from above, it would look like this:

Pain point #1: The Customer Support team wants to make better staffing and allocation decisions

The ticket backlog is growing and unevenly distributed across agents. That’s why they’re asking if you can build a model to predict ticket resolution times so that they can optimize routing and bandwidth management.

But when you do a simple analysis of the status quo, you realize that there are much more pressing issues. The Support team currently lacks a good understanding of what ticket volume to expect throughout the week, resulting in poor shift planning.

Judging from historical data, this should be fairly easy to predict, and a quick sizing exercise shows that it would be more impactful in reducing backlogs than a model that optimizes ticket routing.

➡️ By working with the Customer Support team to properly understand their pain point, you uncovered a higher-impact opportunity your team can solve.

Pain point #2: The Sales team wants to understand how many leads and opportunities they’ll end up with by the end of a given month

They are asking you whether you can train a ML model for this purpose.

You’re considering it, but when you take a quick look at past months’ data, you notice that a simple linear extrapolation would have landed, on average, at over 90% accuracy. If you apply seasonality for weekends etc., it would be even more accurate.

When you check with the team, you learn that this is sufficient for their use case. Great — you can deliver the simple linear extrapolation in a few days, before the quarter even begins.

➡️ The Sales team overestimated what was required to solve their problem. By scoping the solution together with them, you were able to find a simple fix and avoided spending weeks on a model that wasn’t needed.

So what changed by acting as a thought partner? Two things:

1. You avoided building the wrong thing for the Support team

2. The deliverable for the Sales team could be substantially reduced in scope

As a result, you can now resource an additional deliverable:

Checklist: Operating as a thought partner instead of a ticket taker

Even after reading this post, “acting as a thought partner” can seem like a somewhat vague concept.

• Are you maybe doing it already?

• Where should you consider adjusting your approach?

For reference, here’s the checklist from beginning again:

It’s not a binary distinction. As discussed earlier, going from a “ticket taker” setup to being a thought partner to the business is a gradual evolution.

Hopefully, this guide provides you some ideas on where to start.

Made with 🩷 by the folks at Omni

Subscribe now

Starting a new job as a data leader is hard.

First, you’re often inheriting a messy situation. The data infrastructure is held together by duct tape as the data team (if there even was one) is struggling to keep up with the pace of the business.

Chances are you’re also underresourced. You might even be starting as a team of one and have to fight for headcount and budget in an environment where CFOs are scrutinizing every incremental dollar.

Despite all of this, expectations are usually sky-high. Everyone hopes the new data leader can solve their most pressing problems — ideally, immediately. To top it off, you likely don’t have much time to prove yourself: The average tenure for Heads of Data is only 30 months. Time to impact is key.

Since it can be difficult to know where to even start, we put together this guide. If you’re starting a new job as Head of Data — whether you’ll manage a team or come in as an IC — it will give you a comprehensive framework for surviving your first 90 days, creating measurable value, and setting yourself up for long-term success.

A framework for success

What do you need to be successful in a new data role? At a high level, four things:

1. A deep understanding of the status quo; this includes both the technical side as well as the business

2. Quick wins to show the value you can create

3. A robust long-term plan to set yourself up for success in the following 12 - 24 months

4. Trust and support from your business partners

Earning trust and support from your stakeholders depends on doing a good job on #1 - #3 (and takes time), so we’ll focus on those steps in this post.

The key points are the same whether you’re taking over an existing data team or if you’re the first proper data hire. However, we’ll get into specific considerations for each scenario and mark them accordingly:

• 💼 = sections relevant for those leading a data team

• 👩🏻‍💻 = sections relevant for ICs / teams of one

• 💼👩🏻‍💻 = sections relevant for both

While the specifics will depend on your organization, here’s roughly what your first 90 days should look like:

Now, let’s get into the specifics!

Phase 1: Analyzing the status quo

Timeframe: Intensive deep dive day 1 to day 45; afterwards continued learning

This is the foundation for everything else; without understanding the status quo, you can't figure out where you can drive impact. While it can be tempting to immediately go into “doing mode”, that’s a mistake.

Don’t expect to be able to just copy / paste what you did in your previous roles:

• Every company is different and requires bespoke solutions

• You will not get a lot of supporters if you aren’t showing an open mind to how your new company operates

As a result, the focus of your first few weeks should be on listening, listening, and then…listening some more. This will help you understand three core things:

1. The business

2. The data stack

3. The data team

Let’s dive in.

💼👩🏻‍💻 Understanding the business

Why is this important?

If you hope to make an impact with data, you need to deeply understand how the business works.

Without that context, you can’t be a strategic thought partner who helps your biz stakeholders identify opportunities and maximize impact. And more tactically, you can’t define metrics, design experiments, or run analyses that actually create business value.

Starting here also has some additional benefits:

• When you dive into the data stack, you can evaluate it in the context of the needs of the business

• To do this, you need to talk to stakeholders across the company, allowing you to start building relationships. You’ll need their trust and support if you want to get anything done later on

Since “understanding the business” can feel a bit daunting, here are the most important aspects you should focus on:

1. Understanding your company’s industry and business model

No two businesses are alike, especially if they operate in different industries or use different business models.

I learned this lesson the hard way when I moved into B2B SaaS after working on marketplaces and social media. Some fundamental insights and best practices were transferable, but many weren’t.

First, you need to understand the industry your company is operating in:

• What types of companies operate in the industry and adjacent ones (e.g. in the automotive sector, there are suppliers, OEMs, transportation networks etc.)?

• What customer segments exist; are they growing or shrinking?

• Who are the main competitors; how does your company compare?

To get an initial overview, you can read the quarterly reports of public competitors as well as industry reports from investment banks, VCs, or industry analysts.

I’d also recommend asking people in your company who the industry thought leaders are since this can be a great way to learn about up-and-comers who may not yet be on the radar of larger firms.

However, the real actionable insights come from going one level deeper. For example:

• Follow competitors on LinkedIn to stay on top of their marketing and product announcements, as well as how their audience does (or doesn’t) engage

• Review their websites, marketing materials, and support documentation to get an in-depth understanding of their functionality and positioning

Next, you need to understand your company’s business model. The best way I’ve found to do this is to map out the revenue equation and build a driver tree, which is simply the mathematical formula of how the business makes money.

For example, for a B2B SaaS business, it looks roughly like this:

Then, you can go deeper and break each component of the equation into its key drivers. E.g. if your company is Sales-led, you can break “New ARR” into a funnel from Leads to Won Deals.

This driver tree will give you a good idea of the metrics that matter, and where analytics projects can have the biggest impact on the top or bottom line of the business.

2. Understanding organizational structure and internal processes

Whether as a data team lead or an IC, you won’t get anything done without involving the rest of the business.

You’ll need to partner closely with your stakeholders to define priorities, plus you’ll need to get their buy-in in order for your work to get implemented and drive impact.

This means you need to know who does what. The org chart is a good place to start, but you’ll quickly find that you actually need to talk to people to understand the true dynamics.

• How do Marketing and Sales work together? Where are the typical friction points?

• Where do Strategic Finance and BizOps plug into the business?

You’ll also want to understand key processes in the company, such as:

• How does quarterly and annual planning work?

• How do you secure your headcount and tool budget?

• How do decisions get made?

Understanding these things is not just a formality; it will give you a foundation to set yourself apart.

For example, quarterly planning is usually a massive source of friction between teams. So if you learn about the planning cycles early and run a smooth process, you’ll leave a lasting impression.

3. Understanding your business stakeholders’ priorities & pain points

Once you understand who your stakeholders are, it’s time to figure out what’s on their minds. If you want to establish yourself as somebody who creates value quickly, you need to prioritize the most pressing issues the business is facing.

Questions you should ask include:

• What are their current priorities? What are their biggest pain points?

• How are they working with the data team (if one existed in the past)? What’s going well and what isn’t?

You can (and should) also ask them directly what you / the data team could help them with; just take it with a grain of salt. To be successful as a data leader, you need to translate business problems into analytics solutions rather than taking and implementing analytics requests verbatim from stakeholders.

When we’re providing customer support at Omni, we think of this as understanding the “why” behind the ask.

4. Company culture

While you should definitely use your advantage as a new hire to spot inefficiencies and things that don’t make sense, you’ll still need to adjust to the cultural norms and respect that the company has done (and will keep doing) things a certain way.

Here are a few key cultural factors you’ll want to understand:

Why this matters: If your new company moves fast and handles most things in a pragmatic fashion, you will likely face a lot of resistance if you want to implement a formal intake process for data requests that requires stakeholders to fill out lengthy JIRA tickets.

5. Metrics and data consumption

Most teams are held accountable to measurable goals, and you need to know the relevant metrics to understand what they care about.

Some key considerations:

• ⭐ Does the company have a Northstar Metric, and if so, what is it?

• 💼 What metrics are reported to the exec team?

• 🏛️ If the company is public, what’s being reported to Wall Street?

• 🎯 What metrics are teams and ICs goaled on?

Similarly, you’ll want to understand how people consume this information:

• Where do people go to see metrics (e.g. slide decks vs. dashboards)? What dashboards are being used the most, which ones not at all? Does everyone have access?

• How do people learn about dashboards (e.g. is there a repository)?

💼👩🏻‍💻 Understanding the data stack

Why is this important?

The choices your company made in its data stack determine what you can realistically do during your first couple of months and what friction you have to prepare for (e.g. missing or stale data, issues with consistency etc.).

Step 1: Mapping out the data stack and flow of data

Most companies, especially early-stage ones, don’t have proper documentation. To the degree they don’t exist, consider creating architecture diagrams and charts for how data flows through the various systems:

• What tools are being used? Who is the main point of contact for each?

• How is data being ingested, stored, and organized?

• Do they have a data model? Where / how is data modeling taking place?

It may take you months to get a full understanding of the entire data stack and code base (if you ever get there).

So to maximize the usefulness of this exercise, work backwards from what you learned when talking to business stakeholders. For example, what data pipelines are powering their key reports? In what layer of the stack are their core metrics defined?

Creating the documentation yourself has a few benefits:

• It will make your onboarding easier. You’ll be overloaded with information during your first weeks, and you will find yourself asking the same questions over and over again if you don’t create a summary

• It gives you a tangible artifact to share with others early on. Having something valuable with your name on it circulate will give you instant visibility

• You can get feedback and validate your understanding before you move forward. As a new hire, it’s easy to miss some of the nuances of the system that’s in place

• You’ll later be able to refer back to this to show how much you improved things compared to when you took over

Step 2: Identifying gaps and opportunities

Even if there wasn’t a proper data team before you arrived, someone was in charge of data. Find out who that was and ask them what the biggest pain points with the current data stack are.

In addition, you’ll want to do a simple audit yourself:

• Are any of the tools or architecture choices not a good fit for the needs of the business at this stage?

• What are the contractual terms for each tool? You’ll want to understand 1) which ones are most expensive and 2) how quickly you can get out of the respective contracts if needed

Based on this, you can assign each tool in the stack a rating to determine next steps: Keep, Remove, Replace (not urgent) or Replace (urgent). Plus, you can flag missing tools as well so you can later make the business case for them.

💼 Getting to know the data team

If you’re coming in to lead an existing team, getting to know each person, learning about the historical context, and figuring out what is/isn’t working should be a major focus during the first 45 days.

Here are some areas you might want to dig into:

• What did the team ship in the last 12 months? What are the current priorities?

• What does the team spend the most time on (this doesn’t always equal the highest priority)?

• What are their biggest pain points and frustrations?

• How does the team work with the business (how requests come in, how they triage and prioritize them, how they keep the biz in the loop etc.)?

• What are the key processes and rituals (and what does the team think about them)?

• How do they gather business context (where do they physically sit, what meetings do they attend etc.)?

• Where does each team member’s expertise lie? Besides informing staffing later on, you can lean on subject matter experts to accelerate your onboarding

💼 Reflecting on what you learned

After you talked to both business stakeholders as well as the data team, it’s a great time to reflect on what you heard:

• Do the descriptions of how the teams work together match each other, or are perceptions wildly different?

• Are the priorities aligned, or is the data team working on projects far removed from business goals?

If there are any glaring issues, pick the most critical one or two and include them in your immediate action plan (see below).

Phase 2: Getting quick wins under your belt

Timeframe: ~ Day 30 to day 75

Now that you have a good understanding of how everything works, it’s time to realize some quick wins. This will give people a preview of the value you’ll be able to create for the business and help you build a reputation for getting things done.

Now, every onboarding guide will tell you to get some quick wins, but how do you actually find these opportunities?

Don’t worry, we got you. Below are a few relevant categories to help you focus.

💼👩🏻‍💻 Creating business impact and building trust

There are dozens of things you can do to drive business impact, but here are 8 concrete ideas to get you started:

• Quick win #1: Fix the biggest visibility gap: Pick the most glaring reporting gap from your conversations with business partners. The initial solution doesn’t have to be a polished dashboard with all bells and whistles; often, a quick MVP can get you 80% of the value with little effort

• Quick win #2: Fix the most painful metrics misalignment: You don’t need to launch a fully-fledged semantic layer as a first step. Just pick the most frustrating inconsistent metric (e.g. Sales and Marketing define “Opportunities” differently) and work with biz stakeholders to standardize it. Over time, this can evolve into a broader effort to standardize all metrics

• Quick win #3: Segment the data and highlight underperformers: Simply cutting data in a novel way can reveal fresh insights. For example, you might find that the decrease in revenue per user that the company has been concerned about is only driven by a particular user segment, giving the business concrete next steps

• Quick win #4: Look for extreme distributions: People tend to assume normal distributions although in reality, that’s often not the case. E.g. a small group of whales often accounts for a large chunk of mobile game revenue, or a few customers cause the majority of support escalations. Highlighting a few of these cases will immediately lead to different decisions in the business

• Quick win #5: Propose and run “low-hanging fruit” experiments: Often, simple but impactful things you did at your last jobs might not have even crossed the minds of people at your new company. Pick a few that seem transferable and run quick experiments to see if they drive similar results

• Quick win #6: Cut / replace / downgrade a pricey tool: After you reviewed the data stack, it’s likely you found inefficiencies; pick one or two to address quickly. E.g. you might find that each department in the company is using a different BI tool, and some are paying for seats but really just using Excel. Consolidating these can help with metric alignment, costs, and even potentially increase data access because you can focus trainings and cross-departmental reporting. Additionally, this will show that you are a frugal operator and will make it easier to get future budget requests approved

• Quick win #7: Address knowledge and documentation gaps: While you’re getting up to speed yourself, use what you learn to improve your stakeholders’ understanding of key topics. This will help to quickly establish you as someone who brings clarity

• Quick win #8: Bring the team closer to the business: Especially if you’re running a centralized data team, you can always improve the flow of information and reduce friction between the data and business side. Try holding office hours, simplifying the intake process for requests, or simply seating subject matter experts closer to their business counterparts in the office

Remember: These are supposed to give a preview of your impact and help you build some initial goodwill. You don’t have to solve all of the organization’s problems at this point.

So pick a few that you can tackle with limited effort, and then shift your focus to building a robust long-term plan (see next section below).

💼 Boosting team morale

If you’re coming in as a new Head of Data, you shouldn’t just focus on driving the top or bottom line. You can also make a significant impact by boosting the data team’s morale.

Based on your conversations with the team, you should customize your approach, but here are some tips addressing common pain points to get you started:

• 👀 Create visibility: Data teams often feel like they are not getting enough recognition. Address this by sending a weekly email update highlighting achievements, giving shoutouts in Slack channels, getting your team’s work presented in all-hands meetings etc.

• ⏳ Fix the team’s biggest time waster: Whether it’s certain low-ROI meetings, labor-intensive weekly slide updates that end up in the appendix, or a bloated web of dashboards that has become impossible to maintain: Pick something and don’t be afraid to cut deeply

• 🎯 Clarify performance expectations and career pathing: Data teams are often among the newer teams in the company and tend to lack clear career ladders that show what is expected at each level and how to progress to the next. This is a perfect opportunity for a new joiner since you only need limited company-specific context. Note: We will cover how to create a strong data career ladder in a later post

Now, you might be wondering “How is this measurable impact? Team morale seems fuzzy compared to business impact”. But it actually doesn’t take much to make it measurable.

For example, you can send a weekly anonymous poll to your team asking them to answer a single question: “How likely are you to recommend working on this team to a friend or former colleague?”.

If you then use a scale from 0 (not likely) to 10 (very likely), you can apply the Net Promoter Score (NPS) calculation. Your NPS is the sum of people who rated 9 or 10 minus the sum of people who rated between 0 and 6.

Even if the initial number doesn’t mean much to you; if you do this every week over your first 90 days, you’ll get a good directional idea of whether your changes are having an impact.

Phase 3: Setting yourself up for long-term success

Timeframe: ~ Day 60 to day 90

With some quick wins under your belt, you now need to translate that momentum into long-term impact.

💼👩🏻‍💻 Clarify the team’s mandate and structure

The first thing you need to do is make sure the team has a crystal clear mandate.

If there is a clear structure in place, even if you don’t necessarily agree with everything, it’s usually not a good idea to do a reorg right after you join. However, in many cases, how the data team operates has organically evolved rather than being the result of intentional choices.

For example:

• Everyone on your team has the title “Data Scientist”, but they spend most of their time building dashboards or fixing data pipelines

• The team is supposed to be the “official” central data team, but in reality, it’s only really working with Finance & Customer Support. Other teams are doing their own analyses and reporting, leading to a patchwork landscape with inconsistent definitions

Or, there might not even have been a proper data team before you joined.

In cases like this, you need to clarify the team’s mandate and how it will operate; otherwise you can’t determine the resourcing you need and the projects you should prioritize.

Whatever mandate you land on, make sure that:

1. It reflects the needs of your stakeholders, and

2. You have leadership support for this path forward

💼 Build out the team

If you’re taking over a data team, chances are it has been underresourced historically.

For example, when Gordon Wong came in to revamp HubSpot’s BI team, he estimated that the team was ~15% of the size it needed to be given its scope.

Based on the mandate you landed on, you should develop a rough estimate of the resourcing you’ll need:

• How large does your team need to be at a bare minimum to fulfill the mandate?

• What mix of roles and seniority do you need?

You might have to show impact before you get this approved (and you might only unlock additional headcount piece-by-piece), but the earlier you develop this view, the earlier you can start lobbying for your needs.

And once you have a rough idea of what you need, you should start hiring immediately.

You’ll likely drown in stakeholder requests, so it’s easy to want to punt on this. But the longer you wait, the more difficult your situation will become.

💼👩🏻‍💻 Create a roadmap

Quick wins tend to be somewhat random low-hanging fruit, and you need to show that you can create a compelling plan for the team that maps to the company’s strategic priorities.

When you’re putting together your initial roadmap, the most important thing is to stay realistic. In most cases, the status quo that you’re inheriting is a mess, and you’re not going to untangle it within a few months.

There is a delicate balance, however: Even if some of the larger strategic projects will take multiple quarters to show an impact, you need to create measurable value for the business in the interim, or you might not get the chance to see them through.

As a result, the goal for your first roadmap should be to deliver enough business impact every quarter to “move to the next stage” (i.e. maintain leadership support and unlock more resourcing).

Remember: Nobody outside the data team knows or cares how much effort went into something. So prioritize projects with low effort and high business impact and/or visibility:

A few principles to keep in mind to set yourself up for success:

1. Focus on pragmatic solutions

You show your value by solving business problems, not by building fancy solutions. Overengineering just puts an unnecessary drain on your limited resources.

For example, instead of committing to building out an end-to-end data model as a source of truth (and risking becoming a blocker to the business), develop the data model incrementally as the need arises as part of other projects.

2. Set realistic expectations

Business stakeholders and leadership often see the arrival of a new data leader as an opportunity to voice their (unrealistic) wish list, and it’s up to you to push back.

This is especially important if you’re a team of one or resource-constrained. In order to be collaborative, you can present options as a “Give X, Get Y” (e.g. “with two more data scientists, we could build the lead scoring model”); that gives leadership a tangible trade-off.

Just make sure you’re taking into account that it would take several months to hire and onboard these additional resources.

3. Don’t rush into big data infrastructure projects

When you evaluate the current data infrastructure, you’ll likely find a number of issues that make your life harder. It can be tempting to try and fix them right away, but you won’t do yourself any favors.

The business impact of these projects is often not straightforward to articulate to non-technical audiences, and it’s easier to get buy-in for them once you have built up your credibility.

4. Make yourself essential

You should aim to both 1) integrate your team — or yourself if you’re a team of one — into key processes like forecasting, as well as 2) own discrete strategic projects (e.g. building predictive models):

• If you are only doing strategic projects, you’re less essential. Projects are much easier to cut than core processes when priorities change

• But if you are only supporting core processes, it’s hard to drive outsized impact. Nobody ever got promoted for keeping the lights on

5. Leave room for ad-hoc tasks

You should reserve at least 30% - 40% of total bandwidth for ad-hoc requests.

This is one of the most common pitfalls for (new) data leaders: If you don’t account for ad-hoc needs of the business, your roadmap will fall apart before you even really start executing against it.

When you’re planning timelines for your projects, make sure you account for the seasonality of these requests; right before big meetings (e.g. board meetings) and during planning cycles, you’ll drown in “quick asks” from your stakeholders, so ideally these periods don’t overlap with critical phases of your strategic deliverables.

6. Get explicit buy-in

It doesn’t matter how good your roadmap is; if the business and leadership aren’t bought in, it’s pointless because it won’t drive impact.

Get your stakeholders involved throughout the process to ensure your roadmap is aligned with their needs. If you keep things close to your chest and then realize at the last minute that there are fundamental issues, you won’t have much time to address them.

If you partner closely with them, however, the final sign-off will just be a formality.

Quick recap: 90-day checklist for new data leaders & ICs

Made with 🩷 by the folks at Omni