The Missing Tool For Data Exploration

If you interact with business data on your job, you probably wonder: with an abundance of data and analytics tools these days, why is it still so hard to answer the most important business questions?

Traditional business intelligence (BI) and even newer product analytics tools work well for answering reporting questions like: What is the number of users visiting my website/app per day? However, they don’t extend well to deeper ones like: Why has the revenue stopped growing in the last month? How can we improve user conversion rates? These are the actionable questions, which businesses are really after.

Today’s analytics tools were built for “what” questions and often fail to answer more open-ended “why” and “how” questions, because answering “exploration” questions is structurally different from the “reporting” ones. As any data practitioner can attest, going after exploration questions is a creative process of discovering patterns in data, forming hypotheses to explain observations, finding evidence to support or falsify these hypotheses and finally building a convincing narrative.

For reporting, we know questions beforehand and so can precompute aggregated data views and work with the traditional metrics-based analytics and static dashboards. But to unlock the full power of exploration, we need to have access to the underlying data — streams of raw events (logs). Working with events is often done in the context of temporally adjacent events for the same user (think actions per user session, retention, attribution). That’s why we believe that sequence-based analytics is the next-generation approach unlocking deep data exploration.

Let’s look at a typical question all growth teams face: How can we improve user conversion rates? The common approach to answer it through metrics-based analytics is to build a user funnel. It can roughly point to where the biggest user drop-offs happen but not why or how to improve. In contrast, using sequence-based analytics, we can compare common website/app session paths of users who convert to those who don’t. This gives us the exact pages and user states of drop-offs and helps see what non-converting users do instead. Much easier to think about possible reasons for differences in user behavior and ways to improve conversions!

Exploring event sequences has higher inherent complexity than viewing metrics, and analytics tools haven’t discovered yet what makes a great interface for sequence analysis. The common approach is Sankey diagrams, but how often have you actually found them to be useful? We can do much better.

Data exploration is inherently visual and interactive, engaging human curiosity, pattern matching, intuition and trial-and-error. It’s almost tactile, like figuring out a Rubik’s cube by playing with it in your hands. Expanding on the work of Bret Victor — an inspiring visionary of better tech interfaces — we need to extend data exploration from “information software” of the traditional BI into the space of “manipulation software”.

What does a better tool for data exploration look like? We can draw inspiration from visual tools for experts in other creative domains. Designers have Photoshop and Figma, game developers have Unity, architects have AutoCAD. For example, take a look at the work process in SketchUp:

Feels very different from viewing a dashboard or writing SQL, doesn’t it? Note some of the key guiding principles:

• Rich and intuitive representations of objects on the screen with broad context, instead of only narrow slices (think: full user paths instead of predefined metrics)
• Understanding through interacting rather than only looking (dashboards) and thinking (SQL)
• Visual and even tactile ways of performing operations: dragging, rotating, stretching, zooming, etc
• Real-time interactivity, supporting continuous operations: pixel-perfect stretching, rotating, zooming, etc
• Manipulation and its result displayed in the same space (instead of separate panels in dashboards or SQL editors)
• Iterative reversible workflow, forgiving of mistakes and encouraging trial and error.

We have started Motif to create a medium for thinking with data visually, which feels more like creating in Figma than viewing a static dashboard or writing SQL in a text editor. The term “motif” represents an important pattern in sequences: music, novels, DNA, architecture, fabric, — and captures the spirit of sequence data exploration. We — Misha and Theron — have previously led solving these problems at Google, Uber, Waymo and Coursera, and are ready to bring this to the broader ecosystem.

If this problem intrigues you, we’d love to get in touch and show you what we’re working on: hello@motifanalytics.com. We’re currently in stealth, looking for early partners and actively hiring for positions in design, data engineering, frontend engineering and analytics.