Many Faces of Data: How To Begin With UX Success Metrics?

UX designers are famous for wearing many hats. We can be artists, business consultants, workshop facilitators, researchers, et cetera. The hat of the UX Success Metrics is a new addition to our collection that can sit just as well.

As designers, we run the products we build to be satisfying and easy to use for a long period of time after we build them. But how can we know that's the case? Well, let's explore how to start with UX Success Metrics and prove that our design work is worth the investment.

Let's start with our main hero of the day. Let's look at our Heroic Hugo. Hugo is a UX designer. He is good at what he's doing. And he loves it too. For the past year, he managed to build a successful new product. Meeting all the user needs through design thinking methodologies. So, everyone in the company started to refer to him as Heroic Hugo performing heroic UX deeds and he believes that UX outcomes are so good and clear, why would anyone want to measure them? The project is done. The success itself is self explanatory. Everything works. Well, our Hugo poses a great question. Why should he measure UX especially if it seems that it is a success? I would throw a curve ball and ask hugo, what does he mean when he says that the product is successful? How does he know? Is it going to be successful after one year or two?

Crossing the finish line is a great feeling, but seasoned UX designers know that there's always more to be done. To build a product based on user insights isn't enough nowadays. We need the data that shows our continuous success, we need to start tracking UX Metrics. So, let's find out how can we do it.

Hello everyone. My name is Justyna Belkevic, I'm a Senior UX Designer. I have worked with many fantastic clients through the years and I can tell you that UX Metrics is still a new kid on the block. So, that gives us UX designers a fantastic opportunity to be trailblazers in this new field and wear these hats before anyone else.

So, let's start from the beginning. I have been using this UX Success Metrics term quite a bit now. So, let's unpack it. UX Metrics are a set of key quantitative data points used to measure, compare and track the user experience of a website or an app over time. And most importantly, they need to be aligned to business goals so we can measure behavioral and attitudinal data points. Behavioral metrics focus on what users are doing and attitudinal is how they're feeling when they using the service or a product.

So, let's look at the example, let's take Superman as our main hero and the business goal for Superman is to make sure people of Metropolis, well, don't die in accidents. So, the Success Metrics, reflecting that goal could be rescued people per hour, flight time to arrive at the crime scene, or maybe even outfit change time. And so, the key metrics are usually different for each product and should always be determined individually. As you can imagine now, the metrics that would fit to measure the success of our Superman might not fit in the call center environment.

Okay, let's move to the big question. The 'why' question? The use of Success Metrics reduces opinion decisions, helps prioritise problems and improves our products. It's always a good idea to have UX data with us when discussing any changes to the product. And an even better idea is to always start with the questions you want to get answers for, and only then think about which data to use. So, for example, to our Superman, we can ask questions like that. Which skills make our Superman most useful for the citizens of Metropolis or maybe which outfit leads to the most social media engagement. Interesting, isn't it?

The question, when is something that I haven't seen being answered frequently in the internet space, but I believe it is as important. And from my professional experience, I want toshare that it is never too late or never too early to start working on your Success Metrics. I personally like to fire such discussions to the stakeholders as soon as I complete the initial research and discovery. And the reason being is that as we saw in the previous slide, those discussions don't need to begin with data, but with a goal-oriented questions first. You don't need to have a wireframe to already start imagining how the product's success might look like. And in between the research and the final user testing activities, there's plenty of time to change the goal and the tracking methods. Most importantly, speaking about UX Success Metrics early will help you to manage the expectations early on too. So, if during your research, you discover that your users will probably use your product once or twice a month, you can already manage the expectations of the stakeholders and the advice that you shouldn't choose the daily visit rate as the metric to track your success.

That all sounds great. So, how do we start? And I'm glad you've asked because I have five step guides on how to start with UX Success Metrics.

So, start with step number one. And that's always, the first step is the most important one. One way of creating Success Metrics, we need to start with the goal. Okay. Amazing. But how did we choose it? Is there anything that we can do to help us? Absolutely, yes. When I am thinking about the goals, I like to ask myself this question, "what will happen if our design is successful?", and the answer might be we will increase orders or maybe our call centers we get fewer calls about the log-in issues or a decrease in the crime rate in Metropolis, if our question is targeted to our Superman. So, to help with choosing a goal, I want to share with you the most frequent goal types. So, those are revenue, acquisition, and engagement. You see spending and thinking about our goal is really important before we jump into the nitty-gritty of choosing a specific metric to track it. It is like building a product itself. We don't jump into UI before doing UX, right?

So, one clear goal that your team and stakeholders agree on is prioritise your success events and success events are actions that you want your users to take towards your goal. And I advise teams to have micro-conversion and macro-conversions. Your micro-conversion is basically your goal. For example, registering for your newsletter, maybe the allowing a software package, et cetera, and your macro-conversions are all necessary steps that users can take in which all helped this micro-conversion to happen. Example, if you are an online course company and you are micro conversion is to have users registering for a course. You're a micro conversion might be a number one signing up for a newsletter. Number two, downloading a brochure. And number three, maybe you're requesting with an instructor. So, knowing your success events will help you to prioritise them and understand where to give you all.

Once you have the strategy and prioritisation in place, it's time to think about the metrics. Here, we'll not be discussing necessarily all pros and cons of every single UX Metrics. I strongly believe that the go.com or many other presenters at this conference will do it better justice, but we will look at the metrics from one level higher up. We are going to answer the question of how you can choose metrics that are right for you and your project. Good news. It is actually pretty straightforward. Like 1, 2, 3.

So, number one, make sure that the metrics are aligned with your goals and success events. I know, I know you might be thinking, why am I mentioning such an obvious statement, but trust me sometimes in the weeds of stakeholder discussion I need to agree on the metrics that had nothing to add to the success of the project but just sounded rational. So, number two is to choose actionable metrics. Ask yourself, what would I do if the metric is out of line? Do I have the leavers that can impact it and measures that track final outcomes like revenue or total customers don't give you much time to react or guidance about what to do next. And number three, focus on three to five key metrics rather than trying to capture and analyse everything.

As Yuval Noah Harari in his book, Homo Deus said, 'In ancient times, having power meant having access to data.' Today, having power means knowing what to ignore. Great. So, everything that happens after this point, you need to have a working product. All the previous steps, you don't need to have it just yet, but now we have it. We have either in the app or a website. And your dashboards or reports should be easy to understand and provide enough data to give your audience context. I'm sure many of us have seen dashboards that stubbornly show some of the miracle’s values, but which we can't decipher on our own and which then have the potential to be ignored even if they are super important. So, when you will create your own artifacts, remember to share insights not just data, tell a story about the numbers, make it easier for the reviewers to remember your point. For example, starts with setting the scene by explaining the origin of the data, then introduce your characters and the plot, making the viewers care about those numbers. And don't forget to add the conflict through a story and a happy ending or a resolution.

So, let's do it now together for our usual client Superman. Many citizens of the Metropolis are just like you and me. We want our day to go crisis free. Recently, we noticed that 66% of younger citizens of metropolis are curious, and they want to know more about Supermen, not just that he solves the crimes but our Superman is so busy solving approximately 10 crimes a day and saving the city $1 million every half a year that we can have him hanging out with the kids. So, in order to solve this challenge, we decided to experiment with social media and created an account for him. Now, the Superman has 15 million subscribers and we measured that it positively impacts Metropolis citizens who are now 10% happier and 14% feel safer by seeing what Superman does on his social media. So, stories like this will help your stakeholders remember the data and care about the user.

Slowly but surely, most companies have arrived in the 21st century and are collecting large amounts of data every day. But having a lot of data or tracking everything you can think of will not equate to successful UX Metrics. As a UX Success Metric enthusiast, we need to make sure that we can all us read between the lines of what our data is showing us because even simple metrics can be easily misinterpreted and I will give you an example. Think about time on site, which people often equate with engagement. More time, could it be positive, but it could be also negative. For example, time that the user spent feeling confused, distracted or even frustrated. Even if you track engagement by looking at time spent on your site and pages per visit together, it's not clear how that equates to engagement all the time. The skill is to read between the lines and I'm sure that will come with practice or would the use of more and more sophisticated tools. So, be always aware of the many faces of data.

Great. So, let's summarize the most five points. In order to start with the UX Metrics, we shall follow these five steps. We will start with a goal. We'll prioritise our success events. Then we will choose what is right for us and the project at hand. Then we will turn numbers into a story and we will always be aware that the basis of the data and as UX Success Metrics enthusiasts, we will be now equipped to make sure that those ones and zeros are tailored to our goals and help us to achieve our design outcomes.

But the wait, a bonus. How to be one step ahead of the game. I'm sure you'll be thinking about it. Well, organisations across the board are seeking more meaningful metrics that go beyond the defaults of the shelf analytics that can provide. The newest development in metrics is to leverage different methods to collect data about the same phenomenon, rather than relying on single signals. For example, using the number of page views as proxy for engagement. The trend is now towards setting metrics that draw from multiple sources or event streams. We call it, Triangulation. Having multiple points around the behavior. We want to study which enables us to make better decisions.

So, when you are measuring your UX Success Metrics with the help of analytics, and you sprinkle in some moderated usability test or user interviews. It will enrich your understanding of how users use the product. And then we'll help you minimise the limitations and blind spots of single methods.

So, if you remember one thing from this presentation is that successful UX metrics start with a goal. Once the UX metric is goal is defined, then we can talk about the tools and the data itself. This practice, the goal setting will be the strongest building block and your new journey with Success Metrics. So, it's time for UX Success Metrics to have the status of deserves in your organisation. And the sooner you start measuring the sooner it can slip into your Superman outfit.

Thank you so much. My name is Justyna Belkevic and it was my pleasure to share my knowledge with you.