The Impact of Research Beyond the Report

Hello, everyone, and welcome. My name is Kendall, I'm very excited to chat with you all today about the indirect impact of research beyond the report.

So my name is Kendall, I lead the user experience research team at Uber for the rider experience. And this is a topic that's really close to me, because it has really kind of shaped the way that I lead research for myself and my team over the past few years at Uber. And to kick this presentation off, I really just want to start with a quick story.

So a couple of years ago, I was getting ready to make my next career transition, I was looking for my next opportunity. As I was in the process of getting all my materials together, updating my portfolio, my resume, I was working with different recruiters and consistently, I was hearing the same thing, like make sure that you highlight your impact. On the surface, it sounded totally fair, of course, I would want to highlight my impact. But for some reason, this got me really nervous and anxious, because when I thought about the impact that I was having, I didn't have a lot of metrics or numbers that I could point to, to show the impact that I was actually having on my work and my team.

I didn't have things like CSAT scores I could point to I didn't have a lot of launch metrics, many of the projects I'd worked on, hadn't quite made it to a meaningful launch for our users. And in fact, the project I was most proud of was still in an early concept phase hadn't even made it out to users. And so I was getting really nervous and anxious of how am I supposed to highlight my impact if I don't have any numbers to show for it?

At the same point, I was thinking, I've been in this role for over two years, what have I been doing? If I haven't been having an impact? What is it that I've been doing for these past few years, I have to have been doing something. And so this is the moment that I just sat down and started to write down. Okay, what have I been doing? What have I been doing for the last few years? And this is kind of where that lightbulb moment happened for me, this is where I realized that some of the activities that I had been doing, really were impactful, but just not in the way that I had thought about it before.

So for example, I was convincing skeptical stakeholders about the value of research, I was introducing new research methods to the team, expanding the quality and capabilities of the insights we were actually able to capture from our users. I was even onboarding new designers and helping them to lead their own research that we were actually able to scale our practice. And so when I, you know, looked at this list of things that I had been doing, each of them actually was an example of impact, it just wasn't the way that I had framed it before. And so ever since its realization, I've been really intentional about leaning into all of these kind of indirect or, you know, at times they feel immeasurable forms of impact in my role here at Uber. And so when you think about what it means to have impact, it's to have a strong effect on someone or something.

So it's not always something that needs to be measured with metrics and data, it can just be a matter of recognizing when and where you're actually having these effects on others, and then how you can continue to have these effects and impact in your work moving forward. So today, I'm excited to chat with you about the three techniques that I have found that have stuck myself and others on my team up for success, to ensure that we're kind of increasing and maximizing the impact that we're able to have with our teams.

And so I'm going to touch on building authentic relationships, the importance of bringing the team behind the scenes, and then what it means to actually go beyond the research report and make sure that your impact lives on even after your research study is completed.

So to kick it off, building authentic relationships, this to me is just the foundation that the rest of your research and practice really going to build on top of, it's what's gonna set you up as this credible resource that your teams can go to when they need research, guidance or support. And it's really going to help you expand the reach and impact that your work can possibly have. So a few ways that I like to think about building these relationships, I really start by identifying who are your allies. And these are going to be the folks that will help elevate your research, make sure that you're able to action on your insights. And so to start, just take a look at who it is that you're working with directly on your projects.

Maybe these are designers, product managers, engineers, data scientists, whoever it is, take a look at who is in that core network and start to build relationships with them individually. The way that I like to think about this is get to know them as people first and coworkers second because this is where that authenticity comes in. If you're not really interested in building this connection, the person at the other end of the conversation is going to be able to tell and so really focus on how you can set up really strong foundation of good authentic relationships with your team from the very beginning.

Then once you've built these relationships with your immediate team start to expand your network outward a little bit. So who are the partners of your partners? Who are the people that your core network is engaging with the most frequently, is like, who are their stakeholders, what matters to them and set up some time to just introduce yourself to these folks, kind of establish yourself as that research counterpart to their work. And even if you're not going to be engaging with these partners on a regular basis, you'll still be setting yourself up as this familiar face, when your research does, in fact, intersect with their work. Then, lastly, identify who are your influencers, these are going to be the decision makers, the people that have a lot of influence in your team, and try and understand what matters most to them, how have they engaged with research in the past, what's worked and what hasn't. And all of this is really going to be important for you to understand how you might need to maybe frame your research or insights so that you're able to set it up to be as impactful as possible for the people who are really able to action on your insights and recommendations.

This is where it can be really helpful to have that group of allies built around you who can also help elevate your work and make sure that it's having the maximum impact possible. Next, I like to think about reframing the role of research within the team. So to me, this means thinking about research as a partner, and not as a resource. And when I think about research as a resource, the way that usually kind of shows up is that it means a researcher is brought into the process, maybe after a project has already been kicked off, we're kind of asked to swoop in, deliver some insights, and then leave and move on to the next work.

But if we reframe our role to that of a partner, we're actually setting ourselves up to be a member of the team from the very beginning. So there's a couple of ways that you can think about setting yourself up in this way. So number one, really embed yourself in all of the different team meetings and rituals, and really not just the ones that center around research. So this means go beyond research kickoff and research report read out as your primary points of connection with your team. And make sure that you're attending standups, design critiques, product reviews, any other meetings where decisions are actually being made. Because these are all places that your research can actually inform these decisions, it's going to set you up as a visible member of the team. And it's also going to help you gather those valuable insights about what the team cares about the most. And these might even be things that didn't initially come up in that research kickoff.

Second, think about continuing this partnership with stakeholders even after the research is completed. And so if research is treated like a resource, the expectation is that we maybe just leave and move on to the next project after the research is completed and the insights have been handed off. But instead, try and find a few ways that you can really stay involved with the team after this research phase is complete. And so maybe you're doing this by setting up a regular one on one, either bi weekly or monthly, or maybe you're just attending the different stand ups that your team is involved in.

However, this really shows up for you. The key piece here is just continuing to make yourself that visible team member, which leads me to my last point, act like you belong because you do. And so this is your chance to speak up in meetings, share your perspective, really just be that active part of the team. This is a piece that I used to really struggle with, because I used to see my role as a researcher as to just share the insights, share the recommendations, anything beyond that, I felt like I was overstepping my role as a researcher, I was just here to, you know, tell the team what I learned and that was it. But eventually that I realized, you know, I was actually embedding myself in these team meetings, I was showing up with the team, I was starting to be empowered to contribute more to the conversation, I really realized I had a seat at the table. And it was actually up to me to take advantage of that seat.

And so I started to just share my perspective, share my thoughts a little bit more often. And what I actually saw is sometimes I was bringing a brand new perspective that the team hadn't even considered before. And so remember that once you embed yourself as a member of this team like you belong to be there, you deserve to be there. And so take advantage of that.

Finally, as you're considering how you can build these authentic relationships, it's also important to remember how to compromise and when it's time to compromise. And so maintaining the strong relationship is more important than maybe digging your heels in and say this is the right research and this is the right approach. Building trust is a long term goal, and it's going to require some compromises along the way.

One of the ways that I see this kind of becoming a bigger issue is when maybe UX or can't support a research request, or maybe the request that comes in isn't necessarily the best candidate for research at this time. And so it ends up kind of feeling like we aren't able to be a good partner or keep that relationship strong because the research team is saying no for one reason or another. Whatever the reason might be that you're in this situation, I try and find different ways that I can continue to support this team and keep them moving forward, while maintaining that positive relationship with research. So one of the ways that I really think about this is identifying some alternative solutions to keep the team moving forward. Is this a question that could be answered by desk research? Something that we can just kind of see what's been done before? Can we bring in a vendor to support this project, if we don't have the capacity to do it ourselves?

Maybe this is a chance to take a look at what other research is happening right now and say, you know, we can take this one question you've got and actually fit it into another study that's already happening, maybe that one question on its own didn't merit a full study. But we can still answer that question in conjunction with another effort that's already ongoing.

Maybe your role as a researcher is to not do the research at all, because there's actually a better team who's best suited to help this team get the information that they need. And so maybe you're pointing them to other insights functions, like data science, or market insights. And you're able to do this because you've built those relationships up front, and you know, who's in your network. But really, just try and find ways you can keep that relationship with your partners strong, and also keep them moving forward.

So as you think about, I've started to build these relationships, am I having an impact yet, there's a couple of different things that I like to look out for.

So number one, you might be receiving more research requests. This is usually one of the first ways that I've noticed the impact really kind of taking shape. So you're starting to receive research requests, to conduct new research to maybe find past research. Overall just being that go to person for your team whenever they have any type of research question. And it's important to note here that just because you're getting more research requests does not mean that you necessarily have to take them all on, this is where that power of compromising really comes in.

Another thing you might start to notice is that you're actually brought into the process a little bit earlier. So research is now seen as a key part of the kickoff process because you've started to embed yourself as a member of the team, you're part of the team. And so maybe you're actually doing more upfront research instead of being brought in at the end. And you're doing research that informs and defines the rest of the roadmap instead of validating a solution before it goes to launch.

Finally, you start to notice that you actually have a larger network of stakeholders. So you've built these core networks, that extended network, you found out who those influencers are. And you're really building these intentional relationships across the organization. This is opening you up for more opportunities for your research to inform their work and to have impact across the business.

So pay attention to how many people have shown interest in your work, how many people are reaching out to you about it, where they're coming from, to realize just how large of a network you've started to create.

Next, I want to talk about what it means to bring your team behind the scenes. So as researchers, I think we sometimes forget just how much we know about our craft, we've been doing it for a number of years, it just starts to kind of feel second nature to us. And unfortunately, that means that sometimes we're unintentionally leaving our partners and stakeholders behind in the process because we just know what to do next. But the more that we involve our partners and stakeholders in the process, the more often that we're actually aligned in the goals and outcomes. And I've actually found it can create a lot of empathy for the role and the work that we do as researchers as well.

So, I like to start by just getting my team on board with understanding what certain research methods are best used for, and the point in time that they're going to bring the most value to a project. So as the saying goes, when all you have is a hammer, everything looks like a nail, definitely found myself in situations like this before, where maybe my partners and stakeholders look at research and say, well, we can run a survey and interview or usability study and that's it. And it's just because they don't realize how many tools we actually have at our disposal to answer all the different questions that they have. And so it's not unlikely that your team might be coming to you with the method in mind first.

So to combat this, I like to start by explaining the purpose and trade offs of different methods until I actually reached that point of understanding with the team. So it's not just enough to say, this is the method we're going to use. And to just move on with it. I really have to make sure that I'm bringing my team on board and keeping those relationships really strong together. So I focus on describing what the method is that I'm proposing for the team, how we're actually planning to conduct the research, why I chose this method over others and describe the other options that were available to me, and then what the team is going to expect to learn for on these insights.

I found that without all this alignment upfront on all the different pieces, this is when the research runs the risk of, you know, my team maybe disagreeing with it, or not acting on it, because it's not what they expected to learn, and they're not actually sure what to do with the results. And this leads me to my next point, align with your stakeholders to understand how they're planning to action on the research. So this is where it's important to figure out is this research a must do effort? Or does it kind of fall into that nice to have? In my experience, I've seen this happen a lot with surveys. Sometimes a team wants to run a survey, and they want to learn how our users responding to a new product or feature.

And the way that I like to reframe this is to kind of flip the question back to them. And I asked, like, what if, let's say, this is unlikely, but every single respondent hated it. They just didn't like it. Would your team actually do anything with that data? And if so, do you know what you would do with that data? If the answer is no, maybe this isn't the right time to do the research. Or maybe this isn't the right research method. And we need to go back to the drawing board and make sure that we're asking questions of our users, that we're getting useful data that we know what we're going to action on, and what we're going to do to keep the business moving forward.

Sometimes one of the best things a researcher can do is determine that there's not actually a need to do research at all. And we can point to other ways that we can kind of get the insights that we're looking for. And so it's really important to make sure that your stakeholders are aligned on the expectations of the methods where they fit best in the process, and what they can expect from the research insights after the study is completed.

Sometimes, though, despite your best efforts to get everyone on board up front. It's still challenging to bring stakeholders along, especially if you're maybe introducing a new concept or method. And so this is where I find that the power of showing the method or the value of the research really comes into play. So a specific example was a couple of years ago, I was working with a new product partner. And we were reviewing the research questions that they had. And my proposal was to run an unmoderated study. So this is we would launch the study, ask users to accomplish the task, answer a few questions, we get video clips back within probably hours, and the team would be able to move forward pretty quickly.

It seemed to work with the research questions that they had, it definitely worked with my timeline as a researcher and my ability to support but the team was skeptical. They had never done an unmoderated study before, they were really nervous about the fact that there wasn't going to be a researcher, they're kind of guiding the participant if they have any questions or helping them out or really digging deeper to getting the insights.

And so this is where it's really important, again, to remember those compromises, I decided to suggest, well, let's go ahead and run the unmoderated study, let's just find out, let's just learn. And then if we find out that we're not getting the results that we're looking for, we're not getting the insights that are going to help us make the next decision, I will set up some time to follow up with users and do a set of interviews. And that was a method that was familiar to the team. So we launched the unmoderated study. And I think probably to the surprise of all of us, it was so successful that the product manager came back a week later and was asking when can we run another one. And so this is just an example of how powerful it can be to really show the value of the research to your team.

So a couple different tips that I consider when leveraging this technique. First, consider introducing a new research method in conjunction with a method or approach that the team is already familiar with. So for example, maybe you're introducing a card sword, which is a new method to them, and you'll do it in an addition to an interview. This way you can kind of like temper any concerns that they have because you're introducing something that they're familiar with, in addition to something that's a little bit newer.

Secondly, I would say invite your team to attend the sessions live, this is one of the best ways to make sure that they're getting access to the insights in real time. And I think it's an especially great tool when you have those maybe skeptical stakeholders who question the legitimacy of the research, or can we really trust this, we only learned it from six people or we only talk to six people or whatever it may be. This is a way that you can allow your team to really see the pattern starting to form for themselves and understand what it is that we're actually doing as researchers and why these insights and findings are so trustworthy and credible to the business.

And then finally, as you're developing your research report, really prioritize the video clips and direct quotes in your research. This way you can make sure that you're addressing your team's research questions or concerns with those video clips or quotes. It again strengthens the validity of the insights that you're capturing, I've also found it's a really great way to kind of like separate yourself as the researcher from maybe a controversial finding, you're maintaining that really strong relationship. And you're really just putting it front and center and saying, look, this isn't me saying this, this is the users and this is why we need to pay attention to this and action on this finding here.

And so making sure that you're kind of showing the value of research by putting your users front and center, showing how these methods can be introduced, and the value that they bring, and all these different things that your team is actually going to learn that they maybe didn't have access to before.

Then finally, it's important to recognize the power of language. Research terminology is tricky, and at times just confusing. So there's a ton of jargon in the craft. And I found that if we haven't set the right expectations upfront, then we're increasing the likelihood of just miscommunication down the road. So one of the first ways I like to address this is just a line on what we mean when we're talking about different terms. So for example, does your team believe that user testing is the same thing as usability testing? What about concept testing versus usability testing?

This came up very recently, I was running a study and we had a very long discussion on what is the difference between find ability and discoverability? I also was recently talking with a colleague, and halfway through the conversation, we had both been talking about user testing. But we found out that she was talking about running a live AB experiment, and I was talking about running prototype usability testing for our users. It happens all the time. But the most important thing is just make sure that you are aligned up front on the terminology that you're using the expectations of it, making sure that everyone's on the same page, because you're just going to move a lot more quickly and efficiently through your work.

Another thing to consider when you think about language is how it impacts the expectations that your team has of your work. One word that I am very particularly mindful of is validate or validation. In my experience, I've often found that when teams are seeking validation, they're looking to confirm their assumptions, confirm that they're right about something. And while it's completely valid for us to run research that evaluates or tests whether or not something is actually like working as well as we hope it will. What I like to do is slightly reframe the language to evaluate or investigate. And the reason I do this, it's a subtle change. But it really opens the team up to understanding that there might be a change we actually have to make, this isn't just going to tell us that something works and there's no changes to be made. This means that if we're investigating or we're evaluating, we're open to the possibility that we might need to do something on our end to deliver a better product experience.

And if the team is not comfortable with that, if this isn't going to work with their timeline, we aren't in a position where we're actually going to be able to make changes, well, then maybe research isn't the best tool for us to use at this point in the process, is to really making sure that you're using language in a way that communicates not only what is it that you're doing, but what the expectations are of the team as well.

So a couple of things to look out for to identify where and when you're having your impact. First, you're going to start to see that your team comes to you with questions instead of methods. This is one of my favorite things. When you start to see it happening. You're no longer getting requests to run a survey, launch a usability study, whatever it might be, you're not coming with method first recommendations or suggestions. You're coming, your team's coming to you with questions. And this means that they really trust you as the researcher to be able to determine what is the right method and the right time to introduce that.

You're also going to start finding out that you're working on more of those high priority efforts. This is likely because your team understands the value of UXR, they know that you have a lot of different ways you can answer their questions. You're an embedded part of that team, you know, you're a partner and so the resource, and you know when it's the right time to say no, this isn't a method or a research study, that's actually going to get us the most valuable information at this point in time.

And so you're refining the type of work that you're doing. And you're only working on those high priority efforts that are really going to bring a lot of value to routine. And finally, you'll start to notice that your team is speaking your language. This is another fun one to look out for. Your team is aligned upfront on the methods. They know what to expect when the research is completed. But when you start to hear your stakeholders use terminology that you've introduced to them, that's when you can really know that you're starting to have an impact.

Your team is using this language on their own. I've seen this happen with my team with the whole find ability versus discoverability. And it's a really great feeling to see that they really have internalized the language and what it means and how it impacts the decisions that they're making.

So finally, I'm hoping that this section is the piece that kind of addresses a common fear that I know I have felt that I've heard from other researchers too. And it's that our work ends up sitting on this proverbial shelf, never to be looked at or referenced again, you know, you've done all this hard work, you've talked to your users, you've recruited them, you did the analysis and the synthesis you've created this beautiful report at the end of it. And you don't want your work to end there. And so this whole section is all about how can you make sure that even after your report is completed, you're maximizing the impact and the reach that it has with the stakeholders who are going to be able to action on it the most.

And so, one of the easiest ways I've found to make sure that this happens is just to cite your research, literally everywhere. This, to me, is kind of a lower touch way to make sure that your research is just reaching the broadest audience possible. And so I do this by sharing it in Slack messages, dropping it in zoom chats or teams chats. If I'm taking a look at a document that somebody has asked me to review, I'm leaving comments and linking the research directly there with them. I'm just making sure that the research work that I have done is shared as broadly as possible. And it's all say I'm sitting in on a team meeting. And I'm presenting or I'm listening to the team present and ask questions about something.

And I will, you know, notice that a theme is starting to come up with theme that we've actually done research on before. This is the moment that I will maybe just link the research into the Zoom chat, I'll drop it in there. I might add a little bit of context here. But it's just a way to kind of get it out in front of the audience that you know, it's on topic. It's relevant. They're discussing this work right now. And what I'll notice is, as soon as I send it out, I see people joining, I see people taking a look at the document. And right then and there, I know that I'm starting to have an impact. And so really just finding as many opportunities as possible to link out your sources, link out the research that you've been doing, and make it really, really visible that you've been having an influence and impact on the work that is relevant for your team today.

And then, in addition to citing your own work, it is your responsibility as the researcher to kind of be the voice of the user in every single room. So I like to think about this as being the squeaky wheel. So the person who's always reminding the team of what it is that we've learned and how it's impacting the users experience. So one of the ways to do this is to just diversify the rooms that you're in. So attend meetings with different cross functional partners, make sure that you're able to highlight your research insights there and how it directly relates to their work.

And this is also your chance to highlight where certain business decisions might be actually going against the insight or recommendations from UX or team. Sometimes the business just moves really, really quickly, and without constant referencing of your work, and constantly bringing it up to the team, it's a chance for your research insights to unfortunately be left behind.

And so as you are constantly referencing bringing this up reminding your team about what it is that you've learned the impact of the users how you can make sure you're delivering that optimal user experience, it is important to remember that while you should be the squeaky wheel, don't be too squeaky. And so what I mean by this is like figure out, like it's kind of pick your battles like where does it make sense to really say, no, we absolutely need to solve this for our users. This is high priority and when is it time to say we've learned this, I've shared it enough times and the team is going in a different direction? How can I make sure that my insights can at least inform that new direction. And I guess, just make sure that you're kind of maintaining that compromise mentality to keep those relationships that are really good level.

And then finally, as you're thinking about how your own research can live on? Remember that it can also be your responsibility to help existing work of others live on as well. And so I like to think about this as connecting the dots, how does my work relate to work that others have done, maybe this is work from other researchers or data scientists. Maybe this is the market insights team, anything that really kind of tangentially connects to the work that I'm doing. It can also be a way to elevate the work of others by finding those connection points. And so this also really helps to strengthen the narrative and triangulate those insights that you've captured by saying, look, we've seen this and other data as well.

As you're doing this connected dots work, you might even realize that some insights that you've captured in your own study, actually have a pretty strong impact to another team, maybe even more strongly than your own. And so you're also starting to create opportunities to proactively identify and propose new research questions for the business. And again, you're expanding your reach of research. And so this is something that happened very recently for us at Uber. We just kicked off a brand new research offer because three different researchers all had different insights, that at face value didn't seem connected.

But when you really looked at it, there was actually a really strong connection point, which led us to a brand new research question that all three of them are now working together to tackle. And so it really creates a lot of opportunities to make sure that you are looking at the business from every single angle, and then elevating the work of others, not only yourself.

So as you start to put into practice these techniques, hopefully, you're going to start seeing impact when your research is being cited when you aren't in the room. You know, it's tricky to say that you're going to see your impact when you aren't in the room. But some of the ways you'll start to notice that this is happening. Other people are going to be citing your research for you. Maybe you're taking a look at a document or you know any work that somebody else has created, and they're referencing your research as an input to the decisions that they're making. Maybe you are in the room when somebody else mentions that you've done this great research, and this is what they learned from it.

The more that other people start citing your work, the more you can trust that this work is being cited when you're not even there to speak to it. And a lot of the ways that this can happen is because you are citing your own work as often as possible. The next thing is that you'll find that your work is sticky. And what I mean by sticky is that it's something that's referenced continuously and often, people are coming back to take a look at it over time. It's not something that just has an immediate spike. And then it kind of dissipates. But it's something that people have found value in this and they want to keep referencing it and making sure that it's staying, I guess top of mind for them.

This is actually a place where you can start to use numbers and metrics to measure this impact. For me, at Uber, we use the Google Suite. And so I rely a lot on the Google activity dashboard to see who's been viewing my report, how often has it been viewed in the past few weeks or month? Is it something that had an initial spike, and then people have kind of forgotten about it? Or are there consistent viewers over time? And so trying to find ways that you can really make sure that your work is sticking for the people that it needs to stick with the most and then also start taking a look at where is it being shared, which leads me to my last point that your research is starting to impact the business and not just your team.

And so you're connecting the dots, you're finding where your research actually has a greater impact to other parts of the business, not just your own. Part of this comes from all those strong relationships that you've built that foundation that we talked about from the beginning. And you are maybe even receiving more research requests beyond your immediate team. And so you're starting to see that the impact and reach of your your research is just getting broader and broader with every project that you're doing.

So thank you all for staying with me, I know we have covered a lot of content, I just want to close with a few final thoughts around how you can ensure that your research is having the maximum impact possible within your team and organization. So we touched on three techniques for increasing your impact. And it's important to know that these techniques are not meant to be used in isolation, it's really the combination of all of them together. That is the force multiplier for that amount of impact that you can have with your team's, impact really tends to build upon itself. And so setting a great foundation means that you are really setting yourself up for success and maximizing the amount of impact you're able to achieve.

And then finally, just because you can't easily measure it, doesn't mean it's not impact. So start to take note of all the different effects that you're having on your team, no matter how big or small. Check in with yourself every couple of months. And just take note of all the different changes that you're seeing and, and the types of like subtle, maybe behavior changes, maybe you're noticing that the team was adopting a certain language or you had that one stakeholder come to you with a research question instead of a method. Take note of all of these because as you're in it, you may not be able to see the change happening. But upon reflection, you should really start to notice that you're having a pretty great impact on your team and the business overall.

So thank you so much. Thank you to UXDX for having me. I hope you had a couple interesting tidbits that stood out to you that you can apply back to your work. My name is Kendall Avery and we are hiring so please take a look. uber.com/careers Thank you so much.