Efficient Teams Do Not Happen. They are Designed. It's called DesignOps

Hi everyone. Thank you so much for having me today. I'm absolutely excited to be here and to be able to share with you, my experience.

I'm Patrizia Bertini - Pat. I work at a company called Babylon Health. I'm an Associate Director of Design Operations, which means I care for my UX team. I care about processes and if I had to define myself in three words. I'm an extremely curious person. I definitely have a non-linear but extremely fun career. So, I started with accessibility and then I moved into research and then I moved into privacy and then I moved into research and now design operations. And this is because I love experimenting.

So, Babylon Health is a fast-growing company and we are hiring a lot of people, both in UK and in the US. So, just to let you know, well, it would be keen to talk to you if you are interested in a new job.

But let's get into the topic of today, I would love to ask you to raise your hands if you know what is Design Ops? I can't. So, I just assume that we all need to start from a common ground and to make sure that we all speak the same language. So, let's try to understand, and let's define what is design.

For the sake of time and just because we need to move to the operations part of it, let's just assume that we all agree on the concept that design is not just what it looks like and feels like but it's how it works. So, it's much more a functional aspect. It's problem solving. So, if you agree on what is design, the question is what is operations? Now operations have a lot of definition, but operations and the definition that I like the most its operations: transform, the existence, the resources, the data, the context into a better at design output and result or outcome and why operations operate this transformation, to create value to the customer.

So, if we start looking and thinking about design operations, we see that it's about transforming how design is, works, operates to create value to the customers. But what are the customers? When we talk about design operation, the customers are not one, not two but three because every act of design operation, it's a balancing act to serve, empower and enable. Three key stakeholders within your organisation.

One of course, it's the design team. I mean, every design team which includes content managers, content designers, researchers and design operations folks, design system people, designers, product designers, UI designers, you name it. Those are clearly the ones that design operations focus most because we want to make sure they are enabled to do a great job. They have everything they need and they can do the best job they can, because they are empowered by the organisation and by the design operations team. The second stakeholder is a design leader. We operate with design leaders and design operational work with them in conjunction. The big difference between design operation and design leader is that the design leader is the person that has the vision and the strategy for the product and the service that you are designing. Design operations is way more focused on ensuring and making sure that this vision, this strategy that the design director has in mind and in vision with the product teams can come to life. And then the third stakeholder is the business. Because there's an increased need to prove the business value of design to the business. And design operations are exactly the function that actually connects the design and the business and makes it most of the business resources to create the biggest opportunities for design to deliver the biggest value, the biggest outcome, the highest quality of a product and experiences that they can. So everything in design operations is to try to serve and create efficiencies and create value to all these three stakeholders.

But actually, what is that design operation does? That's another triangle for you. There are many ways to look at design operations. This is one way it's not the better. It's not the only one. It's my way. Design operations operating and working on three levels. As I was just saying, we need to prove the value of design to the business. So, there's a part that its very, very strictly business operations are everything that has to do with business budget management, spending optimisation, overviewing and reviewing the policies. We're managing the procurement contracting and negotiating agreements with vendors onboarding and affording vendors, making sure the IT infrastructure is working and everything is assigned, managing licenses, assessing and calculating their return of investment of every tool, sequencing and deciding if we need contractors or full-time managing all these aspects that it's kind of basic around the business, but it's where a lot of efficiency happen.

Then the second aspect, it's more about. The workflow and the ways of working and how design operates. So, it's literally end-to-end design processes. It's all about looking at how these tools are actually flowing within the ways of working, how they are integrated in the processes. How is the design team working with the cross-functional teams? How is the collaboration with engineers, with products, with marketing going? Do we have the right processes? Automatics? Do we have the right data government and the data governance? Are we helping the teams with the right design system who is managing the design system? Are we quantifying the value of design system? What are the design standards, the playbook, asset management and knowledge management? So, the second big pillar is everything that actually has to do with hands-on design and design as a collaborative participative discipline that works with critical stakeholders and cross functional partners.

And then there's the fun bit. The people operation, which includes a career progression as key metrics, assessing how we can help people to grow assessing what is the team culture, how we can share more experiences, how we can grow, how we can onboard people and offboard people when they joined the company, how we can help the team to be happier because at the end of the day, design operation does all these to make sure that designers don't work long hours. They are efficient and they can deliver great work and great value to the business and to their leaders.

So, we have seen the design pillars. So, let's get a little bit more into details because the title of my presentation it's about tangible benefits and outcomes. So, how can these be tangible?

DesignOps can have a massive impact on you, your organisation and your team. And there are two elements that actually determine the impact of your design operations practice: efficiency and efficacy. Now they may sound the same, but they're slightly different and that nuance is critical.

So, efficacy is all about doing the right things, the focus of efficacy is about behaviors. We want our design teams to do the right things, so do research and make sure that actually we test hypothesis in a lean environment. And we get proof and we know how to progress and we have a lot of ideas to test. When we talk about efficacy, we start talking about metrics like empathy, user engagement, ideation and experimentation cycle times, the composition of the teams, having the right behavior, the learnings, the skills, the distribution, and designers satisfaction and retention. When we talk about efficiency, though, we do talk about doing things in the right way to deliver a product. There are a lot of weights, there are a lot of tools. There are a lot of processes, iterations that can happen. So, efficiency, its very process focused and it's all about measuring the return of investment of the tools. It's about measuring the time to deliver, it's about measuring the lead time, the productivity, the end-to-end time, the quality of the outcome, the number of iterations, the percentage of rework.

So, designOps really looks at these two elements because they can influence both. And it's important to note that actually efficiency is not a qualitative and subjective dimension. You don't measure efficiency by saying, 'Hey, we did great.' Great can't be quantified. Efficiency is always quantifiable and it always has objective measurements. So, you need to say and when you think about efficiency about processes, what are you gaining? Are you gaining quality of the outcome? Are you gaining speed to delivery or are you gaining an increase in the return of investment in the resources that you have? If you can't quantify, that means that you don't understand the problem properly yet.

And today what we know from a lot of research is that the team's efficiency is way under the optimal stage. So, there's this research from a work from last year and apparently 60% of knowledge workers time is spent doing something that it's not related to design, it's quite a lot. And on the top of that, another research from the same from last year, estimated that the productivity time is less than three hours. So, that's really something, why actually knowledge workers designers are not able to be productive more than three hours a day, more than 40% of their time. What is preventing them? What are the issues? And now a lot of invisible endemic make in efficiencies that have tangible effects. We may not notice and we may even consider these things as part of the overall process, but generally we may kind of start looking at the end-to-end ways of working or the cross-functional collaboration is not optimized. Perhaps in the team, there are people using different tools because hey, I'm coming from a company and I really like X, Y, Z, and I can't live without it. That creates delays in the process in integration, perhaps there's no clear way to get templates or the templates have been modified so many times that that inconsistent. There is poor engagement mode. The briefing process isn't efficient or the collaboration is broken or the Q&A is broken, but actually all these things. And I'm pretty sure, I mentioned at least one thing you all experienced, will have consequences, like long working hours because 'Hey, if something is broken, you will have more meetings. You will have free work to do.' Delays for quality of the outcome, high team churn and low level of engagement. So, it's all there. We have all this.

So, let me take you through a couple of case studies to tell you how you can do, and hopefully this will inspire you to start tackling efficiencies and efficacy in your teams.

So, let's start on a story about improving designers experience, improving efficiency, remember efficiency, it's all about processes. So, I don't know if it ever happened to you, but in my teams, one of the things that always comes as a problem is too much work, long hours of, I can't finish everything. No one understand how busy I am. I'm so busy. I can't do everything and blah, blah, blah. The thing is that a lot of the problems that people tell you those are symptoms. They're not really the problem and the real cause of it. Because the real problem, at least in my case, was that why will people working long hours? Simply because a process was broken. In this case, it was the recruiting process. So, if you think about designers having to do iterative testing with users and customers every two weeks just to adapt to the sprints. We realise and I measured that actually to get five to six users every two-week costed, in terms of time, 2.5 days of designers' time, which means every month there was the equivalent of roughly one week worth of designer’s work spent in non-design time, they can start seeing where these 40% or 60% of inefficiency is going. So, that actually of course, had causes, had the causes on long hours, had causes on people feeling overwhelmed. Actually, this is not the job of a designer to recruit something because this in efficiency had both consequences on the business and on the design team, because it was long working hours, perhaps these cause people to not have so much time to analyse the results from the testing, perhaps they reduce the testing or the number of participants of the quality of the participants. And of course, if you look from a business point of view, what we were seeing was basically, we were wasting 1.5 the equivalent of a full-time employee of 1.5 head counts. So, 350 working days. So, we should also have impact on costs because we were using agency and they cost too high was anywhere between $100 to $360 per participants. So, because that, of course you, you go fast, but you can't really go fast because the lead time with some agency was out to 25 days. And of course, these calls that we are really didn't test as much was it use the number of tests is, and in some cases there may have been some creative and original ways of recruiting that were not necessarily GDPR compliant and this is what actually we mapped out. So, we may have used agency, friend of a friend, recruitment apps, internal list. Don't do the internal list.

And there was no real way to solve it. We started with a hypothesis where we started to saying, if we outsource the research participants to let's say to an internal desk, let's create a new role, someone within the team, because if we were wasting 1.5 years the equivalent of a resource for a year and a half, wow. Probably there's budget to get that resource seen. So, we may reduce designers, non-design work. We will reduce research lead time, increased participant engagements to do more research, reduce the spending. So, for nine months, we ran a test in the UK to kind of see, 'hey, what is happening? Can we actually get someone to run an outsource and manage the end to end the commitment process?' So, we worked with this resource to optimise and redesign a platform that the company had. We reviewed the overall recruitment workflow. We work with all the cross functional partners, like analytics to understand how we can operate within GDPR. We recruited and trained this person. We rolled out the surveys and we started measuring.

Remember that I told you that efficiency can be measured and it's quantifiable. So, what is the data? What is the cost per recruit? How much time are we saving of designer's time and how many people are we talking? Are we actually able to engage with more participant?

Yes, we did. So, after nine months, what we learned was that actually we were having a massive impact. So, after the first year, when we rolled out the service with 65% of capacity, we gained the equivalent of two designers time. And on the second year on a full capacity. So, our recruiter fully operational, we gain the equivalent of four designers time and we have enabled with it's called ambidexterity. So, the organisation can invest equally on innovating and in delivering because we had extra time without having extra head count.

It's about using your designers better and making sure they don't work more. We increased the number of participants in research by 300% and cut the cost by 55% with some kind of peaks in some regions and in some, some markets of even 70 plus percent. So, and of course, attrition in the team went down because if you are happy and if you are engaged, why would you leave? Their engagement score increased and the lead time for research again, it was faster 65% faster compared to before, but there's also the case when we want to improve efficacy and when we want to improve it, because it's all about behaviors, right? So, again, I don’t know, your organisations and your teams, but my teams used to have this kind of a Cinderella syndrome sometimes.

Product really don't listen to us. We go to the meetings with all our wonderful persona. We know it all. We spoke to hundreds of customers and users. We know what the user wants, but they don't listen to us. And yes, that's absolutely true, but that's not the problem. It's not that design is not important. The real problem is that design and design thinking. It's wonderful. It's critical and designers are empathy champions. The problem though is that in the organisations, decisions are made with data, not with empathy. So, it emerged that in this case, designers were not really familiar with analytics. So, going into your platform, looking at the journey from a user behavioral point of view, and being able to understand what are the opportunities in product. We needed to teach designers to talk data in order to be able to become stronger in their approach and making sure that they could have convert informed conversation with a product partner so that they were all talking the same language and they were able to influence designers and product. This is because it's important to start this shift from T-shaped designers that focus only empathy are really great at the craft and they excel in doing design and they execute the brief and they deliver for product teams. And there are systems quality into pie shape designer. Designer's that understand user's behavior, they make decision based on data. They discuss as partners with product and they deliver with product and make decisions with product and use not quality of deliverables as a metric, but impact of the solution on the user behavior. That was the challenge.

So, the hypothesis was you know what? If we expose designers to an all the cross-functional partners to data driven or data informed decision-making training to make sure designers can understand and speak the same data, language that the PM's and PO's talk, then probably we can help them to have better relationship, better focus and better decisions that actually benefit both the user and the organisation. So, for six months, we organised weekly 4.5 hours of training with both subject matter experts and vendors. This has been made with all the buy-in of all the leaders to make sure that this time was blocked no meetings, no nothing to make sure we had people joining and taking advantage of it. We measure also efficacy, it's not just the efficiency, we can also measure behaviors and to do that, what we did was a survey to start capturing where we are and monitoring the awareness that a bit of confidence in data, the level of confidence in the analytics tool, and then creating a whole program and running these sessions of every week, 90 minutes with someone that could inspire them on how to think with data, how to use data in your day and how to make the most.

So, in six months, the results were more than I was expecting. So, we had an increase of over 110% in the engagement with analytics tool. Lots of people didn't even have an account on the product analytics platform or on the web analytics platform. 84% of the designers started using analytics and there's been an increase of over 70% of projects in analytics platforms and tools. We also measured the confidence in utilising data and we had peaks up 27% increased confidence in talking and using and referring to data in analytics and 29% confidence in using the tools and the queries hit a massive spike plus 1800%. I forgot to say that.

So, just to close, I just want to kind of tell you why design is important, then what you can start doing and how you can start thinking efficiency, efficacy in your organization to bring those tangible results and to bring this kind of story.

So, first of all, what I shared with you, if my story, it's not your story. Which means every design team has inefficiencies? Acknowledging that your team, no matter how awesome it is, has room for improvement is the first step because denying that we can all do better, won't help anyone. And then you need to understand your team and you teams’ pain and issue. No two teams are the same and what I did for my team was what was right and a moment in time, because all the inefficiencies change constantly. So, they will evolve together with the evolution of your team of the organisation, of the product strategy. Inefficiencies don't disappear they are there, but you need to start thinking, analysing where they are and how they can be tackled together with your partners in a very systemic approach, because if you help your designers to work better, you will also help your product teams to work better and your analytics team and everyone. So, it's all positive reinforcement and everything and measuring efficiencies. Every project you start with design operations has to have metrics. And if you have not convinced, and you don't know how to measure impact, how to measure efficacy or how to quantify efficiency, keep trying it. You just haven't thought through appropriately yet, and you haven't framed the problems right. The moment you have your metrics, you're ready to go, because it would be able to prove the value of design to your business and the value of designOps to your design organisation. And the great thing of solving problems for a team is that there's a domino effect. You saw through your designers, but you help other teams. So, these will also have to create better relationships with your cross functional partners. It will help you to have better experiences that actually will have users have a better experience. So, and the final recommendation is always remembered teams are leaving old ideas, they change, they evolve constantly. So, the one single most impactful thing and design operational personnel should do is listening. Never stopped listening because what you hear is what will happen on the ground and what will influence, not just the single designer, not this squad, not the team, but their organisation, the business and the user.

Any questions?