Breaking Down Complex Problems - Implementing Change

Hi, and welcome to my talk of Breaking Down Complex Problems. The reason I've chosen this topic is I believe it's one of the most important skills that people can have when working in software developments, because often the problems that we are dealing with are complex and it's a different way of approaching it than will be done in a traditional waterfall way. So, what I'm going to do in this talk is explain the difference between the different types of problems that you can encounter when working in software or in anything in general. And then how do you approach those from a complex perspective and I'm going to give a couple of examples of how that happened in practice. The first example I'm going to give is let's say you were working with some software to track the progress of building a software product. And one of the things that often comes up is a Kanban board style of item, you want to move tasks when they're completed into a done column. That's very simple. There's no ambiguity here and it's very easy to develop. There's no real complexity there. We can make it a bit more challenging by saying, okay, once it goes into done, let's kick off some automation. So, we might email some people. We might change some mandatory fields on it. We might auto-populate some fields. We can do a bit of workflow around what happens when it goes into done. And that requires, that's a bit more complicated because it requires analysis of what needs to be done, who was impacted, et cetera. A complex challenge will be to create a system that optimizes product development. So, those first two examples, that's what people are used to building in software. It's okay. We need to build this workflow. We need to change this, but how do we know that's really optimizing product developments or how do we know that we're not just automating an inefficient process? So, that's the difference between a complex challenge of creating a system that optimizes it versus just ticking the boxes and delivering a system that might not deliver as much value as we could. And then to finish it off, there's chaotic. It will be the fourth category where nobody, there is no system, nobody knows what's happening, everybody's running around and there's no order. So, to classify those examples, there's a very simple problem. A complicated problem that it's a bit more challenging, but with a bit of analysis, you can figure out what needs to be done, a complex problem, which is more challenging again, and then chaos. So, to those of you who are familiar with the Cynefin framework, this will look familiar because this is the Cynefin framework, which is a way of sense-making around problems that you encounter. So, by figuring out which box it belongs in, you can determine what's the best way to respond to that. In software development, we tend to be focused on the complicated versus complex problems. With a complicated problem, waterfall is the best way of solving this, because what you need to do with complicated is you need to do a lot of upfront analysis. Figure out the full problem space, the domain, and then solve it. With a complex problem that won't work because with complex, we don't actually know what we need to do. We need to try things out, see what the response is and then pivot based on how, what feedback we get. So, with that, I'm going to give a couple of examples to try and get out of the theory and into the practice of how to do breaking down complex problems. And the three examples. I'm going to talk through one. It will be on people, the second up process and a third on technology. So, on the people's side, I'm going to talk about customers because when we're building products, that's what we're building for. We're building for our customers and customers are complex. Just take the example of what do I want for breakfast today? If I have the exact same thing every day, I get very bored. So, I like a bit of variety. So, it's next to impossible to know when emotions come into it, what exactly people are going to do until you give them something to react against. The example I'm going to give is airspace, which was a premium seats short-haul offering that Aer Lingus released back in 2019. I believe. I have the product is basically the no-frills locals carriers had changed dramatically. The short-haul model in airlines used to have the economy part of the cabin and the business part of the cabin but no frills airlines got rid of the business cabin and it's much more efficient because load factor is incredibly important in airlines. And that's how many people are in the seats on the plane. Because if you fly with empty seats, you're losing money. And by keeping all the seats the exact same, you could then just drop your prices, change your selling strategy depending on how many seats are booking. And it's become the dominant model in the airline industry. But because of that, there's now a gap. There are the people who want to pay for a more comfortable flight experience but with the no-frills model, they don't have that option. So, this is where this product came in. Basically, it is the same seat, but you're guaranteed that the middle seat would be free. So, you have the armrest space, which makes a huge difference for those of you who are familiar with flying back when we used to be able to fly, But this challenge, what I did with the team was I said, okay, let's approach this as if it were a complex problem. How would we do this? If it was a complicated problem, we would do it in a traditional waterfall way. So, we would do a detailed analysis phase. We would look at, okay, what are the different elements you need to search for your flights? So, okay. Let's show the flights that allow this priority boarding or the, the, the airspace seats, the bookings listening. We need to be able to show when you see all of the different options, do I want a regular seat? Do I want the area space seat? The seat map would need to be changed to show. Okay, we're going to hide off these middle seats. There'll be rules for seat selection, et cetera the extras. Well, if you're getting this seat. And it also included meals and things like that. Well, we should hide those from the extras because we don't want you to accidentally buy those again, or just inconvenience you by showing them and then managing it. So, if you want to change your flight or change your seat, okay. There's a lot of rules there as well. So, in a traditional way, we do a very long, detailed analysis phase to figure out exactly everything that needs to happen, but let's see how we would approach this? In a complex manner. So, still looking at those core areas, which is there's a journey there. They think about flying, they search, they book, pay for any extras that they want, pay for their flight and then fly. Well, the first thing would be to validate the demand. And so, I'm a big fan of fake door options, which is where you give a button on the website going, would you like to upgrade to get this option? And then you just show them that, Oh, sorry. We're booked out at the moment. And you would just have that run for a few days as an experiment to see how many people would actually be interested in purchasing this, because you really want to validate if there is a demand before you spend any time or effort in building out this complex system. Second then, okay. So, once we validate that there is demand, what is the simplest product that we can build that can satisfy this? So, if we look at the fly column first, that's probably the most complexity here because we have to train all of the staff on how to manage this and how to actually deliver the dle's empty middle seat, the meals, et cetera that come with this product. So, that's a lot of complexity of training those people, but I'm just going to brush over that because I want to talk about the software side of things so the booking, the seat map is probably the most complexity. But what if we just got rid of the steep map? What if we just said, if you book this product, we're going to give you one of the airspace seats that would make it really quick and easy to develop this because we're getting rid of one of the most complex parts and for upselling, all of the extras that kind of, Oh, we have to take some of them off sale and some of them on sale. Why don't we just skip over the whole stack? Because again, we're not trying to get this perfect. We're trying to make sure that there is demand for this product and the MVP versus the fake door option is when people are going to be paying. So, it's that extra bit of making sure that there is real demand. Once we validate that, then we can upgrade our marketing, really start promoting this add in some extra. So, give the seat map because it is a better experience. Then we can increase the route options. And the reason why in the MVP, we start with a single route is because we don't want to show this to absolutely all of our customers. If there is a risk that we're going to pull this because after each one of these steps, if it doesn't deliver on the value that we're expecting, we should be considering pulling the product or changing it dramatically. So, by limiting it early on, we're limiting the number of customers. It could be impacted. At the end, it would be testing the extra. So, put back in those extras to show different options, see which ones people are interested in. So, that's how we would approach it as a complex problem. Now, the team had been very used to working in a more traditional way. So, the first reaction we did this as a team, we got everybody together into the room and said, how do we think we can approach this? And it was a great experience with all the cross functions, collaborating quite well, trying to say how we think we should break it down. But at the end of it, the response was this is really inefficient because What we're doing is we're building something and then throwing it away. We're building something, we're throwing it away. So, the fake door stuff, we throw that away. As soon as it's done the no seat map. Well, we're going to throw that logic to skip the seat map away. Once we introduce the seat map and the single route versus all the routes, we're going to throw that logic away as well. So, it's very inefficient. We're, we're wasting time when we know that this needs to be released for a lot of, excuse me, a lot of different routes. So, why are we bothering with all of this wasted effort? And the example that I use for the airline industry is, well, we don't know that we need all of these things. Until we've gone through these steps because maybe we don't, we're assuming that there are the demands but we don't know. And priority boarding is the example that I like to use because this makes sense as a product that means it's referred to as its own bundling in the airline industry. A traditional full service carrier offered a lot of perks with a business seat. You got the better seats you had the priority boarding, you had the lamb access, you had the meals, you had the free drinks and on bundling is basically where they separate all of those intersect per products and sell each one because maybe somebody doesn't want the lounge, but they do want the nice seat. Maybe they don't want the meal, but they do want the priority boarding. So, priority boarding was set up as a product and it didn't deliver as well as expected and people weren't a hundred percent sure why, but RyanAir figured it out priority boarding. Once you have a guaranteed cease, it isn't actually that important. What's important is that you get your bag on board and that's why people were buying the priority boarding. They started talking to their customers and people were saying, well, I'm buying this because I've had trouble in the past, if I board lays or can't get the bag on and hustle into the holes and then I'm delayed. So, they repositioned it to be about getting the cabin bag onboard. So, it's a very similar product, different way of how it's presented in the design and how it's managed at the Gates as well. But this meant they had to go back and revisit all of the work that they had done. They had to change their designs. They have to change some developments. So, all of this rework happened after they built the product that could have been found had they built it in the right way the first time. Luckily they were able to repurpose the product in other instances, and unfortunately, as an industry as a whole up to 70% of features are often, very rarely used in products. So, the first tip, I guess, of breaking down a complex problem is to validate the need. We want to make sure that we're not building and then supporting features that aren't being used and aren't delivering the value. And the reason why it's complex is because customers are complex. So, let's talk about the process. When I joined Aer Lingus, one of the challenges I had was to shift from working in more waterfall away to working as more Agile product teams. So, you draw up a nice process flow and it's not that it doesn't look that difficult. And one of the challenges is some of the people in the organization where we're treating this as a complicated problem versus a complex problem, because they were saying we had the waterfall process, just write up this product team, agile focus and process. And we'll just shift over to that. And we'll move on from there. So, I said, okay, well, let's just talk about what that will entail. So, again, it's a user flow, but it's a slightly different user flow. This is the flow of trying to get work done in an IT organization. So, finance, who will fund it, you govern your team, analyze dev and then run it in production. So, what we're saying is, okay, well, let's shift some, some of these things around. So, in finance, we need to change it from funding projects to funding outcomes. So, we're going to focus on a particular area and we'll improve that. So, the governance that needs to change because we're no longer tracking time and cost we're tracking. Well, how well are we delivering against that outcome that we had said and we're our team structure is changing quite dramatically because we're moving from those kinds of teams that just get together for project disband project disband to, well, let's give a team dedicated time. Long-term time to focus on this outcome. So, that changes a lot of how management works in those organizations, because a lot of management goes into allocating and moving those teams around. Then the analysis, we move into that iterative example that I gave before versus the detailed requirements. This makes it complicated from an architecture perspective, because if we have all of these teams working really quickly and iteratively, we can't have a centralized architecture function that takes a becomes a bottleneck to the process, essentially. So, we need to create a modular architecture that each of the teams can work on independently. And then we need automated releases. So, that will mean that we're touching a huge number of people in the organization. And once you mentioned people, it becomes a complex problem because now we're talking about people who have different skill sets, they have different innate biases, you have different experiences. Some people could have had a really negative experience with failed Agile projects. Other people just don't believe it will work. So, you're dealing with a lot of complexity here as you're trying to change a lot of people. So, rather than do that, it's well, let's treat this as a complex problem, so let's not try to change everything at one go. We'll try, just get one team, we'll fund one pilot team and try and get them working into an Agile development approach. So, we'll shift from time and calls to we're not quite. Doing governance on value, but we're looking at release frequency. We want to be able to be releasing quickly so we can test ideas. We'll try that cross-functional team, we've tried the breakdown of work and we won't be able to fully isolate the architecture instantly. So, let's just look at horizontal stack and I mean most organizations have your UI layer, your services there, your database there. So, let's just pick one of those horizontals because it's neater and it's, it's easier to change quickly. And then the ability to release each of those horizontal stacks separately would be a change. So, we start there and then we go to continuous integration. So, we merged dev and QA automated testing. We need to adopt some practices like trunk based development and feature flags. Then we will enhance again to continuous delivery. So, now, if we're able to be releasing very frequently, let's introduce some new governance metrics around how quickly we can go from an idea to production. And we want to focus on reducing that. So, again, we're not really talking about value yet, but we're changing it to be able to be a lot more. We can spin up automated environments and do a lot more, just the technical challenges. Now what's wearing the ability where we can release really quickly? Now we can start looking at values. So, let's fund the outcomes, have our metrics aligned to those outcomes. We'll merge product and UX into this previously. It was very it driven whereas now we want to be looking at the value of what we're delivering and doing continuous discovery. I'm breaking it into vertical products. And so, what I mean by that is you get the full vertical stacks. And instead of just the UI services, a database and a team owns. All all in their area. So, they have no other dependencies on other teams to release. And then you want to scale that model. So, we've talked about doing small pilots, so rather than trying to just do a big bang of change the whole organization at once we change each team as we go. So, the point here is we need to make it easy. If it's too hard, it won't work. People can be overwhelmed by the challenge of trying to break a complex problem. So, give people the big picture of drawbacks, a high level map of will. These are the steps that I want to go through, but let's just ignore everything below. Step one. Let's focus on step one. Let's get that right. And let's see what happens then and we can react from there. And so, you do really want to get people bought in again, like I said, with the last one, build this map with your colleagues. Don't try to build it independently. So, now you're looking at technology. I often say people could get a bit frustrated at this. I always say technology is the easy bit and it's customers and people are the challenging pieces, but w which can get a lot of technology people riled up, but technology is complex as well. And I'm going to give an example here of what was it seemingly complicated problem and why it became a complex one. So, in right RyanAir in 2013, they had the same looking website that they had had for maybe 10 years. It worked, but it was ugly. So, in 2014 they did a UI refresh. It's actually the exact same website. All they did was they changed basically the front-end layer just to tidy it up, make it look a bit modern, but the challenge that the company was facing was that it was a tangled spaghetti mess beneath the hood. So, while the new UI looked very fresh and modern, it was very difficult to get any changes done because if you change one area, it was causing ripple effects everywhere else. So, the idea was let's do a rewrite but we'll turn it into a very clean services model so that we can change the services independently of the UI. And we can bring up mobile, which is gaining popularity then as well. So, in 2015, there was another rewrite, but this one was much deeper and it actually involves changing all of the layers underneath the UI as well. So, how is it. Got broken ice, similar concept. You had the user journey of the marketing pages would just trying to get the bad for it. You search for a book, you check in, you might manage your flight, change your seats et cetera. And I've thrown a profile in there because it's a good example where if you're trying to build a user journey, sometimes you'll have elements like management and profile, which don't flow fully into a journey. But it's okay. Just to attach those on where you think it best fits and because nothing is ever going to be a perfect clean journey. It was tackled as a complicated problem. Next just let's just do a full rewrite of what we're doing. So, the marketing pages we'll introduce a new content management system and because the existing one just isn't fit for purpose, the search, we're going to write search services. Checkout services, check-in services, managed services, profile services, and how to predictable what people are used to in software, big large scale software rewrites. The date just kept slipping. So, what happened was we said, okay, well, let's do an initial release and there was a lot of discussion. This took a very long time to drop, manage into a second release. So, the idea was if you could get through your booking and then we can sell the tickets, if you want to manage it, you go back into the old website. So, you, you see quite a dramatic difference in how it looks and feels, but the idea was. We had fast follow with the managed services and what actually happened was as soon as we released that, we realized, okay, there's a big performance problem. And performance is one of those challenges. Yes, you can do a lot of performance testing, but even your assumptions about how use people are going to use it can be incorrect and that's what happened here. Usage patterns were slightly different. Therefore the optimization that had been done in the services wasn't fit for purpose. Everything just dropped and it was focused on get the performance problems resolved, which the team did a phenomenal job. Then the next problem was because of the redesign of the UI. And it had been made so much easier for people to check out. Unfortunately, they were checking out with buying, are they buying a lot of ancillary services and the bottle for a low-cost carrier is you make a lot of your profit, you lose money essentially, or close to losing money on the seats and you make up the profit on the ancillaries. So, it shifted then to, well let's focus on making sure that we can get the revenue back right on the ancillaries. And once that was done RyanAir introduced at this time the my Ryan Air product which is you need to log in in order to be able to purchase a flight to Ryan Air. And one of the challenges was that okay, well, this was a brand new product. We didn't know how customers are gonna use it. So, the focus was on well, let's improve those profile services. And then a year later after the initial release, it came down to manage. So, the story here is that what you think is the most important might not actually be the most important because you'll only find out what is the most important when you've released early and your customers tell you what is most important. So, what's key here is to make it flexible. So, as I said in the last step, making it easy, you put out your map of what you think will happen but you need to be flexible because after you do your first release, you will learn what's most important and you'll need to be flexible to shift. And so, everything is priority one until you get customer feedback. So, to summarize how you break down a complex problem. The first thing is to validate the need. You can't treat it the value is assumed and that's what happens in a lot of software products. People assume somebody else has done the analysis. I just need to build this feature and it will deliver the value. But if you talk to the people who've done the analysis, they still have a lot of open questions. So, they're eager to do the the iteration of the testing. So, if you can get together and just work together and do that to validate the need first before moving out, that's step one. Step two is make it easy. It's very easy to be overwhelmed with a complex problem. It seems too hard like that example of process change in an organization but that's where if you can make it easy, if you could make it clear step-by-step while also showing the big picture, because the pushback that you'll have, if you tried to start too simple is well, what about, this problem and what about that. So, by having the full picture, you can say we've considered those problems, but we're just not going to tackle them right now. We're going to start with this and then we'll move on and we'll get to those eventually and then make it flexible because it is going to change no matter what you think. Once you have a complex problem, you're dealing with people and they have complex needs, things are going to change. So, I hope you found that valuable and I will be around to answer any of your questions and I can be contacted on Twitter at RoryUXDX. So, thank you very much.