The Challenges And Pitfalls To Avoid When Shifting To Microservices

Hi, I'm Dana Lawson, VP of engineering at GitHub. And today, yes, today—well hopefully, you know why you're watching this—we're going to talk about the challenges and pitfalls of shifting to a microservices architecture. I know, I know, you've probably heard it a million times, “Microservices the future, monoliths are dead.” I'm sure there's going to be some other new boutique. Do we call it boutique? Architecture is—I don't know. We're calling them boutique today. Architectures, that somebody is going to come say, “Hey, we need to move to that.” But let's focus on the boutique one that we've all probably have encountered, and hopefully the reason why you're watching this video. I'm going to share with you some of the challenges I personally encountered when making the shift. And luckily enough for me, I've had more than one opportunity to try and get it right. So, how about we just dive in?

So first of all, why the heck is Dana Lawson talking to me about microservices? Well, a little bit about me. I've been in technology for 21 years. I know, that's really hard to tell. But it's true, 21 years. I've gone from being called Assist Admin to an SRE, to a DevOps Engineer, to a Software Engineer, to a Products Engineer, to who knows what type of engineer. At the end of the day, I’ve seen it all. Well, I think I have. Maybe not all, but most of it. I've had three monoliths that have been given the edict of “Turning them into microservices,” two in Ruby, one in ColdFusion. You heard that right, folks. One was ColdFusion. And then, four CTOs, over that timeframe, telling me the benefits of microservices. Now, I'm not saying those CTOs read hacker news. But I'm saying, yeah, I've had four CTOs come in and then say, “We're going to microservices.” And I'm like, all right. And then, probably manifested over a hundred plus problems of trying to dive in there without some of these practices and principles that we're going to talk about. And hopefully, you'll avoid them and not do what I've done because it really sucked.

So, you've probably got the—like I said, the edict from somebody above, or maybe even from your own self that said, “You know what, we're going to go from our full-stack, single application architecture or monolith to this beautiful diagram of clearly representative services and products.” You know, one of the goals of microservices, as we all know, is to really have the sophistication and the wall of garden between the different parts of the customer experience, the user experience. Because really, us engineers can get in our own way. When we think about shifting to microservices, the first thing—well, I don't know—maybe the first thing I always say was like, “Technical debt, it's going to screw us or capacity is going to run out.” And so, we come up with this really flat, beautiful kind of architecture that allows us to, you know, move fast. Well, that's the goal.

And then, reality hits. It actually looks more like this. No, that is not my bowl of spaghetti and meatballs. That is an actual true dependency graph of a microservices architecture, because that's the reality of microservices if you don't avoid some of these common, common pitfalls. I know it's going to sound like common sense, and that's why they're common pitfalls. But the reality is, we get shortsighted sometimes when we're close to things. And so, really trying to avoid the beautiful bowl of pasta dependencies is thinking about how you organize, how you're going to orchestrate, how you're going to maintain your life site, and then once again, how you're going to continuously improve. Those are the benefits of microservices. It's not just cause of technical debt and capacity. If that's the reason you're making the shift, don't do it. You don't need to. You know, I was joking at the last UXDX with Chris Slowe from Reddit, and we were doing a panel on microservices. And really, the reality is, once you have a monolith, you're probably always going to have a monolith. It's what amount of a monolith do you not need and where does the extraction points have to happen? Because, you know, you're never going to get to an architecture like you saw. Well, maybe, I don't know. If you do, call me and I'd love to see your presentation.

So, we're going to go over these three, actually four key practices of counteracts to the pitfalls that I've encountered. And I touched on them a little bit, the first one being design, constraints and collaboration. Before you even start writing a line of code or changing up your services to be able to be in this new architectural pattern, you really need to think about how you’re organizationally designed and how your teams communicate and collaborate. Are they operationalized? Are they used to being on call? Do they have the sense of ownership in agency? One of the benefits of a microservices architecture is so that you can push the accountability down to the developers. And hopefully, you know, increase the pleasure and experience of not only their productivity, but your customer's experience.

And then, we're often going to not having a plan or deployment, all these services. With one monorepo, one monolith. Cool. You knew that you commit to master and it goes in and boom, which was probably a challenge within yourself. But now, if you don't think about how you're going to do this pragmatically in your microservices, then you're defeating one of the huge goals at it. So really, that's a huge pitfall. Not putting the energy and how you're going to create your whole development, automation and life cycle. I mean, you're at a User Experience and DevOps virtual conference. You know, DevOps is hopefully in your ethos. If not, this is something that really should be, as you make the shift. If you don't change your internal engineering practices, you're not going to have any of the benefits from this new architecture.

And then, live site operations, it's going to change. Now you're not watching one thing and you're not waiting on your 500. You're watching a whole bunch of services, which is awesome. But if it's not done in the same manner of orchestrating your deployment pipelines, your DevOps cycle, you don't really put the thought and practice into how you're going to monitor it and have that in place, then you're going to have a really hard time. In fact, I guarantee your meantime to recovery and your SLAs and SLS will go down, as you introduce a microservices architecture if you do not put the emphasis in how you're going to maintain and watch it.

And then, finally, hello, continuous integration and innovation and improvement. Having microservices allow you to deploy rapidly to have experimentation. Maybe you have an objective for growth. Well, in a monolith, it's really hard to make quick UI changes. Why do we move to microservices? Does it matter? The reason, one of the benefits is to be able to have an experience be compartmentalized, so that you can change pieces out, you can measure, you can record. So, if you're not thinking again about how you're going to continuously integrate and iterate, then you're once again defeating the purpose. Because it doesn't help with productivity, it does not help delight your customers. And you know, that's what it's all about. So let's dive in.

So, why it’s one of the pitfalls, not the first pitfall organizational structure? I know you all know Martin Fowler who has written the Bible on microservices, but a lot of his talk about Conway and Conway's law, right? Conway's law basically, in a nutshell, is we organize and design systems for how we show up in the world. So, if your teams right now are structured in a way that helps really produce a monolithic architecture and that you have the development practices in place to do so, and you don't think about how you're structured, you’re going to to inadvertently create microservices that have this same pitfalls as the current operators of your products today. I really encourage you to think this, and go and tell your teams to train. You know, when you look at—when you're looking to split in the large application apart, we just tend to focus on the technology layer. Leading to UI teams, server side logic, and the database teams or data teams, and they have these lines that separate them. And even simple changes can take a lot of time. You've got to go talk to that person, you got to talk to that person. So, thinking about how the organizational patterns are, are going to empower you to really think about the boundaries. A smart team will optimize around the things that make sense. So, we have to force them in—you know, into the examples of how Conway's law in action really improves the way that we think about our microservices, one without the other. Martin Fowler goes into detail in how he thinks about building the architecture. But it's so true. You cannot just go and do the tech. You have to do the organizational patterns to be able to promote that. Because what we want to get to is more of a world that looks like this.

Full comprised ownership. We want to take full stack teams and the business capabilities within what they own to be compartmentalized. Not in a way where there's no communication, but really so you can limit those cross team dependencies. So, you're not going and talking to the data team or the UI team when you need to make a change, it's all within your team. And microservices help that be a thing, because that's really once again, why we’re doing this. Why are we doing this? Not because of capacity, not because of tech debt, so that we can improve our customer's experience. And that's for meantime to recovery availability, clear constraints, so that we can experiment, iterate and hopefully delight. Because believe me, there's a million companies trying to do what you're doing right now. So, take the time and make sure you're organized like you want those services organized.

You want full stack teams build, own, and operate? You need to change the way your engineering teams work. And this is actually probably the hardest thing we think about going into a microservices architecture, is the culture. Now, you have teams that have ownership. Now, you have teams that really have to think about ingress all the way to egress, and egress all the way through back into ingress, and that whole communication and transaction time. And that's probably the hardest one, is not preparing them and thinking about the opera—how they're going to operate and communicate. So, don't avoid it. Don't just take what you have now structurally. If you're an engineering manager, think about how you organize. You know, is it the workflow that's going to set you up for success? If not, you need to do it. Sorry, engineers hate reorgs. Reorg or you'll have a hard time. Because if not, you're going to end up with something like this, you know. It's like, “Hey, we've got all these microservices, but I can't even do anything.” So, what's even the point? You know, microservices can harm a culture because now your developers have so much friction, and we don't want to harm them. It's supposed to be the increased productivity so that they can continue to build the features and iterate for the customers. So, think about it.

And now, as we restructured our team, we've avoided a pitfall. We also need to avoid the next pitfall, workflow automation. Hopefully, you're listening to this talk and you're at this conference because you believe in development operations, you believe in user experience. We also need to think about that in our work change internally, our software development, life cycle and delivery cycle. So in a traditional monolithic application, there's a single build, there’s pipeline and out, and the application is executable. And all these different development works feed into this pipeline. If a high-priority bug is found, we have to go and redo the whole build. We have to fix, integrate, test and then republish. I don't—you know, maybe if you're SAS, it's not as bad. But if you have a monorepo, we all know, like, you're on the hook for the deployment. And somebody introduced something that you don't even know about it, we got to go all the way back to the beginning. And one of the reasons that we think about microservices is because we want to avoid that. We want to be able to quickly iterate and fix.

Following a microservices philosophy, there should never be a long release train where every team gets in line and wait. In fact, in GitHub, we do have a piece of the monolith left. Because like I said, once you have a monolith, you'll always have a monolith. It's just, how big is it? And everybody always belly aches about having to get in line. On a microservices architecture, it’s awesome. It's like, cool, I'm going to go into the service provisioning pipeline, it's all interconnected. Boom, new service, I can do it, got monitoring. And hopefully, you've built a repeatable pattern. It can be merged test and deployed and be, as you see, it goes all the way through.

Some of the challenges are, “Hey, now you have a team that is totally responsible and independent, so they own their own build pipeline.” This is why, if you don't think about your build pipeline, your CI, your CD, how you go from—you know, “I did a code and then code to cloud,” then you're not going to produce the outcomes that you want. So, I encourage you to have a unified, automated pipeline to build and deploy services that are not hidden, so that every team knows how the system works. This will allow multiple languages and frameworks to be able to be introduced. And you've thought about it, you’ve built those patterns out. One of the pips here though, is, you know, opening the flood gates too wide [Inaudible] microservices. As we have a tendency to think, cool, now we can use any tool within our toolbox. But really, I would like to say a pitfall to avoid is choosing two, three or four. What are—what do you really have in your ECOS? What can you operationalize? What can you build and produce in a repeatable pattern, so you don't end up with dependencies you don't understand? Because once again, if you don't think about these things ahead of time, you're not going to have the outcome that you desire. So, put the energy and effort into building the CICB pipeline and limit choices at first. You know, one of the guiding principles that we had at envision app when I was there, when we built out our microservices where you can fill whatever language you want, here's the toolbox of ones that we've already built automation to workflow. If not, you're going to have to build the automation workflow yourself, but we want that repeatability. Not in the sense of like a monolith, but still the repeatability.
So, think about the toolbox. Think about your CIC, think about your DevOps. Because if not, once again, we'll go back to that comic. And we don't really even improve our deployment experience. And we haven't built those circuit breakers in place to allow us to iterate and have a great experience and in the event of failure.

And then, my third point in practices, if this again, is all new, build up the culture, build up the muscle of application performance monitoring, your DevOps monitoring. I don't care if you're using Datadog, Prometheus, Sentry, New Relic, whatever flavor version. Once again, making it highly visible, making it understood, building out the groundwork for most of your microservices and how they use, and really ensuring that teams are built there. I think it comes down to when we think about organizing ourselves and having a culture of operations so that we can produce microservices, we also have to think about how are those teams going to be enabled? You know, we want to—you know, take that the ethos of “Build, own and operate,” the Amazon took and put on steroids, and really apply it. Maybe—you know, I'm not saying you have to follow every tech giants rule book on how to do it, but really the principles down to microservices architecture is really meantime to recovery and that customer experience, right? It goes hand in hand.

So, putting that in the energy and effort that you've done into your deployment pipeline, it has to equally be there for the orchestration of your performance. Like, what's your App Decks? Your App Decks is the industry standard for the application performance indicator. You know, you want to have a delightful experience, you've gone and removed your monolith, maybe we're having capacity problems, so now you want it faster. Well, how are you going to watch and make sure and building that out? So, take the time, build it out and think about really, how are you going to watch the system? You know, in some places in the application, you can put things like circuit breakers, you can put in points, you can think about the contracts, hubs and spokes models as well, and how communication lies. You know, what type of database technology need, and have it totally encompassed so that you can monitor and watch it. You're changing your teams a lot, you know. One of the biggest pitfalls is not really preparing the culture for these changes.

When I was—I think it was New Relic—you know, once again, another pitfall I encountered was not thinking, I guess, hello, monitoring company is like, culturally. Now, you're putting everybody on call, right? When you had a monolith, you might've had an SRE group be on call or you had a DevOps kind of be the window into the site for interruptions. Now, you have a whole bunch of microservices. You've organized your teams where they build, own and operate. Are you empowering them? Are you building the culture? Don't you—you don't want to just go blindside and put a whole bunch of production engineers on call and not prepare them, because what's the point? Once again, you're taking away one of the benefits for microservices and creating a pitfall, but not enabling the team.

And then finally, why are we doing it, right? Continuous improvement. One of the pitfalls to avoid, and I would say the biggest one, is not thinking about how you're going to improve the system. You've put all the work to organize your teams, to build dynamic workflows, to be able to monitor it, but you didn't think about how you're going to interchange these things. Microservices give you the ability in the event of failure to always have a delightful or almost delightful experience, just like we're seeing Mona Lisa, GitHub’s famous Octocat. If we think about all of her amazing accessories as independent microservices, this allows us to constantly change and iterate and think about how a customer may interact with your product. Maybe Mona wants a new hat, you can go easily change [Inaudible] that hat. Maybe she represents Canada with her amazing red and white. Cool. Now we can go and change that with [Inaudible] awesome Octocat. But now even a more awesome Octocat. Because of the microservices architecture, [Inaudible] the pitfalls to avoid it. And your whole software delivery life cycle being streamless.

So, don't make the same mistakes that I did. Don't not organize your team. Go think about how people communicate, empower them, give them agency, give the design into the microservices, into the CID and the workflow. It doesn't matter if you create a whole bunch of services. If you can't orchestrate them and deploy them and maintain them, you're taking away one of the benefits. Think about your monitoring, have a monitoring strategy, empower, enable, and train. Train, train, train. Don't just think you're going to go out there and put in a new piece of software to watch the site, and not have people understand what it means. And then finally, build in that muscle iteration. It comes back to planning and iterating. If you have the architecture, the frameworks right, you should be able to go and make those changes. Because if not, you’ve just hit all of the pitfalls. Don't do it. Keep the monolith. Convince your team that adding more capacity will keep you going. No, don't do that. We've all done that. Take the time, do the design and incrementally implement. Because if you do, you're going to win. And you're going to win at building the best microservices architecture of all time.

So hopefully, you learned a little bit, and I'm sure you're going to hit some of these pitfalls. I encourage you all, if you haven't, go read Martin Fowler’s book. It's highly, highly recommended. He can go into even more detail on some of the ways that you can, hopefully, one, avoid these pitfalls that we've all done, or at least I've done many times over, and two, help you influence and really create the strategic plan to make your product one of the future. Thanks.