Evolutionary Architecture And Systems Thinking

My name is Rebecca Parsons. I am the Chief Technology Officer for ThoughtWorks, and I want to talk to you today about the intersection of Design Thinking, Systems Thinking, and Evolutionary Architecture.
Specifically, each one of those two disciplines; Designs Thinking and Systems Thinking, influences what we talk about when we talk about the architecture of systems and I want to do a bit of a dive into each of these concepts. Of course, any one of these concepts could be a workshop in and of itself and so, I am just going to highlight a couple of aspects of Design Thinking and Systems Thinking, and try to develop those ideas in the context of Evolutionary Architecture.
So first, let's start with defining our terms.
In very simple ways, Systems Thinking requires us to look at not just the individual elements of the System, but also the way those elements connect with each other and interact, and finally the function or purpose of that system. So, all three of those components are an important part of thinking about a system. And if when you think about a system, you only look at the elements, or you only look at the connections, you are missing something fundamental about what the system is about and what it is trying to tell you.
Design Thinking - and most of the audience probably knows a whole lot more about Design Thinking than I do, but there are a couple of important aspects from Design Thinking that really influenced what we are talking about with Evolutionary Architecture. And the first is really the focus on the user, on the human, and what outcomes that person is trying to achieve, and focusing it really on that human perspective as opposed to not just thinking about what is technologically feasible. But what we are really trying to do here is focus on the outcome, not the implementation.
Very often as technologists, we get all wrapped up in what we are actually going to build and forget that actually there is a purpose out there. There is an outcome that we are trying to achieve. And we want to use Design Thinking and the process of Design Thinking to help us keep our users at the centre of what we are building.
In the case of architecture, there are all kinds of different humans that are our customers. They might be the actual end customer. They might be an internal person, maybe a customer support call centre person. They might actually be a developer. One of the things that have really started to take hold when we look at architecture is actually talking about what is the developer experience? How is it that the architecture either enables or impedes the experience of the developer to be able to actually deliver something to the ultimate customer? So, Design Thinking has a lot to offer when we think about the overall process of building an Evolutionary Architecture.
So, then what is an Evolutionary Architecture? Well, an Evolutionary Architecture supports guided incremental change across multiple architectural dimensions. And I want to talk about several of these words, but I want to start with why we call it Evolutionary Architecture.
One of my colleagues, Neil Ford, and I have been talking about Evolutionary Architecture for quite some time. And when I first heard him talk about it, he actually was calling this Emergent Architecture. And he, and I had a very robust conversation about why I thought that was a terrible name. Because when you think of emergence, it's kind of just whatever happens. And you have just, "Oh, well, it's just sorts of happening now."
Whereas, when I think about something that's evolutionary, and I am thinking about this from the context of evolutionary computation, you have some kind of objective. You have something that constitutes good. And it is important to realize that there is no such thing as one good architecture.
For some systems, security and privacy are paramount; for others, it is responsiveness; for others, it is overall scale. You can’t get all of those things at once. And so, we need to think about how are we going to actually balance these different parts of our architecture. And that is where the guided comes in.
We have a notion of fitness functions, which tell us for this particular system, for my particular organization, what constitutes good. Maybe it's overall performance, maybe it's resilience, who knows? But we need to make a determination that says, this is how I am going to prioritize these different architectural characteristics.
The second important word in this definition or phrase actually, is incremental change. It used to be, people would literally put up on the wall, their architectural roadmap for five years and 10 years. That's simply not possible anymore. It's not feasible to speculate that far out, and so we need to actually be able to introduce that notion of incremental change into architecture.
Now, when I first started talking about Evolutionary Architecture, people would come up and they would say, "Don't you think you are being professionally irresponsible to talk about changing architecture? That is the bedrock, that is the entire foundation of all of our systems." But we can't any longer say, "I know that this architecture is going to be right for the next 10 years." Because the pace of change is simply too great. And so, we have to enable some kind of dynamic equilibrium, where we are balancing the changing expectations of our customers, the changing regulatory environment, and the changing technology landscape.
And then the third is this notion of across multiple dimensions. And what we recognize here, as I was speaking about in the guided aspects, different architectural characteristics carry different weights in different systems. And so, we want to think about how are we going to balance the tension between our different architectural characteristics. So, that's Evolutionary Architecture.
Now, one of the important things about architecture, and this is paraphrasing a quote from Gregor Hohpe who wrote the “Enterprise Integration Patterns” books, is you don't have an architecture diagram, if it only has boxes. It isn't just the different components or systems or micro-services or whatever you want to call the individual pieces of your architecture.
Those are interesting. Those are important, but you don't really understand how the system works unless you know how those different components interact with each other. How does information flow through? How does behaviour flow from one part of the system to another, to achieve the objectives that we are trying to achieve with our system?
Hopefully, you can now start to see there is really a close tie between Systems Thinking and architecture. Because we can't really understand what our architecture is trying to achieve unless we focus both on the elements as well as the interconnections. And then the overall function or purpose of an architecture is to deliver to whatever the end customer is, the value, the outcomes that they are seeking.
So, now I want to talk about a few basic architectural principles that help us more successfully deliver on Evolutionary Architecture. So, the first one is feedback; and whether you are thinking about feedback loops from Systems Thinking, or whether you are thinking about the more iterative and experimental and test and learn cycles of Design Thinking, feedback is essential.
In fact, one of the nice things about the agile revolution, if you will, is this focus on how do I know the things are working, and building in those fast feedback loops at all levels of the system for us to be able to understand, are we achieving what we want to achieve? Are we delivering the value that we want to deliver?
And so, what this allows us to do is start to enable this kind of test and learn cycle where I am going to put forward a testable hypothesis of how I might achieve more value for my customer and then deliver some code, and then actually test that hypothesis. I have been in this business for a long time and if you just summed up all of the money, supposedly all of these systems we are going to save for all of these different organizations, you would have an incredible pile of money.
But very often in the past, we never actually went back and said, "Okay, did this actually achieve the value that I expected it to?" But because the deliveries were so large, and because the feedback cycles were so slow and weak, we couldn’t actually determine, “Did we do what we set out to do?”
The second fundamental principle I want to talk about is maintainability. Too often when we talk about software development, we focus on developer productivity of writing the code the first time. But we spend a whole lot more time over the life of a particular piece of code, trying to read it and trying to understand it than we ever do trying to write it in the first place.
So, what we want to do instead is focus on how maintainable a code is; how readable the code is. And this is going to lead us down the path of truly thinking about internal software quality. People often say, "Okay, I have to trade-off cost for quality. "But that's not true with software. Sure, you can get a short-term productivity boost if you don't think too much about software quality, but you will quickly lose that advantage as your code becomes harder to understand and harder to maintain. And so, to actually have a system that can respond to the kind of changes that we are talking about, we need to have a focus on the maintainability and the evolvability of both our code as well as the overall architecture.
The third piece of this is to really think about the combined notions of what is the coupling between these components; micro-services, whatever you want to call it, and how do I draw the boundaries. In the past, we have often used system boundaries as the way we think about our architecture. But if you think about it from the perspective of the business and the users, they don't really care whether their functionality is implemented in one box or five. They only care about getting the behaviour that they want.
And so, one of the things we can do to improve the evolve ability of the overall system, is to look at those individual component parts and try to be able to conceptualize, what is the business objective of this? What is the business concept that is being manifested here, or the business behaviour that is being expressed? Because if those boundaries are now around things that come naturally in the business, they are a whole lot easier to rearrange to achieve new business outcomes, because they are aligned with the concepts that the business is talking about.
One of the major advances in our ways of thinking about this comes from the book, “Domain Driven Design” where they talk about bounded context as well as the ubiquitous language. We as technologists need to bring that business context into our vocabulary so that we can have a shared understanding of what again, the system is trying to achieve for the overall end-user.
And how do we make this work? I want to go back to a comment I made previously about, "Don't you think you are being professionally irresponsible?"
One of the important enablers for Evolutionary Architecture has been the introduction of this notion of continuous delivery. And what continuous delivery does is basically sets up the expectation that you always have code that is ready to go into production. And in order to do that, you have to have a tremendous amount of automation, particularly around testing, but also around the provisioning and the deployment of your code.
Now I very often get asked, “Well, I don't want something to go directly into production because some random developer somewhere checked in a line of code.” The important part of continuous delivery isn’t that code does flow automatically without any human interventions, it is that it can. And the important part of this to me, and within the context of Evolutionary Architecture and Design Thinking is, this allows experimentation.
It used to be, because deployments were so risky, we only did them twice a year or maybe once a quarter. And everybody came in at three o'clock on a Sunday morning and they just hoped that things didn't go wrong this time. And any celebrations after a deployment were not about the new features, but the fact that you didn't get a call from the CEO, yelling and screaming at you, because something went so horribly wrong with the deployment.
You are not going to do a whole lot of experiments if there is a lot of risks attached to doing that experiment. And that was the situation we used to have. And so, for me, what continuous delivery has enabled is this ability to be confident that when you release something it's actually going to go smoothly. And if it does not, you are going to know very quickly and you are going to be able to roll it back and the consequences will be minimal. And once you have that confidence, then you can start to experiment.
You can start to allow little experiments to happen in different parts of your organization to see what might stick. Rather than having to place one big bet, we can place 20 small bets and one of those is really going to pay off for us. But it probably would be professionally irresponsible to try to do some of these things if you didn't have the rigour that is required to truly get to continuous delivery.
So, now I want to talk about each of these aspects, Design Thinking and Systems Thinking, in the context of Evolutionary Architecture.
So, let's start with the design and, this whole Design Thinking process is trying to enable creativity. You want to gather inspiration for what might be possible. You want to quickly make that idea tangible so that you can test the idea and see does this in fact, improve my developer experience or my user experience. Or is this product really a value to my end customer; and build in that test and learn cycle.
I am going to throw a hypothesis out. "Oh, that didn't turn out quite how I expected. Maybe I should look over here for something like that."
And so, what we are trying to do here is allow the systems to be incredibly responsive to increased opportunities coming from our users or to respond to competitive threats. The expectations that our users now have on us for the usability of our systems, is far beyond what we used to have to worry about. This means that you as a person who is maybe designing a system for a customer call centre person, you might have to rethink your entire user interface strategy because of something Instagram did. Because they are not going to put up with a clunky interface. Why would they?
The changing needs particularly of the Gen-Zs, as they are coming into the workforce, is going to put even more pressure on us to be much more responsive to what our users actually want. And we can't do that if our systems are too difficult to change.
And now, Systems Thinking in Evolutionary Architecture - When I look at the different aspects of architecture, we called them the “ilities” and there are over a hundred of them. And there is no way that you can maximize all of those different ilities because many of them interact with each other. For example, you cannot both maximize the security of the system and at the same time maximize the performance of that system. Because frequently maximizing security means additional encryption and the heavier your encryption is, the more computation you are going to have and that is going to limit the amount of performance that you can achieve.
So we have to look at the way the components of our architecture interact with each other to manifest business value. But also how these different, individual architectural ilities interact with each other. And that is where the notion of fitness functions come in. Because what we want to do is be able to specify; for this particular architectural characteristic, this is what constitutes good. And then I am going to run those fitness functions repeatedly to ensure that my system does not atrophy over time.
Because that is what often happens. If we don’t keep track of a particular principle, code and architectures degenerate over time, just like data ages. And so, we want to be able to look at how these different aspects of our systems work together over time. And we use the fitness functions as the guardrails that tell us, okay, this change that I made in the system over here to achieve some particular objective, it has now worsened some other aspect of my system that I said was important. And so, I need to do something to mitigate that change. Perhaps I need to reconsider what is important in my system, or perhaps I need to take a different approach to solving my behavioural problem to maintain my architectural characteristics.
The most important thing about a fitness function though, is that it be clear whether it passes or not. One of the things that architects have gotten away with for a long time is to be able to say things like the system must be maintainable, or the user interface must be intuitive. What does it mean to be maintainable?
You and I could actually have a discussion and disagreement about whether a particular piece of code was maintainable or not. Whether a particular interface was intuitive or not. So, we need to get more precise. Perhaps the maintainability is software quality metrics. Perhaps the intuition is the result of some user testing. But there has to be a concrete test that you and I will always agree on whether or not a particular system passes that test or not.
So, how does this change what I actually do? If you are one of those lucky people who is starting out from scratch; you are working with a startup, you are developing a new product and you have got no legacy whatsoever. Then it is pretty easy because you have got a completely blank slate out there to decide, “What do I really want my architecture to be?” And to build it to be exactly that.
But most of us are not that lucky. We are starting with a legacy architecture that may in fact represent several different generations of technology, of technologists, of systems. And so, the way I propose to people that they start, is to look at where the problems are coming from. Where do you get the most complaints from your users? When do your operations people get really nervous and start cancelling vacations when you say you are going to change a particular part of the system? Where do the bugs come from?
Identify those things and write a few fitness functions that will allow you to track those. Maybe your software quality is dreadful. Maybe you have got a very inconsistent user experience across your different products. But identify which one of those things is actually causing you the most trouble and start to work on it.
Very often when we talk about any kind of legacy remediation, the common question that I get is:
· How do I know it's not going to get bad again?
· You are telling me I have to clean things up?
· Are you going to come back in six months and say, you have to clean things up again?
And that is again where the fitness functions come into this, because they allow us to be able to say, "No, it is never going to get this bad again,” because we have got this fitness function that will tell us when things are starting to go wrong so we can bring it back into line.
So, you want to continually monitor your progress against that particular fitness function. And then once you get one of those things into shape, go on to the next source of your biggest problem. And always take the time out to reevaluate. Are those fitness functions still right for you?
Perhaps some change has happened that means something now is more important than it was before. This whole process has to be iterative. We have to continue to look at all of the different changes that are happening and reevaluate. Are the objectives that we have set for ourselves still the appropriate ones?
So, how does all of this come together? Our true goal here is business agility. More and more companies are becoming technology companies that happen to deliver something. The person on the last panel from the health care lab, talked about how they realized they were now a data company because they just had all of this data.
More and more organizations are realizing the technology is a key competitive advantage for them. And so, we need to have systems that allow that business to adapt to changing customer expectations; to changing developer expectations; to the changing regulatory landscape; and to the changing technology landscape. And we do that through this process of continuous design and continuous delivery.
Design Thinking has us focus on what are the outcomes that we are trying to achieve and, the people who are going to benefit from that outcome, how they are going to interact to achieve that outcome.
Systems Thinking focuses our attention on how the different parts of the system interact with each other to manifest those outcomes, whatever they are. And what do we have to do to keep our system in this dynamic equilibrium, where we are appropriately responsive to the changes in the environment around us.
And finally, Evolutionary Architecture provides us with the tools and the frameworks to think about, how do we actually keep our systems poised to continually change, so that we can support this more experimental mindset. We are going to make these ideas tangible and then test them in the real market to determine, “Do they deliver the value that we are hoping that they do, or do we have to try something different?”
So, Evolutionary Architecture is focusing our attention on what does it take to be able to support this experimental hypothesis, this test to learn cycle, and to be truly responsive to our users, whoever they are, and really manifest the potential in Design Thinking.
Thank you very much. And I think we should have time for questions here.