Kessel Run: A Digital Transformation Story Within The World's Largest Bureaucracy

Good afternoon, everybody.
A great stage of speakers today, I'm humbled to be asked to come to speak to you all. My name is Adam Furtado. I'm the chief product officer for Kessel Run named after Han Solo's famous smuggling route. Kessel Run is the United States Air Force Digital transformation initiative to revolutionize the way the Air Force builds and delivers software. I'm not going to talk much about DevOps. There isn't much about UX. This talks barely about technology to be honest, mostly about organizational trends, transformation, and innovation adoption within the world's largest bureaucracy.
I'm a former Air Force intelligence analyst who has a couple of Liberal Arts degrees and, in some ways, signals that technology is not the hard part. It’s the culture change within it that helps us get through these digital transformations. Let me leave that there and I want to set the stage a bit. My wife watched an awful television show the other day. I came home and she was watching an episode of CSI Miami. I have no idea if Ireland has the same TV shows as we have in the States, but if you don’t, you're not missing much. So, I walked in, there were a bunch of agents huddled around this table that had Holograms floating around one guy swiped something up into the air, and we were knee-deep in some dudes' Twitter account and there was augmented reality device around. It was wild anyway, so she looked at me and said, “Adam. If the police have this type of technology, I can't imagine what you used to use in the air force.” I didn't have the heart to tell her the reality, but the reality is the tools of modern warfare don't have Holograms and we don't have authentic reality. This is a picture of the air operations center all around the globe. The United States Air Force has these air operations centers to plan, execute, and excess air warfare and air operations worldwide.
Kessel Run was tasked to modernize the air operation centers and all the work and tasks that are within them. The real tools of modern warfare within the AOC are the same tools that my wife and all of you use every single day, Microsoft Office. If you were to walk around on this floor, you would look at all the computer screens and you would see a bevy of Microsoft Office products with all the pertinent data needed to execute warfare. We would be storing our data in the same application your mother balances your budget in. We'd be building analytical products in the same application you wrote your college papers in, and we'd be visualizing that data and making strategic decisions off the same application that I'm using it right now.
But if I'm being even more accurate, it's probably the version that still comes with Clippy. Now, this is scary, though. Eight so Mark Andresson famously said, “software is eating the world”. We're learning more and more every single day that software is also eating the war. Software is a central differentiator on the battlefield, soon it’s no longer going to matter how many tanks or aircraft or ships you have it's all gonna, to come down to your software, whether we like it or not, whether we're prepared for it or not, software is now a weapon of war, and we must cultivate that weapon, so we can wield it successfully and responsibly.
Earlier today, the gentleman from Capital One was up here: he talked about Capital One which is a software company that happens to do banking. We're trying to turn the air force into a software company that delivers airpower - don't worry, it’s gonna get lighter, it’s not going to be this intense the entire time.
I think that the reason we're here is that the Department of Defense's acquisitions communities for the people who are tasked with providing capabilities to our users have failed for years upon years to deliver meaningful capability at relevant timelines to the people who need it most. And I know that because I used to be one of those users. In 2010, I was deployed out to an Air Operations Center, where I had the same capabilities available to me that we still have out in the field today they're still being used today. not just the same capabilities for the same versions: we're unable to deliver product into the field.
General Mike Holmes, the commander of Air Combat Command, I think says his best” years of institutional risk aversion have led to the strategic dilemma plaguing us today, replacing our 30-year fleet on the 30-year timeline''. And that risk aversion realizes itself in a multitude of ways. We constrain ourselves by these bureaucratic safety nets that protect us from ourselves, but in reality, they make us less safe because we're not able to adapt to change. We constrain ourselves by treating our software organizations the same way we treat our hardware organizations, the same organizations that build and deliver aircraft and aircraft carriers, and the byproduct of that is you start receiving your software on those same timelines.
This picture I think epitomizes our plight. This is a process by which the DoD acquires and builds software or anything, the most interesting part of this slide to me, or maybe the most humorous thing, the scariest is in the fine print in the top right corner. It says this process is so complex and so convoluted you couldn't possibly fit it on a two-dimensional diagram, so try navigating that in 12 parsecs or less. The fact is, nobody can navigate it. All these things, are bad practice on top of bad practice and delivering software in a meaningful timeline becomes an impossibility.
Studies have shown that traditional cycle times are the way the timeline we can get systems into the field in the DoD. The median time is 8 years, that doesn't even take into account the fuzzy front end of your requirements and you're contracting all that our actual timeline average time was 12 to 15 years to get an idea into a user’s hands. In an actual example of this, the program that set out to modernize the AOC before Kessel run got involved was ten years into their development and five hundred million dollars into their development and hadn't delivered a single thing to the warfighter. They were scheduled to deliver in 2020, based on requirements written in 2006. We had a 14-year timeline between the idea and that thing being delivered. So think about what you knew in 2006 or better yet what you didn't know. In 2006 the first tweet was sent. 12 years later, we're now sending 6,000 tweets per second. There's no way to know these things. This should be akin to China drafting up the specs for the iPhone 12 before the first iPhone was even released.
Unsurprisingly, other studies have shown that federal IT programs are 96 % behind schedule or over budget, 40% percent never deliver a single thing. Recently, Eric Schmidt of Alphabet and Google Fame was testifying before Congress in the state of DOD innovation, and he said that DoD violates every rule of modern product development. We are a literal case study of what not to do. So I know what you're all wondering if you guys don't know what you're doing and you're really bad at this. Why are you the speaker that’s keeping us from getting to the pub for happy hour. Because there's some optimism coming there's some good news coming and also, I think it - should mean a lot to say for those of you who have lived within bureaucratic organizations or highly regulated industries, if the DoD can do any kind of digital transformation, any one of you can as well. Even general Holmes’s quote continued and added some optimism. He said, “this dilemma also provides us an opportunity” and that opportunity started in the fall of 2016 when the defense innovation board traveled out to the air operation center in Qatar, the defense innovation boards, a group of Silicon Valley, CEOs, and innovators, chaired by Eric Schmidt, who advised the Secretary of Defense on innovation efforts within the government in the DoD. They were traveling around the AOC to learn how we execute air warfare.
Unlike commercial airliners, in an aircraft that the United States Air Force, we refuel our aircraft in midair, high altitudes super-fast, and it's super complicated to plan. There is only a certain amount of tanker aircraft. They’re a bunch of flying aircraft that need it. So, optimizing the scheduling for that, making sure that aircraft are at the right altitude in the right place with the right hardware is super difficult. So, Eric Schmidt went and saw how we plan this within the air operations center, and it looks something like this. there's a large dry erase board with a couple of computers and a bunch of guys standing around it, mostly pilots who have been pulled out of their jets to do this job for six months, away from their families deployed into harm's way, And we have this first guy here who is tasked with moving these pucks around and connecting the missions that we had to fulfill with the tankers who were available. I like to think he looks something like this. So, as he's working through this plan, he starts shouting across the room at a guy they called the gonker. I like to think he looks something like this. He's putting all this information into Microsoft Excel. There are some macros and scripts in there somewhere, but he's largely doing all of this in his head due to his background, trying to make sure that we have enough fuel to fulfill all those missions. He's beautiful minding it. Right when he feels comfortable with this plan he yells across the room to somebody else that they called a thumper. He looks like this. He's taking that planning and manually inputting all this information into our legacy 14-year-old monolith because it's the only way that we can get this data into the Jets who need it. Eric Schmidt called: that's them happy, so they have this down to a science for the thing that didn't account for what changed right. If there are major changes, they have two options: they can either erase the whiteboard and start all over again. This takes about an entire shift to do, or they could send up another tanker and they could make sure we're covering those missions with another aircraft. This led to an incredible amount of inefficiency, which Eric Schmidt called “the most ridiculous thing I have ever seen”. So, this was the first project that Kessel Run was tasked with actually tackling.
The problem was at the time there were just a few of us who were tired of the status quo and wanted to do something new, but we had no idea what we were doing. None of us had any skills in modern software development or how to manage a digital transformation. So, we had to go to Silicon Valley and ask for help. We partnered with a company called Pivotal and Pivotal labs to not only help us build a cloud-native platform for us to deliver capability to our warfighters, but teach us how to change the way we thought about products, the way we thought about our organization.
We've sent a small group of active-duty Air Force members to their office in San Francisco to embed there. These were guys we saw had a growth mindset, some rebellion in them, and we thought that starting small at this with this little product was enough to keep us under the radar just see if something like this could work within the government. So Pivotal taught us a bunch of things on Lean principles used in the design, DevOps, extreme programming, Test Driven Development. All these things, these practices, and processes that we all hold dear, but the most important thing they taught us, was how to change the culture within your workplace. It’s one thing to read slides, and see talks on why culture is important, but it's another to see in action actually on the floor. Seeing people collaborate and seeing ideation happen, a few people happy at work.
I have colleagues that work in government IT for over a decade and never seen anything reach the user, never going to be fielded, Imagine spending ten years of your life and never seeing any value get to the place that this needs to get to.
So, about a few months later we released a Minimum Viable Product into the user’s hands. We digitized the whiteboard process that they were using already with a lot of data validation in the backend to make sure that we were helping them validate the amount of fuel and all that and try to find some efficiencies. And we did. We found some efficiencies right away. We got to a point where we were sending up one less aircraft on average per day. That sounds small, but it equates to two hundred and fourteen thousand dollars a day in fuel savings alone. That’s not taken into account maintenance cost and cruise and all of those things, that's just fuel savings alone. The actual investment we made in this product at this point was 1.5 million dollars. The application paid for itself in about a week.
So, we had this win here right, but we had to figure out how we could take this win on a small scale and articulate it to the powers that be within this world's largest bureaucracy. So we're trying to find some tools we could use to make the point even stronger, so we started using something called cost of delay. We made the argument that if you would have waited five years to deliver the same products using our traditional waterfall development methods, we would have had a cost of delay of 384 million dollars. 384 million dollars the Air Force didn't have to spend would have wasted. So, as we all know, the software is never done. We continue to iterate, but a year later, thirty iterations later, we doubled these fuel savings to four hundred and twenty-eight thousand dollars a day in fuel savings alone. It doesn't stop there. We found efficiencies where we're deploying two less people to do this job, so it's two less people leaving their families going into harm's way and being pulled from their aircraft. So that one means a lot to me. So, from here, we had this win. Now we had to scale and solve other problems. So here we are today. We have 18 product teams all delivering products that our airmen are beginning to love. We use a balanced team model with product managers, product designers and software engineers focused on lean principles to make sure that we're building and learning the next most valuable thing in the simplest possible way. We’re employing product designers for the first time traditionally in the DoD you'd have some subject matter expert write a 60 page spec, they farm it out to a defense contractor they'd find some front-end developers to fit everything they could into a screen and usability was an afterthought.
Now, with bringing in user-centered design for the first time, at least in our world, usability is a first-class citizen again. We're averaging with all of our teams 124 days from whiteboard ideation into our classified production environment. Remember our benchmark was 10 years. And with that, we're using extreme programming and test-driven development for the first time. We have over 80 % test code coverage which led to us getting the government's first-ever continuous authority to operate. So what this means is traditionally we would finish a software product we'd, give it over to cybersecurity folks, and they would take about a year to 14 months to approve it to go on the network because of our pipelines and security processes, and our test code coverage, they've granted us continuous ATO to go directly into production every time we iterate, which is multiple times a week at this point. This led to us being the first Department of Defense team to continuously deliver into sipper net, which is our classified network.
Now a couple of tips here. If any of you are in the midst of a large-scale digital transformation, I wouldn't even call it that anymore. What you need to focus on is getting a small undeniable win that you can grow from and build a little bit of runway. Sometimes I feel like even a year and a half in, and some of the success that we've had all my job is to get one more win to get me some more runway to keep going.
One thing we learned here as well is to not forget about our platform or whatever our delivery mechanism is. First, we got distracted by the applications and the products. We weren’t concentrating on how to make sure that we were efficient in getting that into the user’s hands. So, we started emphasizing our platform in our delivery and the actual doing the DevOps, and we saw a rapid increase in our software delivery performance so much so that we're now deploying over 300 times a week to our staging environment and 30 times per month to production. We're at this point now where deploying is almost a non-event for us, which is where we want to be. And when you get to this point, where you want to think about delivering anymore, you can concentrate on mission applications that are above your value line.
It wasn't as easy as I'm kind of making it out to be. There's a lot of work along the way within the government and the DoD, I think, within industry. I think Rory mentioned it this morning. A lot of organizations tout that they're doing agile and it kind of hurts the messaging a bit, especially an organization that doesn't understand it. In fact, for a long time and a lot of circles, still the government thought that DevOps meant having developers work with users because in the air force we call our users and our warfighters operators. So, there is a language barrier there as well. So, we had to spend a lot of time, educating upwards and cultivating downwards, but now because I think we've concentrated on that fuzzy front end and the last mile I feel comfortable saying that customer one has gone full-blown, agile, and just so in case anybody's wondering the AF here stands for Air Force, promise.
So, but we had a big thing, a big rock still to move. We had all these processes and technologies in place, but we had a full-scale cultural transformation, a people transformation that we still had to focus on. So the interesting thing that we learned was that to build effective teams in this modern environment to do modern product development, the culture around that. in military culture, as antithetical as it sounds, are pretty similar. Google did a study called Project Aristotle, where they focused on the team effectiveness traits that drove teams to be really good performers, and they came up with these five things. And, interestingly, the first four of these marry up very well with military culture, dependability on the battlefield, dependability, is a necessity right, you're, relying on the guy next to you. Never leave a man behind, all of those things at a different personal scale. We feel the same way about our balanced team model. We have PMs and engineers and product design, making sure they're representing all of their equities, to shre product decisions, and making sure they're all going towards shared outcomes.
Structure and clarity again, another necessity to the military. The way that we use extreme programming, test-driven development, and the way that our office runs are very disciplined. We get a lot of visitors from senior leadership that come in and they get distracted by the stickies and the whiteboard and all those things. It looks like a Wild West environment when you're in there. But in reality, when you dive into these practices, they are very disciplined and it takes a lot of structure to make them work. So, it is a nice way to work for our military guys, who kind of have that ingrained within them And they said you have to have meaning in your work. I think this one we have in spades. A lot of people, ask about retention, famously the government can't pay the salaries that industry can, especially for software developers right now, so we have to make sure that we can provide the meaning in your job in the workplace culture to try to make up for that as best we can.
Recently I was walking in San Francisco and I saw a billboard first time that was sort of like this, and the app was for dog walkers. So, you can optimize their routes to your dog, could pee on grass rather than concrete. A good idea. That company can pay more than I can for a software developer. But I feel very strongly that when our guys go home from work, they think about the impact that they had in the world, the safety of their country and we take that very seriously and the same goes for having an impact.
So, if we think about what a warfighter is, they’re people who were deployed away, who are putting their country on their back and making sacrifices. Well, there are different levels of that. I like to think of our guys as the modern warfighter. Yes, they're in t-shirts and jeans in a lab in downtown Boston, but they're sending features there, sending their work directly into the front lines, and changing the way that we think about being able to react in the battlefield. Continuous delivery, there's a lot for us to be able to, you know, react when things change and they're having a significant impact. But the interesting one is this: last one, this idea of psychological safety. This is the one where we don’t have it all. The military is based on this hierarchical top-down leadership, which is imperative to warfighting into being successful in the battlefield, really a bad way to build really good products. In fact having uniforms, the whole idea behind them is to strip away your individuality when, in reality, that individuality is what we need to solve complex problems creatively. So how have we completed this within Kessel Run a few ways? One we got rid of the uniforms. We got rid of the ranks, everybody talks on a first-name basis. Our balanced product teams are balanced. You could have a lower enlisted product manager working with a higher-ranking officer, they're talking on a first name basis and they feel free to be safe, to fail, to isolate, free to give you their opinion, which is new for the military. So, you're focused on this lot. We focused on building a feedback culture and the military feedback is inherently negative. If you're getting feedback you did something wrong and we're trying to change that model within our organization, so people can find their blind spots and want to learn and grow and get better.
The biggest thing we learned from here is that we had to build a learning culture. We had to build an organization that all we wanted to do was learn and get better and continuously grow. That’s been difficult but is it something we've been prioritizing. Admiral John Richardson, the Chief of Naval Operations, he's the highest-ranking officer in the Navy. He said, “it's about the team that could bring the people, technology, and processes together to learn the fastest. That's the team that has the advantage”.
When I started this job, I focused all of my energy on people, technology, and processes. I thought that was the right way to go, make sure we're building our practices in deploying our product, and I missed this last part. The idea of learning the fastest is the real takeaway here. The first three things are enablers for us to do that, but getting to this point, where our organization is constantly learning, is a real place that we want to be. We open our lab in Boston and we call it a Kessel run experimentation lab. We called it that for a reason because we believe that everything we do is an experiment. We want to promote that within our organization. It’s our island of innovation, away from the bureaucracy that we're trying to fight.
Now, I kind of understand where I'm at right now. I’m at a tech conference with a bunch of experts and DevOps and UX and all those things and I'm trying to tell you how innovative we are because we're using practices and processes that you all have probably been using for 10 years. But being only 10 years behind the industry is disruptive and is productive in government. I know that because I joined the military when I was 18 years old. I’ve been a government civilian all the time after that. This bureaucracy, the world's largest bureaucracy, is all that I've ever known. I never thought that we would be able to get to this point, especially this fast. And we've done that with a serious approach to culture, a dogmatic approach to learning and never saying no for an answer. So why do we name this initiative Kessel run? It’s because we had to smuggle this culture into the DoD. We were a small group of rebels flying as fast as we could on a mission to do what was right. So, the difference is, in Star. Wars were seven, eight movies in the alliances in shambles with no hope in sight, but in our case, our rebellion is turned into a revolution and the dark side is beginning to see the light. Thank you