Paying off Tech Debt - Scaling Up To Grow

All right. Hello everyone. My name is Catherine Cornell. I've, been a product manager for about seven years. I'm currently a Director of product at Stash, focusing on stashes consumer investing business. For the record, all the information I know in this discussion is for educational and information purposes only. And today I'm here to talk to you about Tech Debt and how we as product people can be more engaged and a better partner to engineering in this vital, vital, vital thing that is often overlooked. First, a little background about Stash and what I do here. So Stash is the first subscription platform, empowering middle-class Americans to invest and build wealth. In addition to our brokerage products, we also have a banking debit card and a stock reward, on that debit card program. Our market is middle class Americans. The medium household income of a stasher is around 50 thousand dollars a year. Our over five million customers are diverse coming from all over the US with unique financial backgrounds and situations. The common thread between all of them is their first time investors. So over at the consumer investing business, we really focused on how to make first time investors feel confident and empowered when they're getting started. It can be really scary to start investing. So I'm going to take you back to 2019, which was also a very scary time. I cringe looking at this purple screen. I also cringe thinking about all of the crowded subways, crowded time squares that this purple screen has been pulled up on customer's phones. Totally just in with germy hands, touching it all over without sanitizing. But this is what our app looks like in November of 2019. As you can see, we have groupings based on, I believe, I want, I like, we had spent the four years leading up to this time on invest, really focusing on expanding the number of products we offered. Different types of brokerage products and iterating to find the right fit so that when new investors came in the platform, they could find the right brokerage product for them and get started. We had around 250 investments offered a mix of ETF and single stocks, but mostly ETFs at the time we executed orders twice a day, we went to the market and we added investments every so often, whenever customers were busted. Adding an investment took around three weeks and coordination and required coordination between four teams. The reason I bring that up is during this time we started to hear from stashers that had been with Stash for a bit, that they felt like they had grown up with Stash. They had really taken to the education. They had really absorbed the advice and they were wanting more. This was a great sign for us. It means that what we were doing was working, we had found our product market fit, that we're all searching for, but how did we help these newly savvy stashers grow even further and stay with the platform? Well, they told us. These are two five star reviews from the Apple app store and the Google play store. So this iOS user, sandwiched in between their lovely comments and their five star view. Tell us, unfortunately, you only have the ability to purchase, what is whatever is listed on the app. They might want to figure out how to broaden their investment options, this Android user, after telling us that it's great, it's simple. All the stuff you want to hear from a user. They end with a dramatical leaps, but does it offer full range of companies to invest in? So we really heard that and we heard this many different times that our stashers were ready for more. And if we're going to retain all of these users that we had worked so hard to acquire, we needed to have more investment options of the platform. So we aligned on our mission. We've all been here, right? You have a pretty clear mandate. You feel really confident going forward. Your opportunity size. We were really excited about the increase in potential AUM that's Assets Under Management. That's one of our key metrics we track. We had not only make revenue off of it. It's also a great side of our customer's health and whether the customer is getting value out of the Stash platform. So we knew that whenever we added a new investment, we would see a jump in AUM as well as a jump in our second key metric, which is Buys and Deposits. So doing some forecasting, we were really excited about the opportunity size to grow our AUM and grow our deposit, grow our buy rates , we aligned on the mission. Everyone was excited and we had a pretty clear mandate. Scale from 250 to the over three thousand domestically traded stocks, dramatically traded investments over the counter. So you have a mission you're ready to kick off. You start writing your spec. It's an exciting time and you identify blockers, which is part of the process. That's what we go through. That's why product managers actually have a career because blockers are a thing. What I didn't expect. And none of my stakeholders really expected is we found when we started to actually get stuff onto paper and think things through that, we had 10 pages. Single-spaced of known blockers. Everything raging from the mobile layout was designed to with three segment controls that were scrolling less. You can't really scroll through a list of a thousand things. The iOS app couldn't even handle loading pricing for that many investments. There's the way we did pricing altogether wasn't scalable. We were updating a single table in our model of every five minutes and when we started to think about how we would actually trade the operations tools that we were using tapped out at five hundred that's way less than three thousand and all of those forecasts gains that we had been so jazzed on. Our trading infrastructure couldn't even begin to handle the volume that we were anticipating growing to. So we had to take a step back and also think about the infinite unknown blockers that we all know are going to come up. And it was really worrisome. It was a questioning time of how did we make so many mistakes, right? How do we back ourselves into 10 pages of mistakes? And we all know the answer it's Tech Debt. Tech Debt is the implied cost of additional work in the future caused by choosing an easy solution. Today you're essentially borrowing labor and capital from future you, by taking a shortcut today, you might need to double back around in the future. And it doesn't mean you made a bad choice. It's the cost of building. I think sometimes we tend to think of tech debt as a bad engineering or laziness, but it's really the most practical thing to do. I see it as a graduation. It's like your system has grown and scaled to the point where it's outgrown its little shoes and it needs new ones. So when you encounter it, I want to challenge you to reframe it as a graduation, get your little product, a cap and gown place in vitamin C and celebrate before you get to work. I also want to encourage product to think of itself as the co-signer of tech debt. We're just as responsible as engineering for the choices we've made in the past. And we can't put it all on engineering to fix it. We need to be engaged with the process, as Stash we have a working agreement that engineering is responsible for identifying what tech debt there is and explaining it to product. Product is responsible for truly understanding what the Tech Debt is, what the trade-offs are of addressing it now versus addressing it tomorrow and prioritizing it as appropriate. Product needs to create an environment where it's easier for the two parties to have a consistent and constant dialogue. So I want to talk to you about some of the ways that we identify Tech Debt. These are checkpoints that we discovered along this multi-month journey that we use today. So again, these are check points or moments where it's not really appropriate, but needed to check in with your Tech Debt balance. As we like to say. These are in order from most ideal to least ideal. So if you want to exclude the last two, that's fine by me. The first is roundups. Roundups are a ceremony that the invest engineering team has introduced into our agile ceremonies. It happens maybe biweekly or monthly, depending on the team it's led by an engineering manager or a tech lead. And it's attended by only the engineers in a chapter. So for instance, we have a backend Roundup biweekly with just the invest backend engineers. It's a space for them to think through and opine on what tech debt there is. What risks they could see upcoming. And the most important part of the ceremony is to have documentation. So meeting over meeting, they're writing down their concerns and collecting a backlog that could then be discussed incentive product. In our case, the engineering manager runs this meeting and is responsible for bringing that document back to product and having the discussion of these are the trade-offs. These are the risks we're seeing, help us get these prioritized so that we're on the same page. The next is OKR planning. I know we all love OKR. They're very trendy, right? They've been trending for a minute and I know we're all excited in a week or two to kick off Q3 with our new OKRs and a lot of times I've fallen into this trap of you make an OKR of, I want to increase the top of my funnel by 10 percent. But you don't think if the rest of your funnel can handle the 10 percent increase in traffic, that's going to come if it does. So OKRs whenever you're planning for growth is a great time to check in and make sure all of your downstream systems can handle the growth that you're planning for or aspiring to. The third is a new feature, exploration. If you're planning a new feature inside of an existing ecosystem, you need to check in and make sure that the rest of the ecosystem can handle whatever the new feature is bringing, whether it's more traffic or more complexity. The last is a test win. This is really a last ditch resort. I also would put a marketing launch in the same boat. If you're shipping a test that saw a 15 percent lift in metric Y, you need to make sure that the rest of your system is ready for the entirety to see maybe a 15 percent lift, same with marketing launches. If you have a marketing launch that you anticipate flooding traffic to your app or traffic to your system, you need to make sure that you're sending it out at a rate at which your system can handle. So now that we've identified the debt, I want to talk you through some of the approaches we use for actually tackling it. What this really did for us at Stash was it created a shared language between product and engineering to understand the commitment needed to tackle debt. They're essentially fancy t-shirt sizes with some implications. Let's go through them. So the first is the good citizen rule. I think we've also called, I've also heard it be called the girl scout rule. It comes from the mantra that when you stumble upon a campsite, you need to leave it better than how you found it. In this case, we have developers whenever they come along a code base, they need to live a better than how they found it. So this is opportunistic refactoring while coding features that's included in existing story points. So they go in to work on a distinct ticket. They might be removing verboseness they come across for moving debt or unused code or adding more test coverage when they see problematic code. Next to small bites, these are small improvements baked into the sprint. They're pointed on the smaller side. Think ones are threes. The key to a small bite is that it's a standalone piece of work. It's not dependent on any work being done in front of it, or it's not blocking any work behind it. Examples of this are query optimizations, bug fixing, or creating tools that suck up developer time. I'll talk about this a little more later, but we found that in the existing investment plan for management, we had so many problems that we were just dealing with day in and day out. It's sucked up so much developer time by taking a day to bug fix something or to make a tool or an operations person could address a problem rather than a developer. It freed up so much time for us to not only work on Tech Debt, but also to work on new features. The third is design it in. These are design changes to a larger platform when implementing new features. I spoke before about how we had so many issues with pricing. We knew pricing was not going to work at our monolith. And one of our wider engineering goals at Stash was to be breaking up our rails monolith. If we took this opportunity to break pricing into a microservice and out of our rails monolith, some more global examples of design it in would be moving to maybe share components in iOS when you're designing a new feature in your iOS app, or maybe introducing new technologies into your stack that can make developing easier overall. Designing in it are usually multi ticket initiatives, but maybe they don't span a couple sprints. The last is demolition. These are major changes to a larger foundation and they often spend multiple sprints. In example, we face a Stash was updating from rails five or maybe rewriting a service in a new language or maybe a more global example would be moving to a streaming event architecture, or maybe a migration into AWS or out of AWS. Either way, these are big commitments that often span multiple sprints. And to me, they also implies that there's going to be multiple teams collaborating on this. So it's a big commitment, not just from your team, but from other teams as well. So those are the four types and we used all of them and we had to create a debt repayment plan. So I like to joke that we use the snowball method. The snowball method is a debt for payment in which a person pays off their small steps first while only making minimum monthly payments on their other debts. It's sort of like an MVP, but there's multiple work streams. So that's what we did. He had other lanes of debt we had to be paying out at the same time, specifically small bites and our good citizens. So we continue to work on those while we calculated the maximum number investments we could add to the platform without toppling the system. The back of the napkin math came out to around 250. We decided to be conservative and only add two hundred. So the day after Thanksgiving, we added two hundred, investments to the platform. We took notes of the largest pain points in the actual process of adding the investments, that helped guide what automations we needed and also what tools we needed to empower operations to be owning this process going forward rather than developers. We uncovered some of the unknown blockers and the most important thing was it validated the hypothesis that if we added new investments in the platform, we would see big movements in those key metrics we were trying to shift. So now I'm going to show you the rate at which we added investments to the Stash platform. This is all 2020. Throughout 2020, we added 3,463 investments. We add investments pretty much on a weekly basis today at Stash between IPO's corporate actions. The market's always in flux, and we need to keep up with that. But you can see our little jump at the end of 2019, but most of our investments were added late February or March, specifically with 1,555 being added in March. And I just want to call it in a personal note that we added 800, roughly 850 of those. The first week that Stash was remote in March. We were forced out of our office very last minute a couple of days before our entire city went into lockdown. It was a really scary time to be in our home city of New York. We, some of us were sick, others of us were very scared and worried about our families. Some of whom were sick, some of us we sick, as I said. And I'm so proud that we were still able to come together, support each other and get this done during that really intense time in our city. So you can see the rate, the shape of this graph. Well, let's now look at the rate of buys and deposits in 2020 on Stash. So you can see a slight uptick in December into January and February when we added those two hundred. But when we really started to step on the gas in March, we saw a significant and sustained increase in our buys and deposits. So we saw that paying off her Tech Debt really paid off. We started 2020 with 900 million AUM. We ended it with 2.2 billion for a 142% increase. Our customers earned almost four million dollars in dividends, similar to AUM. We love to track dividends for our customers because. Again, it's a measure of their financial health and also how much value they're getting out of the stash platform and how well we're meeting our core mission of increasing the middle class as wealth and financial stability. We also saw a year over year 37% increase in purchases. And what I'm most proud of is we didn't just get a lot of investments onto the platform in March and then let it go. We continued to commit to our tech identification ceremonies and kept slowly working through the backlog, little by little, always making our trading system and our investment platform better. And doing that actually paid off in dividends. So what you're seeing is Google searches for the word stocks in the United States. In January of 2021, there was a short squeeze on the GameStop stock and some other securities, now known as the Yolo stocks. The media coverage that was given to retail investors, specifically retail investors in the sub Reddit WallStreetBets, created a retail investing frenzy in the United States. Many Americans were learning through this media coverage that they could start investing without needing to front hundreds or thousands of dollars. And a lot of those first time investors found their way to Stash. This is our Stash subscriber creation during that same time period. You can see that in a single day in January, we capture it almost 30,000 new stashers. During that week, we were able to not only capture these new subscribers, but fill all of their orders without incident. There was no concern about our ability to serve up investment prices, investment pages, to facilitate all of the buy the record breaking buy, and that was happening on our platform. And that's only because we had been so committed to our Tech Debt management and our payment plan throughout all of 2020. We didn't just stop. A lot of these stashers are still on the platform because of the advice and guidance we have and because we were able to keep chugging when we had some competitors that maybe work this day. So just to recap, there will always be Tech Debt on that fantastic week. I just told you about, we had other issues with our app. We had issues with our signup, loading the home screen, because you can never anticipate when you're going to go viral. So my advice to you, if you're trying to grow your business, or you're trying to go viral is you need, there always be Tech Debt. You need to be proud of your Tech Debt, because it means that you're growing and you need to have a process and feedback loop between engineering and product to ensure that you're always doing little bit by little bit. So when you do go viral, you're ready to capture all of them in your traffic. Thank you.