How BMW Motorrad Integrates Fans into their Digital Product Development

[Jan Bode]
Hi, I’m Jan from the Testbirds Community Team. And I’m here with our partner, Jann Kirchhoff from BMW Motorrad. And we want to talk a bit about our collaboration and how we assess Birds help BMW Motorrad to test the digital products with real riders of their motorcycles on our platform. So we will talk a bit about the why there is a need for it, and also how does it work. And I will also give you some insights on how we built up a dedicated community of BMW motorcycle riders who frequently test the BMW Motorrad digital products. And I will just hand it over to Jann who can take it away from here. And I’ll talk about our community a bit later on in the presentation.

[Jann Kirchhoff]
Yeah, thank you very much, Jan. Hi, to the audience. My name is Jann Kirchhoff. As Jan already said, I’m working with BMW Motorrad in an Area of Digital Apps and I’m employed as a Product Success Manager. So, what does this mean to my business is, basically keeping the customer happy, giving the customer a seat at the desk when it comes to decisions about our apps and taking care of it. Our digital services actually offer what the customers need, and testing is a huge part of that. And therefore, for testing, for proper testing, for proper—yeah, customer opinions—we are working together with Testbirds. But, first let me take you a little bit into the world of, like, what it is like riding a motorcycle.

If we talk about motorcycles, we are not necessarily talking about like digital stuff. So, when you are riding outside with the motorcycle, it normally gets dirty, you are pretty close to nature, you’re on the road, you’re on the track, you’re together with people, sometimes you’re off-road in the mountains, and it’s pretty, pretty intense. So, necessarily, you don’t need any app. But we want to provide a digital product that makes riding better than before and that helps you doing better rides. We have a small agenda to walk through today. And first, I want to introduce the product to you—what do we actually do, what we offer—then what quality means for us, which challenges we are facing, how our crowd-testing is actually working, how we actually get in touch with the customer, and what Testbirds actually means to us in a—if we talk about tooling landscape.

So first of all, BMW Motorrad is offering a product called the “BMW Motorrad Connected” App. This is available in the App Store for free, and you can install it on your smartphone. Your smartphone can then be connected to your motorcycle that features a 6.5 inch TFT display, and it’s connected via Bluetooth, and you can also connect your helmet. And with this combination, you can take phone calls, you can listen to media. And in combination with the BMW Motorrad Connected app, you can also use navigation from your smartphone, you can record your rides and the data you create, and you can also see valuable vehicle information in the app. You can control all of this with the so-called MMC, within multimedia controller that is mounted to your left side handlebar. So, you actually can control multimedia functions without taking the hands of the handlebar, and this is one of the most important things while riding because you want to keep riding very secure and our riders healthy.

This video just gives you a small impression of what it works. So, on left-hand side, you’ll see how the MMC is manipulated. And in the display, you can see the navigation instruction changing. Again, this comes all out of the app and it’s transmitted via Bluetooth to the display. And that’s why our BMW Motorrad Connected app is also pretty important for the customer, since it delivers a great navigation functionality out of your smartphone.

Talking about quality, the actual challenge has changed. So, we are not—we are also producing motorcycles, of course, but if you talk about digital products, we have like three puzzle pieces that need to go together. Of course, we have to use the vehicles, and the software versions and the vehicle models that we have that are featuring the TFT display. But we also have the BMW Motorrad Connected app that is featuring updates even every month. And on the left-hand side, we have user devices. And as you can see, there are two smartphone platforms. There are more than eight smartphone OS versions featured. There are more than 24 countries that need to be supported. And only 97% of our users have more than 30 smartphone models. And now, you need to make sure that all those puzzle pieces were perfectly together to create this high quality user experience we want to deliver and we want to see for our users, since this is the reason why we get up every morning and get into the office. Yeah, we really want to deliver a high quality user experience, and Testbirds, of course, is a pretty valuable part to make sure this higher quality user experience is created. Of course, we have like our profound testing cycles from vehicle development. We do internal testing. We have other testing partners that ride our motorcycles for thousands of kilometers to make sure the vehicle is properly working. But, if we talk about digital products, the game changes a little bit and we need other tools to cope with that challenge, basically.

So, when we started first with crowd-testing, and we had the first appointments also with Testbirds—of course, Testbirds told us, “Yeah, we have this great Testbirds’ Test Crowd.” And at that point in time, though there were about, like 300,000 testers basically, that we potentially could involve into our testing. And we pretty fastly come to the point that when we said, “Okay, how many of those 300,000 testers have a motorcycle?” It’s like this. “And how many of those 300,000 testers have a BMW motorbike?” It’s like this. “And how many of them, actually, had BMW motorbikes with exactly this equipment ‘TFT display’?” It was like, zero. And this was a big problem actually, because there was just no one who could test our app. Because of course, to test that app, they need a motorcycle. And what we’ve been working on is to bring your own crowd concept—together with Testbirds—where we created our own crowd. So, together with Testbirds, we recruited testers that—or customers of BMW Motorrad that actually have and own motorcycle that is equipped with the TFT display. And so, we build up a crowd that today features more than 250 testers in more than 20 countries, and it’s growing. And this crowd basically is so valuable to us, as we can directly address those customers, ask them for their opinion, ask them to try our app—the newest beta version of our app—and give us direct feedback. Of course, Testbirds did the heavy lifting and recruiting those users. We ran through Facebook groups, we went through influencers, but also our email support—our Affairs of BMW Motorrad—helped to get users into that crowd and to build it up. Jan is also facing some challenges, I guess. And he will now tell you about what he actually experienced during our time with Testbirds.

[Jan Bode]
Exactly, thank you. So, to give you a little bit more of a background, we started working with BMW Motorrad in the fall of 2018. So, and Jann has already mentioned that we had precisely zero suitable testers in our community of way over 300,000 people back then. So, everything that we built here in terms of this testing community for BMW Motorrad, we built from the ground up basically.

The first riders that we recruited for this community were all mostly from Europe, mostly Germany and the UK with some testers from the US, but then very soon winter came. So—and when we sent our test invitations where we need to testers to take their BMW motorcycles on a ride to test the new beta version of the app, some of the testers responses were just a photo from this street covered in snow. So, not the ideal riding conditions. And from that point on, we realized that this community needs to be a worldwide one, where riders coming from everywhere can participate. And somewhere on the planet, there’s always decent weather for riding a motorcycle. So, what we basically did is we also approach motorcycle clubs everywhere in the world. So, there’s literally a motorcycle club for BMW motorcycle riders in Namibia or in Peru. So, that’s what we did in the first place. We approached these BMW clubs and—yeah, we laid out the benefits for riders if they participate here. And for those riders benefits aren’t just monetary, the bigger incentive is actually being part of this development to help BMW test their products, to give feedback, to participate in interviews, and make their voices and opinions heard. Because everything that we do here is going straight into the product team at BMW, and—yeah, to a lot of people, being able to do this even better than the monetary aspect that we also provide to everybody who participates in our tests.

The next thing is authenticity. So, talking about authenticity is—it’s a challenge that we have been facing all the way through the project. We are still Testbirds. We’re a small company based in Munich and not many people have heard from us, especially not motorcycle riders in South Africa. So, naturally people ask themselves, why they would put their trust in us, or if we’re even working with a company, like BMW, or why they should provide us with their personal data. Our task therefore is to create a level of trust. And fortunately, Jann from BMW is very cooperative in that manner. So, we shot a welcome video where Jann explains our reasons for the testing community even existing. And we also attach, sometimes videos to tests, or we give an update on the development status. For example, during the times of Corona where testing has slowed down a little bit, just to bridge the gap a little bit, and all of this direct communication helps us tremendously to create the level of trust that we need to keep our testers active and motivated and engaged.

However, having gained people’s trust, and having them sign up with us is only really the first step to—as successful testing experience. We have to remember that we are dealing with the target group that was a little bit older, and testing is maybe not really familiar to them. So, just for comparison our average tester at Testbirds is around 30 years old. And we also test with the car side of BMW, and the testers are on average 37 years old. And our motorcycle riders are 48 years old. So, we’re not really talking about digital natives all the way through. So, yeah, we need to provide a certain level of guidance and assistance to them all the way through, basically, from explaining how the testing works, and troubleshooting within tests, and also making them understand how the development process works. But, fortunately these testers are some of the most enthusiastic and passionate, and also relaxed testers that I have come across, and that’s just great to see. So, to bring my part to an end, this is the current status of the testing crowd that we’ve built up for BMW Motorrad at its current stage. So, we have really testers everywhere. So, in over 30 countries and more than 250 testers in total. So, again, somebody always has good weather to take their motorcycle on a ride, and test the new beta versions. And, yeah, back to you, Jann.

[Jann Kirchhoff]
Yeah. And this is something we are also very proud of that we have such enthusiastic, not only users and testers, but also fans that are supporting us in the development process, and that are really passionately testing our app, getting us feedback, sharing their opinion. And Testbirds, in that game is really the one who catches them all and who consolidates this feedback to an understandable piece of information that we can use in our development process. And, this is also the big value of having a partner like Testbirds.

If we are talking about challenges, and about, like—challenges, especially if you are a corporate with a lot of multinational-corporate, then it’s pretty tough to get through to the actual user. So, as Jan already mentioned, it’s pretty easy to say, “Yeah, let’s build up a beta user group.” But if you are a multinational-corporate, it’s not that easy. You have to get through to the actual user, and you have—if you are in the headquarter, you are—basically, a product team, you have to go through your sales department, you have to also go through your national sales corporation, and then you need to approach a dealer, maybe—in our case, a dealer—and then you have to go through the user. And ideally, it works like this. So, everyone is bored, and of course, has time to do the process to sign-up, like, data privacy statement, and stuff like that. But, let’s be honest, this is not reality. Reality more or less looks like this. So, it would take us ages to get through this, like, hierarchy, this organization, and in the end, all those people need to do their jobs, they need to serve the customers, and they are not a beta-testing department.

What we did and this is for me as an employee of a corporation, of a big corporation—this is the—one of the most important benefits of Testbirds is, we installed the bypass. So actually, Testbirds is doing all the heavy lifting for us, gets through to the user, and works with them and builds up this beta-user group without bothering our whole organization. And we can work really agile and really, really fast with the users and get direct customer feedback. And we just want to share a short video that basically shows how close we are with the user, and how passionate they work for us, and they want to share their insight, they want to share what they found—bugs, and so on and so on. This video was taken by one of our riders, and he’s basically recording the TFT screen for like—Jan, I don’t know how long the video was—like, 30 minutes? And he was editing it after that, and then he was also commenting what actually went wrong. And we were totally stunned by the way—how he leaned in, basically. And, what we did, we decided to send him a small care package BMW Motorrad, together with Testbirds. And something that was absolutely unforeseen was that he basically posted this in Instagram. And he was really thankful for [Inaudible – 16:22] to him. And he was really, like, wow, he didn’t expect that. And, this is the kind of personal relation we want to build-up with our fans, our testers. And this is what I basically really love about working with Testbirds and our customers, that you can get directly in touch with the actual user that is doing—that is using the product.

Talking about product development, of course we are using a lot of tools. And this is maybe the last point I want to come to. I mean, I think a lot of those tools are well-known, if you talk about Jira, Confluence, Bitbucket, Jenkins, and so on and so on. And Testbirds is one of our tools to make that certain product happen. And in a nutshell, if someone would meet me on the street, and would ask me, “What are the benefits of Testbirds? Why do you need Testbirds to do your job?” I would say, “First of all, it’s testing done by real authentic-users, so they are not biased, they are not—they really paid the money for the product, and they want to get the value.” Also, “Testbirds is a good instrument to motivate our community and to really maintain our fanbase in that small beta-user group.” People really feel special about being part of the product development. We can fastly generate new ideas and also validate our ideas if we are on the wrong track or on the right track. And, there is a huge geographic coverage, so we can easily do a test drive in Spain, if we need it tomorrow.

One thing that is overseen by a lot of product development team is localization coverage. So, we can check for localization coverage, and we can check if our translations are actually right or if we have a bad translation quality. And of course, in the end, there are a thousand different models of smartphones out there, and we are not alone on those smartphones. We are next to other apps. There are system settings talking about like, energy-saving options with Android. Testbirds and the test crowd and our testers are a great opportunity to test our app in the real world that is not isolated in the laboratory, before actually releasing it. And this makes Testbirds a great tool for us. I think, I’ve come to an end. Famous last words, Jan.

[Jan Bode]
Well, I think you made it pretty obvious how Testbirds is helping you in the product development. And, I think, one point that I would love to add is you’re also doing a lot of interviews with our testers. So, when you’re developing a new product, it doesn’t even have to be for the app, it can just be anything. And you’re early in development stage, and you can basically reach out to our testers directly, have one of our Project Managers on it, and interview the testers to see and validate if you guys are on the right track, if the product or the prototype makes a lot of sense. And I think that is also something that helps you a lot on your way to developing the final product for this customer, that is, in the end, who you want to buy the product. And, yeah, I think that’s something that I want to add in the end. There’s not much more for me to say, besides thanks everybody for listening, and—yeah, hopefully see you soon. Bye.