Getting The Progressive Web Apps Edge

So, we'd like to call ourselves, not Amazon of India. In fact, we would like to call Amazon the Flipkart of the United States. Anyway, I was going through these slides last night, and I was told I only have 20 minutes. So, I will try to squeeze in as much as I could about the journey we've had in the last two years. I have to skip over some of the slides. So, please feel free to catch me off the stage if I didn't cover any of those details that you would like to know for me? How many of you still do this on your mobile sites? How many of you throw these badges on your mobile sites? Please install our app, please install our app. Do you guys do that? Yeah, we used to do this in 2015. In fact, we went through the other extreme of shutting down our mobile site, and just put up this one page saying, please go install the app. We thought, just because the users can come on to our app, which obviously performs better than the Mobile site, we would throw out numbers just in the other direction. Guess what it didn't work for us and the blessing in disguise was it actually led us to a journey that we're all proud of today.

Before we get there, what is actually app means to the user, not to us, to the user. Users have this perception that apps work offline, not every app is actually built to work offline. But they do have this perception that even offline, I open the app, I could see something which does not work in the web properties. With the ability to actually have its own dedicated space or focus on the user screen, you don't go to one utility app to launch this thing. You have your own space for the app, which also means the user gets it as his or her top of the mind when they see the app. And most importantly, the push notifications. We just had a big billion days. It's like Black Friday of India which started this construct four years ago, this is the biggest sale that India goes through in a year. Some of the numbers that we just told about Flipkart. Those were the numbers that we used to do in a year, we just did those numbers in five days. That's how big this whole construct has become for us.

Do you how much we actually spent on marketing this time for that event to reach a country of a billion people? Okay, forget about billion people, around 300 million people, or 300 million devices online. 100 million active shoppers in the country. We spent less than $2 million. Do you know why? Because of the push notifications, we were able to engage users very contextually, right in the moment about the offers that we were actually enabling for them. And it was very personalized to the extent that if you are a customer that was actually looking to buy a new phone, we targeted you with the right kind of push notification, and the conversions were like never before. And these numbers we would have not met if it wasn't for these contextual and more relevant push notifications, again, an advantage that you get with native apps. But let's see this in the context of a web property, offline behaviour. Whereas the web property actually gives you no weightage, no baggage to install an app, you just go to a browser and type in a URL, you're good to go. While you get the home screen thing, you have incredible URL reach, you can pretty much give a URL to every single page in your website, you can actually take users to wherever you want in your property. Push notifications, yes, but the tabbed experience that users were always used to, especially in the E-commerce space, where people are used to comparing prices, SLAs, all of that is gone. You're in a walled garden or in your own container. So, we actually thought about it. How about actually bringing the best of both worlds together? This is actually a story I was just sharing with some of the folks from Google, and how the Progressive Web Apps happened. 2015 when we shut down the mobile site, the first people to call us. Any guesses? Google called us, why the hell did you guys shut down your mobile site? Because fortunately, or unfortunately, we are so big in the country, whatever we do makes news. So, it became such a big new thing. Flipkart shut down that web properties and if you just put the dots together, web, SEO, ad, money, Google, so Google was worried.

So, Google called us up and said, why the hell? He said, guys, it's not working as your own Android app, native app. So, it doesn't make any sense for us to invest in this mobile site, knowing that the customers are not buying on our mobile site, so we don't want to do it. Then Google actually asked us, what do you mean, what do you want from your mobile site? Are there any web technologies that need to be changed? We said, no, it's just a browser, at the end of the day, no matter what we are writing as code, it has to actually run in the container called a browser. If the browser cannot do things, which would actually be only possible in a native app, there's no point in actually doing anything. So, then they asked, what is this browser? What should the browser do? We said the same thing. If it can do all of this, then yes, we will actually launch our mobile site on this. That's when this happened. Progressive Web Apps. So, this talk is actually about a very high-level journey of what really changed between traditional web development and progressive web apps. So, this is pretty much true with almost every website today, in Standard Model. There’s a browser, there's a web server, there's a request going to the web server, which actually goes all the way to your back-end API and then the response comes back. Then you have CDN, obviously, for the static content, unless you have edge side includes. Pretty standard one. It worked actually for no JS browsers as well.

In fact, one of the data points from India is, the most used browser in India, the mobile browser in India is? UC Browser because they actually offer a feature called data compression, which actually means a no JS browser. So, users have this perception that the data will be used less if they use if the browser on UC browser. So, we actually had to do a no JS version of our property to just work on the browser. Again, it was not working for us, UC browser was not meant for transactions, it was only meant for viewing. More than anything, this is what happened. With the server-side rendering your page load time suffers. How many of you agree with that? Guys don't agree with that. So, we had to actually reject it. So, what did we change? It's pretty much the same thing. First Page Load is still the same. We didn't change anything in the first page load, I'll talk about how we actually address the first page or first paint time. But what we introduced was something called Service Worker and this is what the Service Worker did. After the first page load, Service Worker actually reaches out to all the content, it actually acts as a network proxy to go and fetch the content in a more intelligent way. Your application context, your user context comes into play and based on that, we were actually able to fetch the data in a way that actually made more sense to the user and was perceived as performant. So, what does Service Worker actually mean? It's as simple as a script run by the browser in your background. It's like the shared worker. This is one fundamental difference between the web and the native space. In the web space, you have just one main thread doing everything, from fetching data to laying screens to literally intermixing all of this by one main thread. Whereas in native apps, you have worker threads. Service Worker actually brought that functionality to the web and you have now the ability to run JavaScript before the page even loads. Okay, now I have to really move fast. I'm being told from behind. What did we do on the client-side? We actually went to a single page app. There are challenges with the single page app, other than the SEO and all of this, but we solved it with the right selection of technologies. What we actually did at the end was we actually did a very lightweight implementation of flux pattern. We open-sourced it. If you guys are interested, you can actually look at it. It's called front end JS. It's a single page app, where we're actually using React JS, and, of course, the React Router, just to keep the URL mapping to the content on the site.

So, we were actually able to do push notifications. I'll actually show what offline first means we did background sync, and more. Here's the challenge. The first paint still suffered. Because if you remember from the diagram, we were still actually getting the first page load like the conventional ways. So, what did we do? We looked at our application, and we divided the application state into two states. The loading state and the loaded state. What is loading state actually mean? It's the state in which your property is before anything reaches your browser. What does this mean? This is what happens when you type in flipkart.com. You're typing in flipkart.com and did you see a blank page? It's almost not there. So, that entity that actually loaded, the moment you hit the URL is called a shell. So, we introduced something called HTML page shell, it's actually very simple to do. I'm sure you guys must be using Webpack to create the bundle. So, we actually send all our core components through Webpack and we create bundles and we run render cycle on it to create page shells. So, these are the page shells that are pretty much the skeleton of your page, which is giving a structure to the kind of data your page would load like. So, this is what it actually does.

So, on the first page load, we are not waiting for the entire page. Now we are only waiting for the shell, which is very lightweight compared to the entire page contents, or the first fold of the content, it's much lesser than even the first full contents. Other HTML shells will again be fetched by Service Worker and the most important thing are the advantages that we have with page shells is they can be cached; they can be rendered in the loading state. Once the data is fetched, if you have to re-render, you can re-render them and if you're actually using something like React, you can actually re-render the partial areas of your shell, you don't have to re-render the entire thing, which is one good thing about something like React. On top of it, SW toolbox, we actually use that to fetch the data. SW toolbox is a network proxy library built on top of Service Workers. We were the first ones to actually use it in our Flipkart light when we launched progressive web app in 2015. So, now here is the challenge. Now, you've solved this problem. Now you have too many shells and every page technically have a shell because every page could be a completely different structure altogether unless you have only one structure for everything. Then what would happen if you have too many page shells to handle with, which again defeats the whole purpose. So, if you had to make a choice between one big monolithic single page app versus multiple single page apps or smaller single page apps, so, we went down the path of multiple single page apps, which actually gave us the flexibility of building and deploying each of them separately. It wasn't that trivial, we had to actually share the common libraries and utilities. In fact, our front-end JS, the flux implementation that we did, takes this into account very intelligently. The one more thing is SW toolbox actually supports the route based fetching model. So, you can get all of this, you can actually see the difference in on a 3g, these are the response times you're looking at, because of the bundling because we were able to split JS into multiple JS’s, we were able to split it into app shells and all of it. So, your page loading is no longer dependent on one big chunk or one big file that has to be downloaded.

On 2g, it pretty much remained the same. Because the content is so less right now for the first meaningful paint to happen. So, the network technically went out of the picture here. Again, Add to Home screen was one thing that we were able to do with the manifest file. The splash screen is one more important thing, you can actually see this is actually programmatically rendered splash screen. Splash Screen also turned out to be one of the very important features for us. In fact, this version of Splash Screen is from when we launched progressive web apps. If you actually look at our splash screen today, we're actually using it to drive any new features, new product launches, all of it when we are trying to actually load the page shells and the content behind the screen. So, we were able to capture the attention of the user while we were doing something in the backend to load the homepage offline experience. So, the network proxy that I was talking about where we were able to prefetch the data, it's actually sitting on your device.

So, we thought actually, we could use that to give an offline experience because we already fetched all the content that you browsed, that you looked at. So this is what we did with the offline thing. So, offline mode, and it's visually clear that you're in offline mode and this is completely working in an offline mode because it's the data that we fetched anyway when you are browsing through these things. In fact, the network proxy gives you the flexibility of fetching data that users didn't even browse. If your business logic actually expects the users to do certain parts of your app offline, you can actually do go fetch that data when the user is online and can give a very relevant offline experience. And then we started having, thanks to Chrome and thanks to all the other browser vendors, now, the browser also gives us access to a lot of hardware API's. Geolocation, it's very important for us as a marketplace, we cannot sell everything in every part of the country. So, it's important for us to know where the user is coming from and it's very important for us to tell the customer that in your location, we cannot sell this TV and it was just a pain point for us to ask the user, please type in your PIN code and all of it on a mobile site. Now it's geolocation, it's actually precise, compared to a native app. Accelerometer, in fact, we could not figure out the use case for this in the E-commerce thing. So, we actually introduced an Easter egg, where if you play with the app in certain directions, it unlocks an offer for you. So, it's something users keep playing with it. This was the part I really wanted to cover. Performance. This is actually the only feature the engineering team in Flipkart brings to the table. While our product team is actually busy bringing all the features that the users would like to see from Flipkart, this is the only feature the engineering team is working on. Performance, it's something that we do not compromise on, because the whole premise of actually building progressive web apps and for that matter of Android app, which happens to be the fastest Android app in the country, and the fastest in the space across the globe, ecommerce app, because we take performance very seriously. But here's the question, what is actually performance? What does this thing mean?

People actually have their own perception; you must have heard tons of advice. DOM manipulation or DOM changing is actually slow, CSS animations, all kinds of stuff that you keep hearing about what is best, what is not good for the performance of the app. People throw out like first paint to DOM content loaded to frame miss. People use different metrics to measure the performance of their properties, be it an ecommerce property, be it a gaming property, or whatever. But nobody ever actually asked the question. What does the user feel? So, we actually started looking at our own user studies, only to discover that somebody actually made this research way back in 1993. So, Dr. Jakob Nielsen, wrote this amazing book called usability engineering in 1993, where he called out three important time limits for a user property. The first one, your reaction times should be less than 1/10 of a second. When a user does something on your glass screen, the pixels that need to change under the glass screen needs to happen in less than 1/10 of a second. Your flow changes your progression, based on the user action needs to happen in less than a second and most importantly, your app or your site should be 60 frames per second.

So, that's how each frame should be rendered in its 16 milliseconds if you just do the math. So, it was totally 1993. Why is that no property or why is that most of the properties are still not as performant as they should be. Because nobody ever did any mapping of what these three timings actually map to the real time scenarios. That is where this model came into existence. This model actually, brilliantly explains how those three numbers match to your real time scenarios. In fact, they did a very clear grouping of user activities on your property into four things. Response, animation, idle and load. Response, input feedback in less than 100 milliseconds. Your feedback should be fast enough. If it is less than if it is more than 1/10 of a second, the React-action-reaction thing breaks for the user. The user actually thinks that it is not working properly. The second thing is animation. You don't do animation, just because it's cool. You do animation because you want to give the seamless experience to the user to seamless transitions, but if you don't do it right, can you see the difference? The first one is actually moving at 60 frames per second, the second one at 30 frames per second, the last one at 15 frames per second. It could be a problem to the customer, in fact, you're actually making it worse and not everything in your application need to happen in real time. A very good example is we collect data when the user is actually on our property, we do not send the data back to our servers in real time we actually batch it. Because if you have to send it in real time, you're using IO, you're using threads here, which could actually be used for the operations a user is expecting to see from the property. We do it offline. We do it asynchronously. So, you actually have to make a call between what are the user critical operations and what are the operations could be done asynchronously

And the last but not the least the first meaningful paint. It's not the first paint, it's the first meaningful paint. If you actually look at it, the way we actually render our property is the first thing that you see is a search bar. 67% of our traffic interacts with Flipkart using search. So, the first thing that we load is a search bar. So, we measure our performance by the time it takes to load the search bar, not just the first file that was downloaded to the browser and yeah, this is the very first version of Flipkart Lite. In fact, if you see the Flipkart Lite today, it's much faster and much better performing but I wanted you guys to go back in time when we launched it. In 2015, this is how it was when we launched it. Look at all the animations, look at the fluidity in animations. And last, but not least, I was just talking about the big billion days. 2015, when we started Flipkart Lite, it was an experiment with our friends from Google. Today, it's the second most visited channel for us, which is bringing in 30% of our revenue. So, the performance of your mobile web properties can change the game for you and we are actually telling it from our experience. Thank you, guys.