Why Measurement Matters – the case for analytics

Creating a successful app business takes a lot more than a good idea and the skills to develop an app and upload it to a store. As we’ve discussed before, developers who promote their apps are almost 3 times as likely to break-even as those who don’t. This is the simplest difference with a massive effect on success. It seems obvious…

Creating a successful app business takes a lot more than a good idea and the skills to develop an app and upload it to a store. As we’ve discussed before, developers who promote their apps are almost 3 times as likely to break-even as those who don’t. This is the simplest difference with a massive effect on success. It seems obvious, no marketing means a good chance almost no-one discovers the app and thus no revenue, yet some developers still don’t do it. Picking the most lucrative platforms seems like another obvious candidate; considering only those developers who are interested in making money, those that develop for iOS are a little less than twice as likely to be above the “app poverty line” (38% earn over $500 per app per month) than those developing for BlackBerry (20%). Getting your revenue model right can also make a significant difference; according to our survey, developers using a subscription model earn nearly 2.5 times as much as those using other revenue models on average. Not all apps can use a subscription model, however, there is something that almost every developer can do which is correlated with more than double the chance of being above the app poverty line and earning more than 2.5 times the average revenue compared to those that don’t…

Measuring user activity and crashes

You managed to build an app and get people to download it but how many of them still use it? When do they use it and how often? What devices do they have and which firmware versions? What features do they use the most? Which parts of the app should you focus on improving? Do your changes to the app make users interact with it more or less? All of these questions require measuring the user interaction with the app by recording data and analysing it. Unless your app regularly interacts with your own backend service which you can collect relevant statistics from then this would be an expensive capability to build. Fortunately there are a number of third party usage analytics services to do it for you and the two most popular, Google Analytics and Flurry, are free to use.

Similarly, no app is perfect when it is launched and it’s almost impossible to test on every device and firmware combination out there in the market. If some of your users are getting crashes that you can’t reproduce and fix quickly then you’re likely to get a lot of poor reviews, which will reduce the chances that other users download your app in the future. Unless you have tooling in place to capture details of crashes and report them to you then it’s very hard fix them. Although there are libraries that can report crashes to you directly, unless you want to analyse every single one manually, you’ll want tools that do that for you and categorise them such that the ones with common causes are grouped. Without this there’s no good way of prioritising what to fix. Again, crash analytics & bug tracking service providers can handle this for you and one of them, Crashlytics, was recently acquired by Twitter and now provides all of its features for free. Other providers also have free tiers.

This is not to suggest that the free options are the best for every app, just that there’s very little excuse for not using these tools because there are free ones. If you need convincing, take a look at this infographic, which uses our survey data for just the respondents that were interested in making money and earning less than $50k per app per month (we exclude the very small number of top earners as they can distort the stats, although in this case they would only make the argument stronger).

Correlation is not causation

It’s really important that we note a strong correlation between usage of analytics tools and increased success for developers only. Correlation is not causation. It’s clear that simply integrating these tools and doing nothing with the data is not going to make the slightest difference to anyone’s results. They are trivial to integrate but provide no direct end user benefit at all and no additional revenue stream. It could be argued that most providers only make these services available for the leading platforms and the extra success is primarily due to that. This is not the case – restricting the data set to those developers whose primary platform is Android or iOS produces an almost identical pattern of results. Another argument is that crash analytics services are typically focussed on native stack traces, which don’t often provide much diagnostic value for non-native apps. However, anyone tempted to blame a poorer non-native user experience produced by development tools that are also less likely to support these analytics tools should note that cross-platform tool users make more revenue.

One valid argument is that until you have a successful app with a fairly large and diverse user base, collecting crash data and analysing it is not the biggest problem you need to solve, those who use the services may simply be the ones already big enough to have the problem. There are two arguments against this. First, integrating crash analytics after you’ve acquired a large user base is too late; how many users will give you a second chance installing at least 2 updates to a crashy app (you can’t fix it until the update after you integrate the analytics)? Second, this doesn’t explain why those integrating both types of analytics tool have a significantly greater chance of being above the poverty line than those that only integrate one or the other. Similarly, whilst there is almost certainly an experience factor at work here it is clearly not the whole answer.

The most plausible explanation for these results is that those with a more scientific approach to their app business tend to collect as much data as they can and use it to drive decisions about their development. This produces more successful apps than intuition and guesswork. So, sign up for analytics services, integrate them with your apps and act on the data they provide. This process can’t magically turn a bad idea or unwanted app into a success but it can help make a decent app into a much better one. If you apply some of this data-based reasoning and scientific methodology to deciding what to build and how you market it too then that’s likely to further increase your chances of success.

Got an alternative explanation for this data? Let us know in the comments below.

By Mark Wilcox

Mark is developer who has worked on everything from the lowest level smartphone firmware to games and apps that sell pizza. He's also a project leader with a focus on lean methods and a consultant who loves rapid prototyping, app economics and business models.

Leave a Reply