Why Developers Should Embrace User Analytics

In our latest Developer Economics survey we were really surprised to see a dramatic fall in the level of adoption of User Analytics tools. A year ago they were the most popular category of third party tool with 38% of mobile developers telling us they used them. This time, only 21% of developers said the same thing. Why has there been such a significant drop? Are the hordes of new developers flocking to mobile just unaware of what’s available? Is all analytics being tainted by the spying scandals? Why does it matter?


The new mobile developer tsunami

At WWDC this June, Apple said that they’d gained almost 50% more developers in the last year. The vast majority of those will be iOS developers (rather than OS X and Safari). Android has been growing even faster in terms of new apps, so it’s likely there was even greater growth in the mobile developer population there. Windows Phone also saw significant growth in developer adoption. With so many new developers getting started building mobile apps we’d expect that they’re not all aware of the tools available to help. However, the reduction in user analytics adoption was greater than for other tools, indeed some tools categories saw increased adoption. So, although lack of awareness is clearly a contributing factor it’s not the whole story.

Tracking behaviours, not people

Information on government spying activities leaked by Edward Snowden has turned sentiment amongst many of the technically aware against digital surveillance. Perhaps some developers feel they shouldn’t be spying on their users’ actions? [tweetable]Does adding analytics to your app violate user privacy? Not usually.[/tweetable] User analytics tools are primarily for examining correlations between user behaviours and changes in the product or its marketing. If you make changes to a sign-up screen you need to see if it improves the fraction of users who sign up. If you’re trying to convert free users into paying users, it’s vital to analyse which changes in design or pricing result in greater revenues. Even to improve a paid up front or entirely free app, understanding what users do when they’re using it is incredibly valuable. As long as you’re not collecting personally identifying data and associating it with activity then it’s hard to see this as spying. Even segmenting users for behavioural targeting of functionality or advertising seems harmless as long as they’re anonymous. When analytics providers aggregate data across apps to build up user profiles for advertising, that may cross the creepy line for some. If this bothers you then choose an analytics provider appropriately, or collect your own data. If it doesn’t then having a privacy policy that clearly states who sees what data and what it’s used for is the responsible thing to do.

Coding blind

[tweetable]Updating and iterating on apps without analytics is apparently incredibly common[/tweetable]. It’s unlikely that all of these developers are regularly running user testing panels to get detailed feedback. Is everyone coding blind? To avoid constant app updates which would annoy users (and not even be possible with the review cycle on iOS) developers need to include multiple changes in every new version. If user sign-ups or revenue goes down after an update, how can you tell what went wrong? If things improve, which changes helped? Which parts of your app do people use the most? Are you working on improving features that no-one is using anyway? Without some kind of analytics, how do you find the answers to these questions? Might you be wasting a lot of effort?

Just integrating a third party analytics tool isn’t enough to answer these questions either. Apps need to be instrumented to log analytics events for every major action in the application and how long users spend on various activities. Beyond that, to really understand the effects of changes, users need to be analysed as cohorts – groups that started using the app around the same time. The value of an app to its users may increase or decrease over time and users who have got used to a UI are more likely to be unhappy with any changes. For an app that’s just starting to grow a user base, a change that causes it to lose 10% of existing users might still be positive if it increases revenues from new users. To know the real effects of some changes, it’s necessary to analyse specific sets of events as funnels. For example, in a registration or purchase process, increasing the number of people that start the process is no good if the number that end up completing it is reduced because they feel misled.

Tool choices

Developers that decide to embrace analytics have a lot of choices, however, the market is quite polarised. By far the most popular tools are Google Analytics and Flurry. These are (mostly) free. Google Analytics has the advantage with Android developers because it is integrated with Google Play, allowing direct tracking through the acquisition and download process into usage. It does have a (very high) limit on usage. If your app gets incredibly successful you’ll have to use the premium tier, which has a flat annual fee of $150k. Flurry on the other hand is free at every scale, although not as advanced as Google Analytics in terms of segmentation, funnels and visualisation of data. Both services are collecting data across apps, effectively to sell to advertisers.

There’s also a premium-only market with better support for cohort analysis, funnels, segmentation and visualisation. Paid offerings also link the analytics direct to actions, allowing automated real-time messaging to users, or the sending of push notifications to specific groups. Mixpanel and Localytics are two examples of tools in this space. These premium tools are also designed to be a lot more usable by non-technical staff, allowing a marketing team to analyse and react to analytics without always needing help from the development team. These tools have a fairly low user or datapoint count for a free tier and then start charging hundreds of dollars per month for successful apps. However, this means that they don’t sell the data from your app to anyone else.

A third option is the relatively little known Amazon Mobile Analytics solution. So far it’s relatively basic on the analysis front and doesn’t have the prettiest interface. However, you can collect 100 million events per month for free and beyond the free tier it’s just $1 per million events. By contrast, Mixpanel charge $2,000 per month for 20 million events. Amazon don’t share your data with anyone or report on it (no word on whether they use it internally for their own purposes).

Give it a try

Thinking about your apps as places to experiment and measure how the users react takes a change of mindset. If you’re not just building apps for fun then this shift can make a massive difference to your success.

If you’ve had a major change in results because of something you discovered using analytics, share your story with us in the comments.


HTML5 performance is fine, what we are missing is tools

HTML5 is perceived as a lower quality platform, mainly because of performance. This comes both as a result of survey data, as well as developer interviews. Yet, industry experts claim the problem is lack of tools. So what is the HTML5 really missing, performance or tools?


In April 2013 VisionMobile asked mobile app developers what stops them from using HTML5. 46% answered “Performance issues”, followed by 37% who said “Lack of APIs” (sample size: 1,518 developers).


We spoke to developers about their views on HTML5 performance. Apostolos Papadopoulos, author of 4sqwifi, a highly acclaimed public WiFi password app, noted “Quality and user experience is top priority for us. Therefore, we prefer going with a Native API”. It’s a common practice for developers to go native for better performance and user experience. But user experience, meaning following the behavioural conventions of the native platform, is a different story and HTML5 can’t help much. Developers can try to imitate but for a truly native UX they have to use Native SDKs; unless we are talking of Firefox OS or the long-awaited Tizen.

Ciprian Borodesku, CEO of Web Crumbz, added “From a business standpoint, there’s a lot of education needed for the acceptance of HTML5. There’s a gap between what we developers can provide and what the clients think we can provide”. The perception of HTML5 being a less capable platform is also common amongst people who commission apps.

Experts point to a tools gap

As part of our How can HTML5 compete with Native? report, VisionMobile conducted 32 interviews with industry experts, from Miško Hevery (author of Angular.js) to Max Firtman (author of “Programming the Mobile Web & jQuery Mobile” published by O’reilly) and Peter-Paul Koch (author of Quirksmode).

It came as a surprise when Robert Shilston, director of FT labs, champion of HTML5 apps, noted that “the biggest issue for HTML5 is the maturity of tools”. He emphasized not performance, but tools, as the key HTML5 gap.

Ran Ben Aharon, head of front-end development of, explained it in more colour: “Hearing Mark Zuckerberg denounce HTML5 made me angry at first, but then I looked at some data and realized that the main reason was not performance or APIs but the lack of memory management and debugging tools”.

Even though developers identify performance as the #1 problem of HTML5, a number of experts claim the actual challenge is tools. There’s no contradiction here, performance and tools are related. How can you improve an app, if you can’t measure it? How can you fix a bug, if you can’t replicate it?

HTML5 is like a car without a dashboard

[tweetable]Tools are to HTML5 what a dashboard is to a car[/tweetable]. You can’t run at high speed without knowing how fast the engine runs or you might end up totalling the engine. Likewise, you can’t produce fast HTML5 apps if you don’t have quality debugging and profiling tools.

With HTML5, coding and debugging are two separate processes. There is no self-contained IDE here. Developers code on the editor (e.g. vim or sublime) and debug on the browser, i.e. using Chrome developer tools. But debugging tools are difficult to master and they require a thorough knowledge of the underlying technology, e.g. what is a reflow, how does the garbage collector work, how is a memory leak created.

Louis Stowasser, author of CraftyJS noted “it would be great to have something like YSlow for game developers”. Why pick YSlow and not Chrome developer tools? Well, because the former offers insights on what to fix rather than data requiring interpretation.

Moreover, each browser has its own set of debugging tools. As a result, [tweetable]developers need to become familiar with at least 4 different environments to match the most popular browsers[/tweetable] of the market. And though it’s generally true that these tools look alike, it’s the little bits and pieces that make the difference.

Patrick H. Lauke, former product manager at Opera Software, highlighted the fragmentation of the browser debugging tools by commenting on a W3C public discussion board about our research: “Opera Dragonfly was the first to offer remote debugging and proposed a unified protocol for debugging. Sadly, other browsers showed very little interest and instead went their own separate ways to build something similar but different”. This also touches on the browser politics issue, due to be the subject of another blog post.

Better tools are needed

HTML5, as far as performance is concerned, is adequate for most use cases. And tools like and Goo Engine provide a testament. The question is no longer *whether* HTML5 can produce quality apps, but *how* easy it is to create quality web apps. What the HTML5 platform desperately needs is easy-to-use debugging and profiling tools.

With the right tools we could see external debugging tools hooking to multiple browsers and even apps able to profile themselves via standard debug APIs.

Web development attracts millions of developers who are new to software engineering because of the learning curve; it’s very easy to get started. The complexity gap between building basic sites and single page web apps (SPAs) is too big of a leap for many to jump over. Improved tool usability is one of the best ways to bridge that gap while also increasing productivity for those already building complex web apps.

What other improvements do you think are needed in HTML5? Download our research and participate in the discussion.