Does technology matter in CommCare projects?

Over the last fifteen years, feature phones capable of running third-party apps superseded basic voice-and-text devices. More recently, smartphone penetration has been increasing rapidly in the developing world. In particular, Android devices are now widely available in most countries, and over 10 million new smartphone connections were made every month in Africa during 2013 and 2014 (GSMA, 2014).

This post explores some mobile technology trends across a large number of projects using CommCare.  The CommCare mobile app runs on both feature phones and smartphones. Data can also be collected via SMS or the web. This allows us to track trends and compare usage patterns across different technologies.

Users by device type, Jan 2010 – Sept 2014

(moving the slider zooms the timeline)

Our data shows that Android is becoming increasingly dominant within five years since the release of the first consumer Android device in 2008. Looking at CommCare projects, we find that while in December 2010 there was one project using Android phones, in January 2011 there were 18, and in September 2014 there were 171. Most self-started projects (i.e. CommCare projects created by organizations themselves, without Dimagi involvement) chose Android – of the 63 currently active self-started projects, 55 (74%) use Android, while 14 (19%) use Nokia (the remainder use Dimagi’s Cloudcare web client or a mix of devices). For Dimagi-assisted projects, the percentages are 51% and 20% respectively. This is probably because installing third-party apps such as CommCare on Android devices is much easier than on Nokia feature phones.

In the maps below, we compare the percentage of CommCare users on Android by country for September 2012 and September 2014. Although Android predominates by 2014, there are still a significant number of Nokia users in East Africa (Kenya, Tanzania, Mozambique, Zambia) as well as in India. This coincides with the location of the earliest CommCare projects, which started with Nokia feature phones before Android phones were widely available in the region.

 Android users by country (%), Sept 2012

 Android users by country (%), Sept 2014

The data also allows us to compare usage patterns for Nokia and Android such as the number of forms and visits per month, since multiple forms might be completed in a single visit (e..g forms for both a mother and baby). In the table below, we compare Nokia and Android users in their sixth month using CommCare.

Usage patterns for users in their sixth month on CommCare, Nokia v. Android

From this, we can see that Android users do seem to complete more forms and more visits than Nokia users. They are also active more days of the month, and spend more time on CommCare. Retention three months later (i.e. at 9 months) is high for both groups of users.

A Kruskal-Wallis non-parametric test on means by domain confirms that the device type differences for median number of forms, visits, percentage of days of the month in which the user submitted data, and time using CommCare per user in their six month are significant (p < .05 for the first three, p < .01 for the last). There isn’t a significant difference in the median percentage of users who submitted data in both month six and month nine.

Conclusions

Overall, the findings suggest that Android users generally have higher volume and more frequent usage. However there are a number of other factors to consider. It may be that projects with a larger budget for devices (allowing them to choose the more expensive Android devices) also have more money to spend on training and supporting remote workers. Outcome measures based on the number of forms or visits also don’t consider the content of the interaction, which may involve a lot more than just data collection. Nokia devices have better battery life, are more durable and often easier to learn for users without smartphone experience. They work well for projects with mobile workers and clients in particularly remote areas, where greater travel time may account for fewer visits. Given that retention rates at nine months for users active at six months are not significantly different, the data suggests that established users are not particularly sensitive to device type choice.

Perhaps the most reasonable conclusion is that technology does matter, but not necessarily the way we might think. New technologies have new features and new possibilities, but aren’t necessarily suited to every situation. Nokia feature phones designed for use in the developing world are rugged and resilient, and few smartphone models can match them. Android phones are better for multimedia and location capture applications, and generally have a better user experience for high-volume users.