Is it goodbye or see you later? When users stop using CommCare.

This is post #7 in our Under the Data Tree blog series, where members of Dimagi’s data science team share insights from analyzing CommCare data.

In this blog, we look at how often users stop using CommCare, and—if they do stop—how often they restart. There are of course many reasons for a user to stop using CommCare that are not a function of how well the technology is working or adding value for its users. For example:

• A frontline worker using CommCare may move or quit, and thus stop using CommCare as a result.
• A program might use CommCare for a short-term project that is completed, e.g. a household survey.
• Some programs may use CommCare for a seasonal activity, as is the case with many of our agricultural programs, and so it may be planned for users to stop using CommCare for several months.
• A project may run out of funding and can no longer support the use of CommCare.

First, we start by showing how much attrition rates differ in the first few months of CommCare use vs. later months. The two graphs below summarize data from 19,986 CommCare users who submitted data January 2012 through March 2015. This includes a total of 116,220 user-months of data where a “user month” is defined as one user submitting at least one form in a given month. For each month, we’ll put a user that submits at least one form in one of following three categories:

• continuing (green): if submitted a form this month and the following month
• skip and restart (yellow): if submitted a form this month but not next month, but does eventually submit a form in a future month.
• stopping (orange): if this is the last month the user submitted any forms

We split the data into two groups: (1) users who had not yet submitted forms in three different months (on the left), and (2) those who had submitted forms for at least three prior months (on the right). Because a user can restart anytime, it is possible that a user currently listed in the “stopping” layer could submit a form in the future, and would then be re-classified into the middle “skip and restart” layer if we were to regenerate this graph. We stopped the graph in March 2015 to minimize that possibility.  

The graphs show that during the first three months of use, it is fairly common for CommCare users to stop using CommCare, whereas after three months of use there is high rate of continued usage. We can also see that the majority of users who stop using CommCare in their first three months of usage do not resume using it (that is, the “stopping” tranche is bigger than the “skip & restart” tranche), whereas the opposite is true after three months of using CommCare. In fact, users who stop using CommCare in their first three months only resume 41 percent of the time. After three months of use, 65 percent will restart after stopping.

In future blogs, we’ll investigate other aspects of attrition, but these findings suggest that it is worth spending special attention to users during the first few months that they use CommCare,  since attrition rates are so much higher during that period.

Interested in this topic and want to learn more about attrition rates? Click below to get in touch with our team.