Paper to Digital Data Collection

A few years back, we launched an Innovation Pilot in Niger, equipping 100 agricultural field workers with mobile phones to manage their data.

These workers travelled long distances and worked even longer days to create irrigation channels, advise on livestock production, and help foster micro-enterprises. They also recorded all their activities on paper records and spent their evenings copying all the data from paper registers into their phones.

Duplicating data in this way was supposed to be a temporary measure, in place only until the government administration could confirm the quality of the data from this digital approach. At that point, they were meant to develop a plan to phase out the paper workflow.

But weeks passed. Then months. No test or roadmap was developed to phase off paper.

Moving off paper is a central and inalienable part of making mHealth interventions work

It turns out any organization whose funding depends on good data is naturally disincentivized to risk anything on digital.

The Monitoring and Evaluation (M&E) Officer was responsible for the quality of data, so he was not going to champion the removal of the duplicate data collection systems. The Program Officer deferred to the M&E lead for anything related to data, including which systems to use and when. And the  program funders were enjoying the positive press associated with the launch of a digital program, even with the paper-based system running in parallel. None of these groups were accountable for the painful duplicative workload burdening the field workers.

Moreover, nobody wanted to accept blame for the challenges introduced by the new system. So, the path of least resistance was the most conservative approach: Keeping both paper and digital systems.

A few months later, the Chief of Party (CoP) was visiting the communities in Niger. He talked to his field team, and they told him the truth: The phones were adding an unbearable task at the end of an already full day.

At the same time, Dimagi was working with the program’s M&E team to analyze both paper and digital datasets in an effort to provide evidence that the digital solution was performing well. It was an onerous process, in large part because the paper records were spotty.

After several days of analysis, Dimagi had gathered compelling evidence that the quality and consistency of digitally-collected data was always higher than that of data collected with paper forms.

Our findings emerged the week before the CoP returned from his field visit—and after he had already decided to discontinue the digital pilot. The phones were retired, and the data collection workflows were all reverted back to paper.

This story is notable for how un-notable it is.

The majority of mHealth deployments we see consider migration off paper as an afterthought–a post-pilot part of the hand-off to government, which is entirely outside of their control or jurisdiction. As such, most of these programs never succeed in moving off paper at all.

It seems one of the great aspirations of mHealth—to make the work of frontline development workers better—can be a far cry from reality.

But is digital an effective means of data collection? Is it actually better than paper? Let’s probe a bit deeper.

Imagine if someone asked you to manage your money on paper or with an abacus, just to understand and measure whether online banking works. Or picture spending months sending your friends email, then sending those same messages by handwritten letter on carrier pigeons, in order to measure which method has a higher rate of delivery success.

You want me to do what now?

We wouldn’t do this to ourselves. So why do we ask our agents working in some of the toughest environments in the world to jump through such hoops?

If we find ourselves in these situations, we are missing the point of digital.

Take the launch of e-visa programs over the past 15 years. By comparison, multiple countries have chosen to not require travelers to complete both paper and electronic application forms. These countries typically ran an initial test, verified the system worked as designed, and then launched. Yes, many nations maintain the option of using paper-based visas, but none require both paper and electronic submissions.

How do you prove the new system works if you’re comparing it against broken paper records?

Moving off paper should never be an extra step tacked onto the end of an mHealth deployment. Rather, it’s a central and inalienable part of making the intervention work. It is the value that we as mHealth professionals create.

When development programs make digital systems a part of their strategy, going paperless must be baked into the very first elevator pitch or drafted concept note. If we’re not replacing some part of the paper-based process, then we’re creating additional work.

Going forward, let’s stop asking “is paperless possible?” and “how do we evaluate when digital exceeds paper?” Instead, the question is “when do we have confidence our digital systems add value, are robust, and are user-friendly?” When we can answer this question in the affirmative, digital will be a reality, and paper can be dropped for good.