How to Determine Your Method of Data Collection

The entire purpose of a data collection program is the data. Unfortunately, it doesn’t collect itself. So while the “why” of your program is the data, the “what” and “how” is the way you collect your data.

There are many options to choose from and each one has strengths in a different area. The type of data you are looking to collect, as well as the characteristics of its source and environment, should all inform what method of data collection makes the most sense for your program.

You have what you need to make a choice between the data collection tools available to you

How data requirements inform method selection

The data you mean to collect should inform everything about your program – especially the method of data collection you intend to use. The characteristics of those data will mean different things for that decision, and each method has different strengths and weaknesses. This could be simple, like quantitative inputs might be tough to collect from a focus group, but qualitative information (like quotes) could be much easier that way.

Here are a few questions worth asking to determine what your data requirements mean for the method of data collection you select:

How often are you collecting data?

This could mean a few different things. For one, you might know that you will need to revisit certain data sources to collect updated information. In that case, interviews or mobile data collection could help you keep track of that same source over time. On the other hand, if you know you only have a one-off survey, depending on the scale, the cost of setting up a mobile data collection program (including the platform, devices, and data plans) probably doesn’t make as much sense as paper.

What is the scale of your data collection program?

While much information (both qualitative and quantitative) has been collected on paper or through interviews for a long time, it can be hard to scale up these programs. Many national health organizations have struggled with growing successful pilots because they didn’t have a structure or tool that could handle such large demand. In this case, tools like mobile data collection and IVR could help you reach significantly larger populations.

Do your data require technical inputs?

Sometimes, organizations need to collect information like GPS coordinates, video, or even fingerprints. In these instances, you require an additional tool to capture the data. If you’re working with paper, that could mean copying information from a GPS-enabled device onto paper, tracking timestamps from a video camera, or keeping track of photos taken of fingerprints. With a mobile data collection tool, some of these features might be included.

Reaching projects at the last mile requires you to understand the resources at your disposal

Account for environmental factors

Once you have listed, organized, and described all the variables you need to collect, you still need to account for where you are collecting data and who you are collecting it from. Pro Mujer, a women’s development organization in Latin America, recommends beginning this effort with the beneficiaries. By putting them first, you make sure they are the ones experiencing the greatest impact. Understand the data they can provide and the environment they live in to best provide the services you hope to offer.

Here are a few questions to consider when examining your project’s environmental factors:

• What are the languages spoken by the people involved (both data collectors and beneficiaries)?
• What is the reading level of your typical field worker or beneficiary?
• What is the level of mobile connectivity in the region? Is there WiFi available to workers at all?
• How familiar are your data collectors with mobile devices? What is their level of digital literacy?
• How accessible are your sources of information? Do you have a way to contact them? Or will you have to refer to others’ efforts to collect those data?

When you think the answers to these questions might pose a risk to your program’s success, it helps to speak with the workers on the ground. For instance, we have found that when we cannot get a phone signal, local mobile phone users often have a specific spot that works. Dimagi Senior Field Manager Nick Nestle encountered this issue on a field visit in Zambia:

“When we arrived at the village we realized we had no service on any of the carriers. This was a major problem, as the village was remote and traveling to get a connection would be very difficult. Our program hinged on the data being synchronized every day.  

We started brainstorming ideas – could a more expensive phone get a signal; perhaps we could convince the mobile operators to expand coverage (not likely); should we buy the workers bikes to ride to where reception was? We didn’t have any real viable ideas when one of the local village workers overheard and stopped us.

‘No, no, no,’ she said. ‘Do you see that ant hill in the distance? That is where we get our reception. Every day, at five o’clock, I will go stand on that ant hill and hold my phone up in the air to synchronize. It will be fine.’ This was apparently a well-known solution in that village and one that everyone was used to doing.”

Collaborating with the local workers can help you overcome challenges like these. You might know the right questions to ask, but they are the ones with the best understanding of the environment you will be working in.

Some methods of data collections are more secure than others

Storage & security

Certain sectors lean more heavily on this consideration than others. For some beneficiary populations and projects such as those working with HIV patient data, privacy concerns may be much more important than others. Thus, understanding where the data you collect goes is vitally important.

The actual considerations will depend on the sector you are working in. For instance, projects in the public health domain might require you to consider patient confidentiality and HIPAA compliance. The FDA has shared guidelines for the use of electronic health record data that may be helpful for your project.

As you consider the storage and privacy of your data, ask yourself the following:

• Does the dataset need to be de-identified before sharing?
• Do I need to protect certain data after it is collected?
• Who can have access to the data?
• How long can the data be stored?

All of these considerations will be specific to the industry or sector that you work in, while others will depending on local laws or even partner organizations’ codes of conduct. Make sure you are familiar with the requirements of all parties involved before you decide how to collect your data.

There are many tools out there; find the right one for the needs of your program

Summary of techniques

There are so many different methods of collecting data available today. The following are just a selection:

Surveys

Surveys are likely the most famous form of quantitative research. They offer standardized forms for a consistent set of variables to be collected across a wide audience. The three most common approaches to surveys are:

• Mobile: Mobile data collection is the use of mobile devices (e.g. smartphones, tablets, etc.) to administer surveys directly (via SMS) or through frontline workers (via mobile apps). With proper investment, this approach to data collection offers impressive speed, accuracy, and scalability.
• Paper: Paper-based data collection is a classic approach. Forms are printed out and either administered by frontline workers or filled out directly by beneficiaries. While they are prone to slow collection timelines and data errors, paper-based forms remain the easiest to quickly spin up small-scale programs (especially for one-off surveys).
• Interactive Voice Response (IVR): IVR is a method of data collection administered over the phone, where a series of questions are asked and respondents’ information is collected virtually. While it can be restrictive in terms of user responses, IVR does offer a kind of plug-and-play approach to data collection for target audiences with phone service.

Interviews

Interviews can be based on a common set of questions, like a survey, but they allow for more flexibility in the responses. The organic insights gleaned from interviews can give you answers to questions you didn’t even think to ask.

• Individual: Individual interviews are relatively self-explanatory. An individual with experience in the topic you are curious about answers questions from the data collection team. Individual interviews can be a good way to start other types of data collection programs, especially when you are trying to figure out the right question to ask on something like a survey.
• Focus groups: Focus groups are like group interviews, often including a stimulus for discussion. Focus group leaders will prompt the group with a set of questions or statements and gather the reflections of the group. It’s a good way to quickly compare reactions or information from representative sources and an opportunity to see how group dynamics might affect an individual’s reaction.

Observational

Observational data collection is much more low-touch. The idea is that in the absence of an opportunity to interact with your subject (either due to distance, scale, or other reasons for their inaccessibility), you can observe them to collect certain types of information.

• Firsthand: Firsthand observation allows for the data collector to directly observe and gather notes on an individual or group. This approach is often used when direct observation is available but not direct contact either for reasons of inaccessibility or fears of bias.
• Documents/records: Often used when information is needed from the past, document review allows for secondhand observation of sources when firsthand observation is unavailable.

Some programs require a hybrid approach to data collection

How to choose

As you can see, the method of data collection that is right for your program is entirely dependent upon your program objectives and data requirements. For qualitative data, you might lean more heavily on individual interviews and focus groups. For hard quantitative insights, look in the direction of surveys (mobile, paper, or otherwise). Some programs might call for a combination or hybrid of approaches to collect all the information you need.

As you evaluate your options, always make sure that their strengths map back to the objectives of your program and account for the characteristics of your sources. Once you make your choice, it’s time to put all the pieces together.