In the high-stakes environment of humanitarian aid, good intentions and practices are not enough; they must be backed by accurate data. The KAP survey (Knowledge, Attitudes, and Practices) is a cornerstone of this evidence-based approach, allowing aid workers to understand not just what a population needs, but how they think and behave. However, as highlighted in the ACF learning document Conducting KAP Surveys: A Learning Document Based on KAP Failures, even minor methodological errors can render an entire study useless. By analyzing past mistakes; specifically from a difficult experience in South Sudan; we can uncover the best practices for deploying a successful KAPsurvey.
The Foundation: Rigorous Sampling Methodology
The most common point of failure identified in the reference document is the sampling design. In an attempt to be innovative, field teams sometimes choose complex or "sexy" new sampling methods without fully understanding the statistical requirements. The document advises a return to basics: for most large, dispersed populations in humanitarian settings, the standard two-stage cluster sampling method is the most reliable approach.
A typical KAP survey aims for 10% precision. To achieve this, teams should generally target a sample of roughly 210 households (often broken down into 30 clusters of 7 households each). The "failure" to avoid here is improvising the sample size or neglecting the limitations of the chosen method. If the data collection strategy isn't statistically sound from day one, no amount of analysis can save the results later.
Designing the Right Survey Tool
The design of the survey tool, the questionnaire itself; is where many projects stumble. The ACF document highlights that a questionnaire must be "fool-proof." A major failure observed in the South Sudan case study was a lack of adequate field testing. Questions that seem clear in an office in Geneva or New York may be completely misunderstood in a remote village.
To avoid this, the questionnaire must undergo rigorous translation and back-translation (translating it into the local dialect and then back to English by a separate translator to check for accuracy). Furthermore, the document stresses that the tool must be piloted in the field. If the survey tool is too long, repetitive, or culturally insensitive, respondents will provide low-quality answers due to fatigue or confusion.
The Human Element: Training and Quality Control
Even with a perfect sample and a flawless questionnaire, the quality of data collection rests on the shoulders of the enumerators. A key lesson from the document is that team training must be thorough. In the rush of an emergency, there is a temptation to cut training days short, but this is a critical error.
Enumerators must be trained not only on the questions but on neutrality. A common failure is enumerators "teaching" the correct answer rather than recording the participant's actual knowledge. Supervisors must be present and active during the initial days of the survey to catch these errors early. Quality control measures, such as daily debriefs, are essential to ensure that the data pouring in is valid.
Transparency: Admitting When Things Go Wrong
Perhaps the most vital lesson from the ACF document is the importance of transparency. "Most importantly, when you discover an error in your surveys (and you will), discuss it and find a solution to address it quickly and openly."
In the competitive world of aid funding, there is pressure to present perfect results. However, hiding a sampling bias or a data entry error compromises the integrity of the entire humanitarian sector. A robust KAPsurvey is one that acknowledges its limitations. If a specific cluster was inaccessible due to security, or if a question was consistently misunderstood, this must be reported. Honest reporting ensures that the final analysis reflects reality, allowing for interventions that actually make a difference.
