Lessons Learned in Indicator Standardization

March 6, 2023

By Janine Barden-O’Fallon and Bridgit Adamou, Data for Impact (D4I)

Health interventions work best when they are based on evidence. However, it can be challenging to make evidence-based decisions on the benefits of implementing one intervention over another when the results are not comparable. In the global health context, lack of consensus and inconsistent definitions of key indicators can make it impossible to compare results between programs, approaches, countries, and funding organizations. The inability to directly compare outcomes can lead to poor decision making on a large scale, resulting in incorrect assessments and potential investment in less effective programming. Only when programs align their data systems and metrics can they generate information that can be compared and combined to improve programmatic knowledge.

Indicator standardization helps ensure that the same data are tracked across locations and programs. Sometimes standardized indicators exist but are not consistently used. For example, median age at first sex is well-defined and is the recommended indicator to measure age at first sex. However, competing indicators are sometimes used instead. These include “mean age at first sex,” “average age of sexual initiation among youth ages 14-19,” or “age at first intercourse by characteristics of youth.” The slightly different wording, definitions, and calculations of these indicators yield different information and limit comparability.

---

---

Sometimes standardized indicators are modified in ways that make them unusable outside of a program-specific context. When an implementing program selects indicators for its monitoring and evaluation (M&E) system, a delicate balance is required. While it is important for a program to include customized indicators to capture data and outcomes specific to their goals, maintaining the definitional integrity of standardized indicators is critical. Over-customized indicators can be inconsistent across programs, not comparable, and limit the utility of data for M&E.

Which modifications are acceptable depends on the indicator, a solid understanding of what information it is intended to capture, and how it should be collected. Modifications of time boundaries (e.g., the last 6 months versus the last 12 months) may be acceptable, whereas modifications on age boundaries (e.g., 15–19 years versus 18–24 years) may not be. Modifications that affect the data source, such as using information from a routine health information system instead of a survey, are often problematic as they change the composition and representativeness of the population surveyed. When modifying the data source is unavoidable, the change should be detailed in reports that use these data.  

Indicator reference resources are living documents that are updated as research expands our understanding of a population’s nuances and health needs. D4I contributes to harmonizing and standardizing indicators by updating M&E indicator tools, such as the Family Planning/Reproductive Health Indicators Database developed under our former project, MEASURE Evaluation. We collaborate with funders, implementers, and Monitoring, Evaluation, and Learning professionals to make sure indicators in the database are aligned with researchers’ findings. The figure below provides examples of D4I’s work in this area.

Achieving and maintaining standardization is difficult and requires ongoing effort. With these lessons learned, D4I will continue to collaborate alongside international, national, and local organizations, agencies, initiatives, and M&E leaders to update and align our guidance documents on key indicators. The challenge is immense as M&E expertise is diffuse and there is no single source for indicator resources. But with our focus on data quality, standardization, and research findings, D4I continues to be a leader in indicator guidance, supporting the work being done to improve health outcomes around the globe.

---

---