Sustainable Health Programming in Nigeria: Insights from Sentinel Indicator Monitoring

Information sharing in Ebonyi. — *HPN Multi-Activity Program Information sharing network in Ebonyi*.

By Data for Impact

September 12, 2024

As part of the USAID/Nigeria Health, Population, and Nutrition (HPN) Multi-Activity Evaluation in 2021-2022, D4I conducted an assessment to explore how four HPN activities were contributing to the sustainability of health systems and health outcomes in Nigeria. The assessment included surveys, key informant interviews, and implementing partner self-reporting. In 2022-2023, D4I used an updated approach to identify a set of 15 sentinel indicators to measure progress in commitment and engagement, along with capacity outcomes across five domains expected to support sustainability.

USAID Nigeria HPN Multi-Activity Evaluation

The four HPN activities in the evaluation included Breakthrough ACTION-Nigeria (BA-N), Global Health Supply Chain Program Procurement and Supply Management (GHSC-PSM), Integrated Health Program (IHP), and the President’s Malaria Initiative for States (PMI-S).

Use of sentinel indicators is an approach championed by USAID’s Complexity-Aware M&E Team, which comprise high-level system “proxies that can signal the need for further investigation if trends deviate from expectations.” It is difficult to directly measure sustainability in real time and the sentinel indicators approach was useful in this context. Sentinel indicators reflect factors like state government expenditures on health services, timely payments to health facility staff, and community organizations’ importance as resource and information brokers in program networks. The sentinel indicators were developed through a consultative process with local actors and selected based on four main criteria:

Feasibility to populate with available data and resources
Relevance for comparing integrated and disease-focused programming approaches
Availability of data for all three case study states
Manageability of the number of indicators representing multiple domains of commitment and engagement, plus capacity

D4I’s HPN Multi-Activity Evaluation used four complexity-aware approaches recommended by USAID:
- Sentinel indicators
- Process monitoring
- Stakeholder feedback
- Most significant change

D4I used these indicators to measure progress on sustainability in three states—Ebonyi, Kebbi, and Zamfara—where HPN activities implemented an integrated approach, disease-focused approach, or a combination of the two. Results based on 2021 data did not suggest systematic differences in sentinel indicator results between states. However, it was likely that a variety of contextual factors contributed to the sustainability of the system strengthening efforts, such as the political economy, state priorities, and other programs operating in each state. Also, this was a cross-sectional exercise and longitudinal data could help support stronger conclusions about the relative effectiveness of different approaches.

Relative Betweeness of Community Organizations in Information Sharing

Relative Betweenness Centrality (RBC) was used to measure how important community organizations are to information sharing in state program networks, relative to non-community organizations. Higher results indicate greater importance. In Ebonyi, for example, RBC is 63%: the average BC for community organizations is higher than 63% of non-community organizations’ BC values.

Source: ONA

Health Financing Indicator

D4I relied heavily on health facility assessment data collected under the evaluation for these sentinel indicators. Implementing this complexity-aware approach required up-front investment in systems to generate and use quality data for monitoring, but it has strong potential to support adaptive management for long-term, multidimensional outcomes like localization and sustainability.

Sources: Ebonyi: www.ebonyistate.gov.ng, Kebbi: www.kebbistate.gov.ng, Zamfara: PMI-S

Learn more about using sentinel indicators and network analysis to assess program sustainability in Nigeria by watching this webinar from February 28, 2024, or reading a summary brief.

Additional sustainability findings: