Program evaluation is more methodologically advanced for family planning (FP) than for any other area of reproductive health, thanks to 40+ years of dedicated effort and strong funding support for this work. Several factors spurred the development of evaluation methodologies. The demographic concern that underscored the early FP programs translated to monitoring of results in quantitative terms: numbers of new acceptors and continuing users. The problem of the “unequal weight” of different methods (e.g., one condom acceptor versus one vasectomy acceptor) gave rise to the index of couple-years of protection (CYP).

Critics and skeptics of FP unwittingly strengthened evaluation efforts. The hot debate originating during the 1970s on the relative contribution of FP programs versus that of socio-economic development impelled researchers to develop methods of demonstrating the independent effect of FP. The FP program effort scores (Lapham and Mauldin, 1984) played a useful role in this research. Although this public debate has subsided, the challenge remains to demonstrate causality (i.e., the program interventions have impact) in FP program evaluation.

The World Fertility Survey (conducted from 1972 to 1984) focused primarily on the determinants of fertility, with relatively little attention to FP and related programmatic issues. The Contraceptive Prevalence Survey, first piloted in 1975 in El Salvador, was designed to produce more programmatically useful results with shorter turn-around time. This survey tool later evolved into the Demographic and Health Survey (DHS) and the Reproductive Health Survey (RHS), both national-level representative surveys of women (and increasingly of men) of reproductive age in developing countries (Robey et al., 1992). The DHS and RHS have become the most widely utilized sources of data for FP program evaluation, because of the quality of the information, standardization of items in the core questionnaire, availability of repeat measurement over time, and possibility of cross-national comparisons. Both surveys extend beyond FP to cover related issues of maternal health/safe pregnancy and child health.

A number of factors facilitate the job of evaluating FP programs. First, the “intermediate outcome” — the desired behavioral change at the population level — is a single measurable behavior: use of a contraceptive method (aggregated to a measure of contraceptive prevalence). Second, despite the sensitive nature of FP in many countries (especially in the early years of programs), women are willing and able to report contraceptive use with a high degree of accuracy, assuming the interviewer creates good rapport and the question is clearly worded. Third, the intermediate outcome of contraceptive prevalence is strongly (inversely) correlated with a key long-term outcome: fertility (except in countries with high levels of abortion). In short, FP evaluators are blessed with a single, measurable, and valid outcome variable. Other areas of reproductive health (with the possible exception of breastfeeding) present a greater methodological challenge. Nonetheless, FP program evaluation does have a few problems of its own.

Methodological Challenges of Evaluating FP Programs

• Contraceptive methods vary in terms of their use-effectiveness, duration of action, and likelihood of continuation.

Although all modern methods can protect against pregnancy, some do so better than others. Thus, the program with a higher percentage of users of long-term methods will generally be more effective in pregnancy prevention than those in which users opt for less effective methods will be. The measure — CYP — was designed to address this issue, but it has certain problems of its own (discussed below).

• Large-scale survey data yield the most reliable estimates of contraceptive use, but they have limitations.

Valuable as DHS/RHS data are for tracking national trends, they have three major limitations. First, such surveys are conducted only once every three to five years (if that often). Second, they do not yield precise results for geographical sub-areasin most countries, which is the level at which program managers generally need their information. Third, these large-scale surveys are very expensive to conduct and analyze, a fact that has caused some countries to question the feasibility of continuing to fund them, especially if donor funding is unavailable. Given these limitations, program statistics, such as number of acceptors and CYP, are widely used to monitor FP programs on a routine basis.

• Demonstrating the impact of FP programs on contraceptive use requires more than the simple tracking of contraceptive prevalence over time in a given country.

Many working in FP would like to think that “if contraceptive prevalence increases, the program must be successful.” However, factors other than the program may have contributed to these increases. Controlled field experiments to demonstrate what would have happened in the absence of the FP program are not feasible in evaluating ongoing, national level programs. The single largest methodological challenge for evaluating FP programs in the past decade has been the issue of establishing attribution. (For a full description of the issue and proposed methods of addressing it, see Evaluating Family Planning Programs with Adaptations for Reproductive Health (Bertrand, Magnani and Rutenberg, 1996), Chapter IV). Evaluation methodology is far more advanced for FP than for other areas of reproductive health, in part because a single, valid outcome indicator measurable through DHS-type studies is available. Yet, definitively establishing cause-and-effect is still relatively rare in the evaluation of FP programs.

• The long-term outcome variable for FP programs is no longer clear-cut.

Prior to the 1994 ICPD, one of the primary goals of many FP programs — especially in Asia — was to reduce fertility, an indicator that is reliable and relatively easy to measure. (All indicators pertaining specifically to fertility are included in the Fertility section of this database.)  Whereas many governments worldwide continue to track fertility as a desired outcome of FP programs, a number of programs are repositioning FP within the larger context of reproductive health as a reproductive right or health intervention. Although the field has moved away from a single-minded focus on fertility, an alternative, standardized indicator that reflects the health and reproductive rights aspects has yet to surface.

Almost all of the indicators in this section were taken directly from the Handbook of Indicators for Family Planning Program Evaluation (Bertrand, Magnani, and Knowles, 1994), suggesting little change in the basic indicators for evaluating outputs and outcome in FP programs since that time. However, we have reduced the total number of FP indicators, retaining those which have proven most useful for program evaluation in field settings.

FP Integration

The evidence is building that integrating FP into existing reproductive health programs is a cost-effective, client-centered way to increase client access to information and services.  Promoting integration is one of the Global Health Initiatives goals and more projects are exploring opportunities to design integration programs to reach high-priority, underserved groups.

FP integration is reflected in this database as stand-alone technical areas (i.e. Population-Health-Environment, FP/HIV, and FP/Maternal and Child Health) as well as being included as a vital indicator within other reproductive health technical areas (i.e. Postabortion Care, Reproductive Health in Emergency Settings, and Obstetric Fistula).