The system for obtaining adequate quantities of contraceptives and other reproductive health (RH) supplies and for delivering them to service delivery points (SDPs) constitutes a critical element of family planning (FP) and RH operations. Without the products that clients need and without the logistics systems to provide them, no program can expect to meaningfully improve the RH of the people it serves. In short:

No product? No program.

Commodities are among the key inputs to any RH program, and logistics systems are among the key processes that enable program success. The figure below shows the main mechanisms and sub-components of logistics processes, including logistics management, policies, human and organizational capacity, and financial resource mobilization. It also shows how logistics processes and functional outputs relate to the overall RH conceptual framework.

(Click on image to enlarge.)

These processes and outputs result in product availability to clients – the main direct result of  effective logistics systems. Logistics may involve FP products exclusively (in categorical programs), or an expanded range of reproductive or other health products, as is the case with increasing frequency in many countries. When a program mobilizes human, technical, and financial resources – with a minimum of external assistance – so that the program consistently ensures product  availability, access to services, and quality of care consistently in a way that meets clients’ needs, the program achieves contraceptive/commodity security. To the extent that logistics systems improve product availability and contribute to commodity security, they also contribute to increased use of RH services and ultimately to improved health outcomes.

Ensuring product availability requires attention to six rights: the right  goods, in the right  quantities, in the right condition, delivered to the right place, at the right time, for the right cost.  The logistics system is often depicted as a cycle with components of product selection (the right goods),  forecasting and procurement (the right quantities, cost), inventory management and distribution (right place, time, and cost), and provision to customers (right place, time, and cost). Information for decision-making is central to the cycle, and quality assurance and monitoring take place throughout. Meeting the needs of end users is the ultimate goal of RH logistics systems, and attention to all six rights is essential to that effort.

(Click on image to enlarge.)

Monitoring and evaluating logistics system performance can help managers, donors, and other stakeholders better understand this essential program component and identify ways to improve it. When using the indicators in this section, evaluators should consider the challenges present in several features of logistics systems. Some issues apply to logistics systems in general, and some are unique to integrated RH systems in today’s changing health care environment. Considerations relevant to the evaluation of any logistics system include the following.

Methodological Challenges of Evaluating Commodities and Logistics

• As with many other program components, the causal relation between logistics system  improvements and health outcomes is complex and largely indirect.

Many factors besides commodities and logistics contribute to long-term health outcomes. Although proving the magnitude of the contribution made by effective logistics systems is rarely feasible, it is highly plausible that better systems and increased product availability enable increased use and improved health. But it is beyond the scope of most evaluations to confirm this scientifically.

• Logistics indicators (especially stockout frequency and adequate stock levels) are interrelated and should be used together; interpreted separately, they can result in misleading conclusions.

If evaluators apply the stockout indicator alone, for example, it may not reveal whether products are actually available to  clients. If reducing stockouts is a strong programmatic priority, service providers may hoard or may ration products to avoid running out. This practice may indeed minimize stockouts, but the result to the client is still the same – no product. When evaluators apply the stockout indicator with adequate stock levels, however, they minimize counterproductive results. The stock status indicator will reveal whether a product is overstocked or under-stocked at any given time and site, an indirect indication of whether rationing is occurring and a direct measure of whether products are actually available when clients need them.

• Evaluators should interpret logistics indicators in relation to other RH indicators.

Ensuring that supplies arrive at their intended destinations is not the only objective of a logistics system. Product needs change depending on programmatic interventions, and logistics managers need to constantly communicate with program managers to ensure that supplies go where they can be dispensed appropriately. For example, logistics must be coordinated with training. Clearly, contraceptives such as IUDs, injectables, and Norplant should go only to sites where trained providers dispense them, and where at least potential demand for the product exists. As FP programs are integrated with a broader range of RH services, these issues become important for an ever-increasing number of products, including STI and HIV test kits, STI drugs and anti-retrovirals, vaccines, and others. Unless logistics and programmatic activities are well coordinated, programs run the risk of expired products, stockouts, inadequate service provision, improper use of products, and ultimately, worsened health outcomes.

• The measurement of some indicators requires specialized logistics knowledge and on-site evaluation.

Complete logistics assessments usually require site visits by a logistics system expert. In a typical assessment, monitoring and evaluation (M&E) specialists may be responsible for the design of tools, sampling strategy, data entry, and analysis, while the actual application of the tools is carried out by logistics experts who may or may not have M&E experience. For example, though in theory storage conditions can be self-monitored, with storage indicators captured through supervision and management information systems (MIS), in practice, site visits by a logistics expert may be the only way to get accurate information. Moreover, a complete logistics assessment (using all the suggested indicators) requires visits to different kinds of sites at different levels, including host-country organization offices, central and district warehouses, and service delivery points, with different information collected at each. This requirement makes M&E of logistics systems potentially more resource intensive than the M&E of other program components.

If these considerations did not make the evaluation of logistics programs difficult enough, recent changes in service delivery strategies further complicate the process. Until recently (and even in many cases today), FP logistics systems have served categorical (or vertical) programs. In such cases, those logistics systems had to ensure that a small number of contraceptive products reached their intended supply points. M&E could focus on the effectiveness of those systems at achieving this relatively straightforward goal.

Several changes, however, have led donors, cooperating agencies, host-country governments, and logistics managers to rethink the way they manage FP logistics systems, most recently being the Global Health Initiative’s broad vision to improve coordination and integration to increase impact, specifically integrating FP within an RH context. Health sector reform efforts have accelerated in developing countries around the world, leading to integration to enhance efficiency and economy. Health sector reform has also led to decentralization and privatization; both of which have created challenges and opportunities for logistics systems as well as for other components of national RH programs. All these changes have occurred in a period of shifting donor commitment that requires greater coordination among all stakeholders, greater ability of logistics systems to adapt to differing donor procedures, and greater emphasis on demonstrating measurable results from donor inputs.

• With the broadening of the FP mandate to embrace an RH perspective, logistics systems must manage an increased number of products.

The number of products alone is not necessarily the problem – automated logistics systems can easily manage thousands of products. But larger systems depend on computers, and most developing country programs do not yet have the resources or capacity for automation. To put the challenge in perspective, categorical FP logistics systems typically manage fewer than 20 – often fewer than 10 – distinct products, some with multiple brands. When FP is integrated with programs such as HIV/AIDS, maternal and child health, integrated management of childhood illness, and malaria, the number of products expands proportionately – and the complexity of the system expands exponentially. The increased number and variety of products requires M&E systems to capture, analyze, and manage far more data than they ever have before. Integration may also result in a merging of logistics and health MIS a process that can result in the loss of essential logistics data. In practice, the best way to prevent such loss is to maintain and manage logistics MIS separately from health MIS. If separation is impossible and the two systems merge, evaluators and logisticians should set up reporting systems to ensure that logistics information is easily and continually accessible to those who need it.

• Beyond sheer numbers, additional products also pose special challenges in logistics forecasting, procurement, storage, and distribution.

Because the demand for most other products is not as well known as the demand for contraceptives, forecasting needs accurately is difficult. Many products have short shelf lives and thus require more precise procurement planning and inventory management. Because logistics systems integration and decentralization complicate storage and transport, systems that were adequate in a categorical family planning program may fail as the program integrates. Thus, evaluators must measure and interpret logistics indicators in ways that result in appropriate conclusions.

For example, HIV/AIDS condoms, while functionally equivalent to FP condoms, have a number of unique characteristics that require new ways to manage distribution and new evaluation strategies. In addition to the increased quantity needed (which affects storage and transport decisions), HIV/AIDS condoms are typically dispensed through different outlets from traditional FP sites. Sites for HIV condoms potentially include everything from bars and brothels to markets, schools, work places, truck stops, barber shops/ beauty salons, and many more. From an M&E perspective, such sites may be far less prepared than FP sites are to record and report data necessary to measure indicators.

• Many additional products now being managed are in chronic short supply, with distribution systems based on budgetary constraints or rationing policies rather than on need.

FP products have typically enjoyed strong support from donors, so that supplies are adequate to meet expected demand. FP logistics systems, therefore, operate on the assumption that  contraceptives are “full supply” products, meaning that in a well-functioning system they should always be available. This assumption allows managers to set maximum and minimum desired inventory levels, and to try to maintain the amount of each product within that range. When a full-supply product such as a contraceptive method is within its planned “max-min” levels, it is said to be “stocked according to plan,” and its stock levels are said to be satisfactory.

These measurements do not apply, however, to many essential drugs and other products purchased by national health ministries or provided by donors with insufficient budgets. Such products are purchased too infrequently, or in insufficient quantities, to prevent stockouts between procurements. Max-min inventory control systems by definition can not apply to non-full supply products, and both logisticians and evaluators must treat the desired minimum stock levels less rigidly.

Evaluators must address many factors to fully evaluate logistics system performance, and many of these factors require qualitative assessment, and expert working groups in the past have suggested a two-pronged approach using both quantitative and qualitative assessments. In response, the USAID-funded DELIVER project has developed separate tools to carry out each type of assessment: the Logistics Indicator Assessment Tool (LIAT) and the Logistics System Assessment Tool (LSAT). The LIAT gathers a relatively small number of quantitative indicators to measure key output results that demonstrate whether or not the logistics system is performing well. The LSAT, on the other hand, contains quantifiable sections allowing for monitoring of changes over time, but it serves mainly as a qualitative diagnostic instrument that describes the overall system and helps identify underlying reasons for each system’s strengths and weaknesses.

Indicator Selection

Ideally, evaluators should collect all the indicators as a package, providing a comprehensive picture of the characteristics of a logistics system and its performance, but in practice, not all programs can carry out such a complete assessment. In such cases, measuring any of the indicators individually is still worthwhile, with previously mentioned caveats in mind. The choice of indicators to measure will depend on program objectives, available resources, or other factors. When choosing, evaluators should recognize that product availability (stockout frequency = zero) is the most vital logistics result from the client’s perspective, so in that sense, it may be the “most important” indicator. Since stockout data are usually collected through facility surveys, however, evaluators may simultaneously collect data on stock data quality, storage conditions, local forecast accuracy, order fill rates, and stock status.

Despite challenges, the logistics indicators proposed here are similar to those used in the past by categorical FP programs. Logistics systems – whether categorical or partially or fully integrated, large or small, health or nonhealth – all share fundamental characteristics. All depend on policies to support them, quality data to forecast needs, and to order products in the right quantities. Thus, the indicators ultimately contribute to FP/RH commodity security and improvements in the RH of the communities the program serves.