I-1 The results indicate that the intervention was effective (i.e., that it improved service delivery in the areas identified by the study)

Operations reserach (OR) studies generally either test one or more interventions or they evaluate changes resulting from interventions already implemented. If all studies found the intervention under study to be effective, then research would be unnecessary. This indicator asks whether the intervention tested successfully improved front-line service delivery (e.g., increase in utilization of services, improved quality of services). Negative results can also be instructive, but they would not influence service delivery except to discontinue an ineffective strategy (see I-2).

I-2 The implementing/collaborating organization(s) “acted on” the results

“Acting on the results” consists of implementing the actual services of the intervention or the activities to support those services (e.g., training courses, development of service delivery guidelines, changes in allocation of personnel, supervision, monitoring) if the intervention was effective, or not implementing or discontinuing these services and activities if the intervention was ineffective.

I-3 (If the intervention was effective and continued after the study) The activities tested under the intervention were still observable 36 months post-implementation

“Activities tested under the intervention” are those specific items mentioned in connection with the previous indicator. Where only some of the original activities are observable, the study should receive only a partial score on this indicator. In the case of an improvement that has lasted fewer than 36 months, this indicator does not apply.

I-4 (If the intervention was effective and continued after the study) The original implementing/ collaborating organization scaled up the intervention in the same country

Most OR studies are conducted in a specific geographical area. “Scaling up” refers to implementing the intervention activities in additional geographical areas. It can but does not necessarily refer to expansion to the national level.

I-5 (If the intervention was effective and continued after the study) Another organization within the same country adopted the intervention

An organization that did not participate in the OR study “adopts” the intervention by implementing its primary components (see I-1).

I-6 Another country replicated the intervention

Some evidence must exist that links the original intervention to the activities carried out in the other country (e.g., program managers from other countries visited the project site and subsequently adopted similar strategies).

I-7 A change in policy can be linked to the OR project

This indicator measures legislation or other official changes that potentially affect service delivery, for example, authorization for the sale of the pill by non-medical personnel. For further discussion of policy changes, see the introduction to this section.

I-8 The implementing/collaborating organization conducted subsequent OR studies

“Subsequent OR studies” refers specifically to research activities that test interventions. OR studies do NOT include actions or program activities, such as training and materials production (described in I-2) or research for other purposes (e.g., the DHS, epidemiological research).

I-9 The implementing/collaborating organization conducted subsequent OR studies without external technical assistance

This indicator is included to reflect whether the organization has sufficient capacity to conduct these types of activities as a result of the previous OR experience and has the opportunity to do so.

I-10 The original donor funded new program activities based on the results of the OR study

New program activities are those activities tested in the intervention that the donor had not already funded.

I-11 Other donors provided new or expanded funding based on results of the OR studies

“Other donors” are those donor agencies that did not contribute financial support to the original OR project but subsequently funded the initiation or expansion of program activities– specifically, service delivery or support activities, including training, construction or renovations of facilities, and purchase of supplies and equipment. The indicator does not include funding of additional research only.

I-12 The donor used the data from the OR study for a specific purpose

Does any evidence exist that the donor used the data for its own purposes? Donors may use results for resource allocation, funding decisions, or miscellaneous strategic planning, among other possibilities.