Module 4 | Implementation Monitoring

Click here to print this page

Overview

Implementation monitoring answers the question:

How do I know the activities are being implemented according to our design?

Module 4 covers a variety of implementation monitoring issues, such as,

How to design monitoring systems The types of data that should be collected How monitoring will contribute to strengthening the efficacy and outcomes of your projects and programs Tools are provided that demonstrate methods for monitoring HIV/AIDS programs and determining indicators. Throughout the module, there are examples of lessons learned by program staff from around the world.

A Definition of Implementation Monitoring

Implementation is the process of putting a plan into action. This is the phase at which words on paper become an actual prevention or care and support program. Implementation monitoring is the process of keeping track of how a project or program is being implemented, and how closely the project or program matches the plan. [I-LINK to Glossary]

A project and program plan generally includes a description of the specific resources that will go into the project, such as staff hours, money, materials, or time. These inputs are expected to contribute to certain outputs, such as number of patients seen, HIV/AIDS prevention messages produced, condoms sold, or counselors trained. Implementation monitoring involves identifying and tracking these inputs and outputs to see whether they are proceeding according to plan. When it is found that the program activities are not proceeding according to plan, program managers and staff can analyze the situation, and decide what needs to be changed.

Monitoring is a continual, routine effort requiring data gathering, analysis, and reporting on results at periodic intervals. Instituting monitoring at the beginning of a program’s life will ensure that the goals and objectives defined during program design are clear, measurable, and relevant, and that the actual program is the same as the one that was planned. Ideally, monitoring systems should be planned in conjunction with the overall project or program design, but systems can be upgraded or refined to better capture the data necessary for decision-making and reporting while the program is ongoing.

Implementation monitoring is also called performance monitoring, program monitoring, output monitoring, or process evaluation. To do implementation monitoring, one must:

• Record what the project has attempted and accomplished (identifying inputs and outputs)
• Compare what “is” (facts) to what should be (according to the plan)
• Make adjustments if the difference between the results and planned objectives is too great

Effective implementation monitoring answers the following questions:

• Are we on the way to our planned objective?
• To what extent are planned activities being implemented (actually realized)?
• Are project activities being carried out correctly, on time, and within budget?
• How well are services being provided?
• What services are we providing, to whom, when, how often, for how long, and in what context?
• Are the objectives and targets reasonable?

Who Should Use this Module?

Implementation monitoring is essential to programs and projects, at the community, district, national, and international levels. Yet, monitoring does not depend on highly trained outside experts. In fact, effective monitoring relies on the activities of project and program personnel at all levels. Much of the information that describes program progress is collected as a regular part of implementing program activities. This information might be tallied and collected weekly or monthly by supervisors and then passed on to managers. Managers may then analyze the data and use the information for guiding the project or creating annual progress reports. These progress reports review accomplishments, identify constraints or unexpected opportunities, and suggest necessary modifications.

Because implementation monitoring requires the participation of people at so many different levels who may be scattered around a large geographic area, it is important that participants have a common understanding of the reasons for documenting program activities, and that documentation practices are done consistently. Participatory approaches [I-LINK Crosscutting Issues: Participation] to designing monitoring systems are advantageous because they allow those responsible for carrying out monitoring activities to make sure they are feasible and that adequate resources are available.

Culture and Implementation Monitoring

Indicators should relate to the objectives, structure, and implementation of particular projects/programs, and therefore, are usually best identified locally. Implementation monitoring systems, like the programs and projects they monitor, should be culturally and socially attuned to their specific setting. Decisions of inputs and outputs should be culturally sensitive and gender-sensitive.

[TOOL] What is a “gender-sensitive” program? This tool provides guidelines for gender-sensitive programming for reproductive health but could easily be adapted for an HIV/AIDS program. [M4 What is a gender-sensitive program]

[TOOL] Gender and Project Implementation Monitoring. This tool provides questions that might be incorporated into a your monitoring plan to make sure the project is addressing gender issues. [M4 Gender and project implementation monitoring]

Using the Gathered Information

Implementation monitoring serves several purposes:

Management tool: Implementation monitoring is crucial for program management. Monitoring inputs and outputs reveals early on whether programs and services are reaching their target audiences, or whether issues related to content, program scope, defined coverage areas, or target audiences should be re-examined. Thus, implementation monitoring systems provide information that managers use to improve or modify interventions and to identify solutions to problems and constraints before they become crises. Monitoring also helps in formulation of new annual plans of action by detailing what has or has not been accomplished by previous ones.

Reporting requirements: Most donors require some form of reporting so that they can assess whether their resources are being used effectively, efficiently, and for agreed upon objectives. When programs or projects are funded by several different donors, this often requires writing different reports, each according to a particular donor’s specifications. Having a good implementation monitoring system in place will provide managers with ready access to the basic data they will need to share in different formats with different donors.

Accountability to beneficiaries: Information from monitoring systems should provide feedback to the local level. This feedback may encourage grassroots support from the beneficiary population.

Advocacy: Implementation monitoring provides credible information about program progress, problems, and potential. Advocates can use this information to mobilize greater personal commitment and financial support for improved HIV/AIDS policies and programs.

Evaluation: Finally, program monitoring sets the stage for comprehensive evaluation and analysis of program impact. A full discussion of conducting evaluations can be found in Module 5, Impact Assessment and Evaluation. [I-Link to Module 5]

Both monitoring and evaluation are management tools. In the case of monitoring, information is routinely and continuously gathered, which enables mangers and staff to assess whether they are meeting targets established in the project plan. Discrepancies between actual and planned implementation are identified and corrective actions can be taken while the project is in progress.

In contrast to the continuous, ongoing monitoring process, evaluation occurs periodically. Evaluations utilize monitoring information as well as additional sources of data. They focus on specific questions related to effectiveness and impact in order to influence future programs or services. Implementation monitoring and evaluation are distinct, but they can be highly complementary if they are appropriately coordinated with each other.

Balancing monitoring and evaluation

Source: A UNICEF Guide for Monitoring and Evaluation. Making a Difference.
        
[LINK] Evaluating Programs for HIV/AIDS Prevention and Care in Developing Countries, A Handbook for Program Managers and Decision Makers. Family Health International; 2001, Chapters 1 and 2. http://www.fhi.org/en/HIVAIDS/Publications/Archive/evalchap/index.html| local pdf file

[LINK] Strengthening Monitoring and Evaluation of National AIDS Programs in the Context of the Expanded Response
http://www.cpc.unc.edu/measure/publications/pdf/ws-02-08.pdf
This workshop report, written in English and French, provides background information on international initiatives, country practices and capacity, and country plans. The goal of the workshop was to help countries develop short-term action plans for expanding the monitoring and evaluation (M&E) capacity of their national AIDS programs. This document also provides appendices with PowerPoint slide presentations of action plans for strengthening M&E programs from attending country representatives. This resource would be helpful to other countries in the planning stages of strengthening their M&E programs on a national level.

[LINK] National AIDS Councils, Monitoring and Evaluation Operations Manual
http://www1.worldbank.org/hiv_aids/docs/M&E%20Manual.pdf UNAIDS and the World Bank produced this manual on how to develop an overall monitoring and evaluation system that relates to the national or regional strategic plan. This manual includes an appendix full of tools for developing a monitoring and evaluation program. National AIDS councils and program managers will find this document particularly useful.

[LINK] A Guide to Monitoring and Evaluating Adolescent Reproductive Health Programs
http://www.pathfind.org/pf/pubs/focus/guidesandtools
/PDF/Part%20I.pdf [part 1] |

http://www.pathfind.org/pf/pubs/focus/guidesandtools
/PDF/Part%20II.pdf [part 2] |
FOCUS on Young Adults produced this guide that includes practical tools and explanations of evaluation design, evaluation framework, sampling, data collection, data analysis, indicator design, and result dissemination. There is an entire section devoted to instruments and questionnaires in which examples of checklists, tally sheets, reporting forms, exit interviews, and focus group discussions are provided. This is particularly useful for program managers who are new to monitoring and evaluation concepts.

[LINK] Monitoring and Evaluation in UNAIDS, M&E Library: Joint United Nations Programme on HIV/AIDS http://www.unaids.org/EN/in+focus/monitoringevaluation/m_e+library.asp
This Web site, produced by UNAIDS, is an electronic library of monitoring and evaluation materials. The topic areas covered include M&E manuals/guidelines and related health issues, outcome evaluations/surveys/pilot studies, process evaluations, UNAIDS M&E frameworks, UNAIDS best practice reports and workshop reports. Within the categories, the documents are arranged by date and each document has a brief description attached. This Web site is a great place to start reviewing M&E materials when you begin to develop your own M&E plan.

<< previous section | Stage 1: Prepare >>