Agricultural research and development organizations are under pressure to have and report more impact. Integrated monitoring, evaluation and learning (MEL) systems can help, however they do not evolve overnight. MEL specialists belonging to the Association of International Research Centers in Agriculture (AIRCA) Community of Practice (COP) on monitoring and evaluation (M&E) have found that their respective MEL systems are following a common trajectory that progresses through four overlapping stages:
- Basic accountability – meeting donor accountability requirements, usually at project level
- Outcome monitoring and impact evaluation – tracking project outcomes, both expected and unexpected, and showing their contributions to programme- and centre-wide targets and goals
- Learning and strategic planning – using learning across projects in adaptive programming and strategic planning
- Evaluation research – staff develop new MEL approaches and contribute to research on uptake and scaling processes
The COP members developed a self-assessment tool based on these stages. The tool allows staff to discuss and rate their centre’s current level of attainment against a number of questions, and then collectively explore where they want their centre to be within five years. Its strength is that it can help staff see MEL as a collective and useful activity with which they want to engage.
The development and use of the tool is described in a recently-published working paper published by the International Centre for Integrated Mountain Development (ICIMOD) in Nepal, see here. It includes a write-up of the experience of the International Centre for Insect Physiology and Ecology (icipe) in using the tool with its professional staff.
The working paper also explores the need for complexity-aware MEL and its potential to support emergent and rapid change. The authors developed the model shown above as part of this reflection. It is based on reflexive practice in which participants become better able to reflect on their own actions in a process of continuous learning.
The outer cycle shows the rural innovation system responding to project intervention through the emergence of patterns of behavior that the project subsequently chooses to amplify and stabilize, if the patterns are beneficial. The patterns may serve to strengthen existing outcome trajectories or may themselves represent emergent outcome trajectories. An outcome trajectory is understood as a self-organizing and vertically integrated network of actors that put social, institutional, and/or technological innovation to use over time. The network is vertically integrated in the sense that users are linked to actors who provide, support, adapt, and research the innovation.
The inner cycle shows how M&E supports the outer one through the use of theory of change (ToC) to support reflexive practice. It shows planning for change being informed by the development and revisiting of a ToC built from evidence and assumptions about how change is happening, or is expected to happen. The M&E system documents system response during implementation. Comparing what is starting to happen with what was expected and planned for, helps staff to adjust implementation to be effective.
The main insights reported in the working paper are:
- Achieving a centralized M&E system takes time and a staged process.
- Key to a centralized M&E system is a small set of centre-level outcome indicators to which projects and programs are expected to contribute.
- Also key is a close working relationship between the centre’s M&E team and senior management.
- Developing and revisiting ToCs can usefully strengthen and inform planning, M&E and learning.
- Complexity-aware M&E systems need to be able to recognize unexpected positive and negative outcomes and adapt their planning and implementation accordingly.
- There is often more scope for learning and flexibility in moving from one project to the next within programmes than within individual projects, particularly if projects have a short lifespan. Hence complexity-aware M&E systems are more useful at programme or centre levels.