19 May, 2008
The purpose of this page is to document what I expect will be an evolving approach to how to most appropriately monitor Katine.
Some early thoughts were captured in postings on my Rick on the Road blog, between October 2007 and January 2008:
OECD DAC criteria
My initial approach was to look at the suitability of the OECD Development Assistance Committee (DAC) evaluation criteria. These are: Relevance, Efficiency, Effectiveness, Impact, and Sustainability. They are criteria that are widely applied in the evaluation of development programmes. However, there are two other criteria that needed inclusion: Equity and Transparency. The meaning of all of these criteria is explained in some detail in this webpage, created in January 2008:
The DAC evaluation criteria don’t make any reference to what is called the project’s “theory-of-change” Yet many evaluators would argue that making a projects “theory of change” explicit and then trying to establish if it worked as intended is an important part of good evaluation practice practice. In the January 2008 Visit to Katine I tried develop a simple model of how I thought the project was supposed to work, and I checked that with project staff. This has two parts:
- A description of the expected pathway whereby AMREF would be able to improve the lives of households in Katine. This took the form of an adapted Logical Framework, which I am now calling a Social Framework. It shows how a sequence of organisations are linked to each other in a chain, connecting donors in the UK with households in Katine. AMREF are part of that chain, along with local community organisations in Katine.
- A more complex description of the network of organisations that AMREF has to work with in Katine, in order to benefit households there.
Click on the links above to see images of both models. Click again to see it in better focus.
The DAC evaluation criteria are still relevant. Different evaluation criteria will be important at different points in the pathway.These are mentioned in column two of the pathway table.
In a recent workshop, and in a recent blog, I have suggested that there were three things that could be expected of a development program like Katine:
- To have an impact
- For the staff to know about the impact(s) achieved. Without this knowledge, the impact is unlikely to be replicable by the same organisation.
- For the staff to to communicate to others how to replicate that impact. Without this, the impact of any aid program will be limited, relative to the scale of most development problems. As well as having impact, aid agencies need to produce knowledge about that process, that then becomes an available and usable “public good”. Otherwise the aid agencies concerned will remain “private” enterprises, even though funded with public monies