The purpose of this page is to document what I expect will be an evolving approach to how to most appropriately monitor Katine.

Some early thoughts were captured in postings on my Rick on the Road blog, between October 2007 and January 2008:

OECD DAC criteria

My initial approach was to look at the suitability of the OECD Development Assistance Committee (DAC) evaluation criteria. These are: Relevance, Efficiency, Effectiveness, Impact, and Sustainability. They are criteria that are widely applied in the evaluation of development programmes. However, there are two other criteria that needed inclusion: Equity and Transparency. The meaning of all of these criteria is explained in some detail in this webpage, created in January 2008:

AMREF’s Theory-of-Change

The DAC evaluation criteria don’t make any reference to what is called the project’s “theory-of-change” Yet many evaluators would argue that making a projects “theory of change” explicit and then trying to establish if it worked as intended is an important part of good evaluation practice practice. In the January 2008 Visit to Katine I tried develop a simple model of how I thought the project was supposed to work, and I checked that with project staff. This has two parts:

  • A description of the expected pathway whereby AMREF would be able to improve the lives of households in Katine. This took the form of an adapted Logical Framework, which I am now calling a Social Framework. It shows how a sequence of organisations are linked to each other in a chain, connecting donors in the UK with households in Katine. AMREF are part of that chain, along with local community organisations in Katine.
  • A more complex description of the network of organisations that AMREF has to work with in Katine, in order to benefit households there.

Click on the links above to see images of both models. Click again to see it in better focus.

The DAC evaluation criteria are still relevant. Different evaluation criteria will be important at different points in the pathway.These are mentioned in column two of the pathway table.

Other expectations

In a recent workshop, and in a recent blog, I have suggested that there were three things that could be expected of a development program like Katine:

  1. To have an impact
  2. For the staff to know about the impact(s) achieved. Without this knowledge, the impact is unlikely to be replicable by the same organisation.
  3. For the staff to to communicate to others how to replicate that impact. Without this, the impact of any aid program will be limited, relative to the scale of most development problems. As well as having impact, aid agencies need to produce knowledge about that process, that then becomes an available and usable “public good”. Otherwise the aid agencies concerned will remain “private” enterprises, even though funded with public monies

Advertisements

From late October 2007 until mid-January 2008 I encouraged the staff of AMREF, Guardian, Barclays and Farm Africa to participate in a short online survey about the expected role of the external evaluator. The detailed results, available so far, can be found online here.

Invitations were sent out, and responded to, as shown below. The overall response rate was 20%, with the highest response rates by Farm Africa and Accenture and the lowest response rates by AMREF (Africa) and Barclays. AMREF (Africa) staff were also given the opportunity to complete the survey using hard copies, given during my visit to Uganda in January 2008.

Participation in this online survey is now open to anyone visiting this website. Please take part! Click Here to take the survey

Sent

Responded

AMREF UK

12

3

AMREF Africa

22

3

Guardian

20

4

Farm Africa

2

2

Barclays

20

2

Accenture

4

2

Panos

4

1

Other

3

1

All

87

18