On Tuesday 22nd September Joseph Malinga wrote an article titled

Building work begins on Katine produce store: Katine farmers dig the foundations of a produce store that should help to improve livelihoods in the sub-county

I posted a Comment as follows:

Negotiated agreements on AMREF-assisted developments

It is good to see the issue of uncertain and conflicting expectations being dealt with in this article about a specific development activity assisted by AMREF: the building of a grain store for use by farmers groups in Katine. The need for written agreements about expectations has been raised by the farmers and AMREF has responded to (or has anticipated) this need. The only question in my mind, and perhaps others, was whether this agreement should have been negotiated and signed, before the commencement of the work on the store. On the other hand, a positive feature of this agreement is the willingness of government to make its contribution, along with that of the farmers groups (though this has its risks as well). And the agreement does not seem to be one sided in its expectations. Such agreements could go further than specifying the inputs each will provide, and management arrangements once completed. Reference could also be made to the objectives of this investment, to help forestall any future misuse of the store, e.g. the private use of the store by one of the farmers group members, or someone else completely. How ever the agreement does develop, it would be good if AMREF could share this example agreement via the Guardian website.

In my August 2009 comments on the future of the KCPP I had suggested that Associated with this clarification of expectations, agreements need to be developed that will spell out not only what AMREF will provide, but also what communities will provide, AND what the government will provide. Multiple agreements may be needed, perhaps component by component. One generic agreement will probably not work, because responsibilities will become too generalised and fuzzy. My proposal did not go far enough, agreements about individual developments, such as the grain store, are better still. They are smaller in ambit, and more manageable.

This is the headline of a recent post on the Guardian Katine website, by Joseph Malinga, a Guardian journalist based in Katine.

I posted the following reply…

Could Joseph Malinga do a  follow-up to this story explaining what the government (local and/or central) is doing to support community initiatives like this one? Joseph’s story describes what the community is contributing and what AMREF is providing but there does not seem to be any matching contribution from the government. The government appointed head teacher was there already. The only government contribution so far seem to be doubt, about whether the quality of the school building will be adequate. Yet government is supposed to be one of the partners in the Katine Partnership Project, and advocacy is reported to be an important part of AMREF’s development strategy. What contributions or commitments have AMREF secured from government, in return for the work AMREF is doing with these two community schools? For example, the provision of more government paid teachers. At present the community is paying the teachers salaries.

This is an issue I have raised in my review of the Mid-Term Review, at https://evaluatingkatine.files.wordpress.com/2009/09/a-process-review-of-the-katine-mtr.pdf

In the recent Mid-Term Review (MTR) of the KCCP some local officials questioned the amounts being spent by AMREF on management, and the amounts remaining for what they thought was most important, the construction of infrastructure, like clinics and schools in the Katine. They wanted more for infrastructure and less on management.

In her report, the MTR consultant said “ Just how much money is available is  a figure that can be misrepresented but the entire sum when broken into management, transport and staff costs leaves a reasonable – but not excessive – amount for activities and work on the ground.  The figure is at least 70 per cent of the total budget and this is regarded as an acceptable amount in development projects worldwide.”

In the minutes of a subsequent meeting between AMREF and its donors (Guardian and Barclays) it was reported that the Guardian “asked for costs to be streamlined however [AMREF noted that] management support costs need to be factored in since this came up as an important resource to consider in the review. The MTR recommended a 70%/30% split as common with other projects”

I have real disquite about this position for several reasons, which I will explain. The first reason is it is actually not so easy to calculate this percentage in a standardised way that is applicable across all organisations and understood by all donors. The 70/30 split is probably a common view because everyone in the aid agency world has formed the view that this is what is acceptable to everyone. In other words, it is a herd judgement. I doubt that it is a percentage that has been found through any systematic assessment of aid agency costs.

My second reason is that this crude measure of overhead costs is based on a false assumption of how aid agencies work, a view which is captured by the simplistic but appealing notion that what matters is “whether the money gets there”. In practice in most aid programs very little money or goods actually reach the hands of poor households, because that is the way projects are designed. In Katine more than 50% of the activities in the workplan are training activities, directed at different members of the community and local government. The money spent here goes on staff salaries, and allowances for other trainers, which are spent mainly in the towns. Building costs are another important part of the project. Until recently, most of the money spent on infrastructure was spent on contractors hired from Kampala. Only in the livelihoods component is there much in the way of actual transfer of project resources directly to poor households, in the form of seed supplies. But in parallel to these activities is the UWESO-assisted savings and loan project, where instead of giving people things, the aim is to help people save and make best us of the money they already have. Only in humanitarian emergencies is there any deliberate and substantial real transfer of assets to poor households. [PS: But a small number of aid agencies have been experimenting with cash transfers to poor hoursholds, in non-emergency contexts]

My third reason is that the percentage spent on management costs versus delivery of services is an efficiency measure. Notionally, the smaller the proportion spent on management the better. But this completely ignores  the issue of  how effective the services are that are delivered on the ground. Low overheads will not redeem poor quality services. High overheads may contribute to better services. What matters here is cost effectiveness – what can be achieved for a given unit of cost. And cost here include not just immediate costs like building materials, but also the associated management costs, and (proportionally) the costs of the managers of the managers, etc.

In a recent posting by Dan Pallota on his blog “Free the Non-Profits” he quotes some findings from America which may also apply in the UK:

In 2002 the Better Business Bureau Wise Giving Alliance commissioned a study that asked respondents what information they wanted when considering donating to a charity. Seventy-nine percent wanted to know what percentage of their donation went to charitable programs. Remarkably, only 6% wanted to know if the donation would make a difference. How can that be, you ask? Well, the media, the watchdogs, and the sector itself have done an amazing job of training the public to think that the two things are the same, i.e., that if a charity has low overhead, it must be making a difference. Major studies on the relationship between organizational strength and impact find otherwise.

My advice to AMREF, the Guardian and Barclays is to forget about the 70%/30% ratio, however it is constructed. I agree with Dan Pollota when he says the worst question to ask about charity is, “What percentage of my donation goes to the cause?”, also known as the admin:program ratio, the “efficiency” measure, or the overhead ratio. Whatever you call it, it’s hopelessly flawed, widely abused, utterly useless, a pathetic substitute for meaningful information about a nonprofit’s work, inept at exposing fraud, and a danger to human life

Okay, then how do we best address the concerns expressed by local authorities, during the Mid-Term Review. In my process review of the MTR I made the following suggestion:

The Guardian and Barclays Bank could take a further step, and request that each six monthly narrative progress report on the KCPP include seperate sections on the activities of the AMREF London and Kampala offices and the costs they have incurred in carrying out these activities. If this step is taken, these narrative reports should then be routinely shared with the Steering Committee and Management Committee, as well as being made publicly available via the Guardian website as at present.

These additional sections would detail not only the costs incurred by different sections of AMREF (London, Nairobi, Kampala), but also what they were able to do with that money i.e. some description of their effectiveness.

There is an important larger lesson here. Aid projects like the KCPP involve long and complex supply chains, bringing funds and technical expertise to communities of concern, from distant locations. In the private sector intense effort is often invested into making every part of supply changes work as quickly and efficiently as possible. But in the world of development aid often the focus is almost wholly on the final link in the chain, the organisations delivering assistance at the grassroots level (e.g. the AMREF office in Katine). Very little attention is given to the more expensive[1] parts of the supply chain lined up behind them. The Guardian needs to turn its journalistic attention towards the issue of supply chain costs in international aid delivery. The diagram below shows just how complex these supply chains can be, even in a modest project like the KCPP

supply network2

Thick blue lines = financial transfers. Broken blue lines = information transfers (not including most of those between yellow nodes (intermediaries between donors and recipients))

For further reading see  Dan Pollota’s posts on

The Worst Question to Ask About Charity 9:44 AM Tuesday June 16, 2009

“Efficiency” Measures Miss the Point 3:56 PM Monday June 22, 2009

Beware of Highly “Efficient” Charities 10:44 AM Monday June 29, 2009

Efficiency Measures Discriminate Against Lesser-Known Causes 10:40 AM Wednesday July 8, 2009

Efficiency Measures Short-Change Individual Action 2:20 PM Monday July 13, 2009


[1] In terms of the costs of staff time and transport costs involved


This is the headline for a story on the Guardian katine blog, where Annie Kelly discusses a report that “criticises donors, governments and NGOs for installing boreholes and wells in rural Africa without ensuring their long-term sustainability”

Four comments have been made so far, including one by AMREF and myself.

AMREF pointed out that ” In 2002, AMREF helped communities in the Kathonzweni area to develop 50 shallow wells…By 2007 — five years later, and several years after the projects end — the communities had maintained the existing wells and built 20 new ones.”

This sounds very good, almost too good.

My comment was that “It would be useful if AMREF could provide (on this Guardian website) some information on (a) how they are monitoring the functioning of the wells they have helped to establish in Katine so far, and (b) the data that has been collected. Hopefully information is available on both the functioning of the wells and the functioning of the management committees responsible for each well.

There are problems with wells in Katine, like just about everywhere else. This is to be expected. Equipment does get damaged and worn. What matters more is how long those problems remain unresolved.

There will be a mid-term review of the Katine project in a few months time, which could look at the functioning of the wells then. But that would be a once-off inquiry and the risk is that (as elsewhere) extra efforts might be made to make sure things are working well at that time. Its the monitoring that matters.

regards, rick davies, external evaluator for Katine”

This was the headline on the Guardian katine website, on Tuesday 24th March 2009.  Eliza Anyangwe asked “Are short-term development projects like Katine a good idea? The Katine project in Uganda is scheduled to last for three years. But with such a short timescale can it deliver lasting change, or will it leave a bitter aftertaste?”

There were 12 comments including mine, which ran as follows:

“I have been working for aid organisations for about 28 years, and doing monitoring and evaluation of aid projects for about 19 years. Throughout this time most people I have spoken to who have been associated with aid projects have expressed the view that 3 years is not long enough to make a substantial and sustainable development impact. Especially if you are trying improve the way in which government services work. Many of the comments above seem to share this view.

In the case of the Katine project, AMREF have expressed the view that ” the Katine project really will contribute to lasting change in one of Ugandas poorest and most vulnerable regions”. The Guardian editor also seems to share this view. Are they right? Lets try to test those views.

I have already proposed to the Guardian and AMREF that at the end of the three year period AMREF, and its local partners, should make a number of testable predictions about what things will look like in Katine, in another three years time. That is six years after the project started and three years after it ended. Then the Guardian, or perhaps some more independent organisation (with no prior involvement in Katine), should fund what is called an “ex-post evaluation” team to come in and see how many of the predictions were successful, and how many have failed. And equally importantly, why some were successful and others not so. Then let those results be shared and discussed in Katine and elsewhere via the Guardian blog.

regards, rick davies, external evaluator for the Katine (KCPP) project

PS: The problem with this proposal is the uncertainty about who will be around, six years after the start of the Katine project. Will any of the Guardian journalists now engaged with the project be around and interested to see what has happened, or will they have moved on? Will any of the key people in AMREF still be around and interested, or will they have moved on? The big question seems to be how do aid (and other) organisations develop and maintain a longer term view on what they are doing at any one time.

This AMREF document is available on the Guardian newspaper Katine website, in an article titled “Stakeholders question timescale of Katine project

I posted the following comments:

“I was pleased to see the article about the recent stakeholder meeting held in Katine (“Stakeholders question timescale of Katine project”) and to be able to access online a record of the meeting (“Katine stakeholder meeting and Preliminary project steering committee meeting April 3rd 2008″)

I would hope that further meetings like this are also shared on the Katine website. In the process, I hope that some more detail is provided on the participants who are not government representatives. It seemed that in this meeting almost all participants were government officials or representatives. If that was meant to be so, it would be useful to make that clear. Similarly, records of future meetings should be very clear on who they were meant to include.

My main reason for emphasising this point is that one of the evaluation criteria I will be using (as the external monitor/evaluator) is equity. This means fairness of process as well as fairness of result. There is a second dimension here as well, that of transparency. If a record is kept of the participants in such meetings it will be possible for myself, the Guardian journalist (Richard Kavuma) and anyone else, to make follow up contact with the meeting participants later on, both to hear their views of the meeting, and of what has happened thereafter.”

A response to

Comment on Amref’s first six months, May 7th 2008

I was pleased to see the article about the recent stakeholder meeting held in Katine (“Stakeholders question timescale of Katine project“) and to be able to access online a record of the meeting (“Katine stakeholder meeting and Preliminary project steering committee meeting April 3rd 2008“)
.
I would hope that further meetings like this are also shared on the Katine website. In the process, I hope that some more detail is provided on the participants who are not government representatives. It seemed that in this meeting almost all participants were government officials or representatives. If that was meant to be so, it would be useful to make that clear. Similarly, records of future meetings should be very clear on who they were meant to include.
.
My main reason for emphasising this point is that one of the evaluation criteria I will be using (as the external monitor/evaluator) is equity. This means fairness of process as well as fairness of result. There is a second dimension here as well, that of transparency. If a record is kept of the participants in such meetings it will be possible for myself, the Guardian journalist (Richard Kavuma) and anyone else, to make follow up contact with the meeting participants later on, both to hear their views of the meeting, and of what has happened thereafter.