“Janet Anyango’s son Yuventino suffers from cerebral malaria, and Janet has faced a constant struggle to care for him. But with little or no income and six other children, it’s been virtually impossible for her to cope. As Janet recounts her story, further tragedy strikes” See this video on the Guardian Katine website.

This video is gutwrenching…there is no other word for it.

It must prompt some hard (evaluation) questions in the minds of many visitors to the Guardian website. Such as:

  • Why are the necessary drugs not available in the health centre?
  • Why cant AMREF help her directly, with the money the Guardian has raised?

In the field of maternal health it is common to carry out a detailed analysis of the causes behind deaths during childbirth (called maternal mortality audits) and in cases of “near misses“, where the mother almost dies. This makes a lot of sense – if some one has died, when they should not have, then every effort should be made to learn as much as possible from this tragedy, so it will not happen again. Their lives deserve respect.

Janet Anyango’s story deserves the same kind of attention. While here in Uganda on my second visit to Katine I will be discussing Janet’s story with AMREF and others. AMREF staff in Kampala have already pointed out to me that there are a number of points in Janet’s story where good decisions could have made a big difference. And where AMREF’s support to the health services in Katine could increase the chance that such good decisions will be made in the future. For example:

  • the distribution of treated bednets. Did the family receive any from the health services?
  • the availability of early diagnosis. Early treatment is better than late
  • the correct diagnosis of malaraia, versus a fever of another kind
  • the appropriate prescription of drugs
  • the availability of those drugs
  • the correct use of those drugs

All this considered, it is still hard not to feel angry at someone for the empty shelves in the health centre. More hard questions need to be asked about what is wrong with the supply chain. There are likely to be both local and national reasons. I will follow up the local reasons next week, in Katine. Some of the national level reasons have already been explored by others in Uganda, incuding recently by the mass media (see the weblink at the end).

The video has already prompted me to ask how AMREF is responding at the national level to this problem of lack of availability of essential drugs. One of the criticisms that has been made of area-based projects like the KCPP is that they ignore the importance of national policies and practices. That is where changes are needed, it is argued. From my discussions with AMREF staff in Kampala yesterday it is clear that AMREF are engaging with national health policies via a number of channels, including on the issue of malaria drug supplies. I have encouraged AMREF to document that strategy and publicly report the progress being made (by AMREF in association with other important actors in Uganda). That work needs to be seen, including all the difficulties involved. Because this is a public issue that needs continuing public visibility.

On that theme, I hope to include some links here on related actions already being taken by others in Uganda. Here is one

In October 2007 Paul Bradshaw wrote an enthusiastic post (on his Online Journalism Blog) about the Guardian’s involvement in what could be seen as a crowdsourcing experiment with AMREF, an African NGO working in Katine sub-country in Uganda, and supported by the Guardian. In that post Paul quoted Guardian editor Alan Rusbridger:

“We’ll need money obviously. But, just as importantly we need advice and involvement. Among our readers are water engineers, doctors, solar energy experts, businessmen and women, teachers, nurses, farmers. We absolutely don’t need a stampede of volunteers, but we would like a technical know-how bank of people who are prepared to offer time and advice. We’ll let you know how to get involved as we go.”

Paul then emphasised that this was “In other words, crowdsourcing – but not crowdsourcing as seen so far in newspapers, where the focus is on asking readers to help gather or analyse information for a story: this is crowdsourcing to help address the actual issues identified by the story.”

After I made contact with Paul earlier this week he asked me if there was any evidence of crowdsourcing actually taking place. My initial reaction was reserved.

As described by Alan Rusbridger above, I think that crowdsourcing via the web may be a solution to a problem that is not necessarily recognised. I suspect AMREF might well argue that they have the necessary expertise, and where they don’t have it, they know how to find it. Certainly an early post by AMREF on the Guardian website was cautious about the possibilities. There has however been one important positive example of unsolicited help that AMREF has welcomed. That was the offer of solar panels for the new AMREF sub-office in Katine sub-county, where main supply electricity is intermittent at best. Something closer to crowdsourcing will be evident if and when AMREF seeks out specific types of help via the Guardian Katine website.

One more immediate challenge is the person power needed to “harvest” the ideas that might come via the Guardian website. The project has only recently appointed a communications officer, who will be paying attention to the Guardian website and other equally important more local communication tasks. The solar power offer was mediated via the Guardian, who told AMREF. AMREF did not have to find it after sorting through many less useful offers. There are of course pros and cons to that mediating role.

One relates to what I thought was a conflation of the roles of the Guardian and AMREF, as present in Alan Rusbridger’s statement. In practice they cannot and should not be doing the same thing. This is one of a number of reasons why I have argued that it is important to more clearly define what the Guardian’s specific role is, especially the role of its website. I think more clarity here could lead to more appreciation and use of the website by AMREF.

The other point about Rusbridger’s statement is that it does seem to assume that the main problems are technical when in fact it could be argued that they are really more social and institutional. For example, how do you get decentralised government to work effectively in countries like Uganda, and how do you aid agricultural innovation in a context where private ownership of land is not the norm? I suspect these problems are less amenable to crowdsourcing solutions, especially when the contributors are from other cultures. But I could be wrong.

These comments should not be read as an argument against using crowdsourcing in this context. In my view it is still early days with this experiment of linking a media organisation (as donor/intermediary), an aid organisation and a local community. There are some discussions underway about engaging the local community with the website, via localised internet access. If that happens the results could be very interesting. Local people are likely to have their own views about what information and ideas they want to access from the world that becomes more open to them this way.

regards, rick davies

For understandable reasons, most development projects have objectives that are focused on changing the lives of people they want to assist: usually poor and disadvantaged communities. Yet development projects often involve partnerships between multiple organisations, located at local and national levels in the assisted country, and further afield. Donor organisations are often based in a different country altogether.

Rarely are donors asked to specify objectives about their own role, and to assess their performance in terms of the achievement of those objectives. Yet, many aspects of their activities can be important, affecting how the recipient organisations are able to do their work. For example, the speed and efficiency of aid transfers and the scale and complexity of their information requirements.

The Guardian is an unusual donor in many respects. Unlike many more traditional donors it is not a “hands off” donor, only wanting to receive a project proposal, then periodic progress reports and then a final evaluation report. Perhaps with a brief field visits once a year. Instead, the Guardian has hired a Uganda journalist to be reporting from Soroti district two weeks out of every month. They have hired an external evaluator to make field visits every six months. Their own staff are making frequent visits to Soroti. And in addition, AMREF will be providing six monthly progress reports to the Guardian. All these activities involve costs, both to the Guardian and to AMREF, both direct and indirect (e.g. staff spending time with visitors versus their own program of work).

Given these costs, a useful question that can be asked of all donors (and not just the Guardian) is: Okay, so what did you do with all the information that you obtained via these various channels? Are the costs of these activities justified by some benefits? If so, what are they? Interestingly, I suspect the Guardian may be in a better position than most traditional donor organisations to answer this question. There is the Guardian Katine website and blog, which is updated at least weekly and almost daily, and which is the primary means of fund-raising. The scale and depth of this website stands in dramatic contrast to many donor websites, which might at best have a static description of the projects they fund, which might be occasionally updated. Options for interactivity will normally be between negligible and non-existent. Overall, the level of public transparency (of the aid process) provided by most donor websites is very limited.

While the Guardian website seems to be going well, evaluating its performance at present is still quite a challenge, at least to me. One reason is that as far as I know, no one has got around to explicitly documenting the objectives for the website, which would then enable some form or monitoring and evaluation of its performance. Websites are simply part of what the Guardian does, it would seem.

Nevertheless, it is clear that the Guardian staff do have some conceptions of what good performance looks like for this website, and others. These relate to numbers of visitors, the numbers of comments posted, their quality, the amount of money raised via the website, etc. And other parties like the One World Trust, also have their views on the value of the website, having awarded it the New Media Award for 2008. The jury noted:

“Katine [website] does a brilliant job of bringing ordinary people from a small African village into global conversations. This 2-way communication is the hallmark of an interesting web project. It succeeds in engaging a wide readership, as testified by the remarkable level of public donations, but above all it brings complex and subtle arguments about development and power into a public space where policy makers meet, engage and debate with both specialists and ordinary people. Katine feels like it has an impact on decision makers.

The quality of debate is remarkably high. Informative and challenging discussions, stimulated by the invitation of knowledgeable contributors, are testament to the engagement of development policy makers. The site is visually accessible, it gives the feeling of being able to interact at village level. There is great story-telling with high production values. The Katine project shows a route for other non-profits to follow. It makes real impact and conveys a feeling of real change.”

This quote is interesting primarily for the potentially useful performance criteria it introduces. How well the website is actually doing on some of these criteria is not yet so well documented. Perhaps more important still is the need to come to some agreement with other Katine stakeholders (especially AMREF) about the relative importance of these performance criteria.

Is it worth paying more attention to the monitoring and evaluation of the Guardian’s Katine website? There are at least three reasons for arguing yes. Firstly, if intended achievements were more explicit and prioritised, and actual achievements more carefully monitored and documented, it seems likely AMREF might be more accepting of some of the costs it feels it is incurring so far. The discussion could move on from a focus on costs, to a focus on cost-effectiveness. Secondly, analysis of performance could help the Guardian further improve its own performance, through having a clearer idea of what it wants and how well it is doing so far. Thirdly, the whole Guardian experience of the Katine website could be analysed, documented and communicated to other donor organisations who, it could easily be argued, should be learning from this unique experiment so they can become better donors.

Siena Anstis “is a Swedish-Canadian freelance journalist currently working with Women of Uganda Network (WOUGNET) in Kampala, Uganda”. On July 11th she posted information about the Katine project on her blog. In that post she raised the important issue of whether investing in one village amongst many was justified. My comment on her blog clarified some points on this issue. Since then Siena has edited her blog to take note of the points I raised.

It may be worth repeating and expanding on these points, because the same issue has arisen in comments on the Guardian Katine website.

1. The Katine project is focused on Katine sub-country, which contains approximately 50 villages. The project is not focused on one village. This misunderstanding may have been encouraged by the Guardian’s unfortunate choice in late 2007 to name their website “Katine: It starts with a village

2. Prior to AMREF’s involvement in Katine there were already inequalities between the sub-counties of Soroti district, as there are throughout the rest of Uganda. AMREF’s intervention has not introduced inequality where there was none before. Katine was one of three sub-counties in the district that were badly affected by the LRA insurgency earlier this decade, and one which AMREF believes had received less help than the others since then.

3. Given the scale of the investment now being made in the AMREF Katine project it could be argued that although old inequalities between sub-counties will be reduced they will be recreated in a new form, as Katine becomes much better off than others. This is probably “a problem we would like to have”. It would imply that the intervention has been conspicuously effective, within the sub-county.

4. In reality other things are likely to happen which will complicate the situation. Some benefits will seep in to other parts of the Soroti district. Staff working on the project will spend their salaries in Soroti and Kampala. Materials to improve water sources, schools and clinics will be purchased in elsewhere in the district, in Kampala and possibly outside Uganda. Improvements to the health services within Katine sub-county, especially Tiriri clinic, will be made use of by people living in adjacent sub-counties (e.g. Tubur and Otuboi). And District officials may make adjustments to budget allocations across the district in the light of what they know about the amount of resources going into Katine. Stephen Ochola, the Soroti district LC5 chairperson, was quoted in May as saying ““What the Katine project has done is to relieve our budget. The money we would have spent can now be used in other areas of the district” Finally, although there may be relatively big investments in Katine over the next three years this fact alone is no guarantee that the impacts will be sustainable. In the worst case, the functioning of the health centres, schools, water sources and local community groups could go backwards in the years that follow.

5. Given these complexities, how can the wider impact of AMREF’s investments be assessed?. One relatively simple method, which may be useful, is a ranking exercise. It may be possible to ask key people at the district level (appointed and elected officials) to rank the 17 sub-counties of Soroti, in terms of their relative standard of living, and to do this at yearly intervals during and after the completion of AMREF’s work in Katine. If useful, this ranking exercise could be done separately, looking at differences in health, education, water, sanitation, and livelihoods. Ranking methods have been widely used as a means of assessing people’s perceptions in development projects. As well as identifying where Katine sub-county fits in any ranking, it would be equally important to identify and assess the evidence the respondents are using to support their judgments.

As well as shedding light on wider changes at the Soroti district level, and placing Katine sub-county in context, this sort of analysis could also be relevant to AMREF’s governance efforts. The results could highlight the types of information officials are using and not using, and how they weigh up the importance of different developments.


See other related posts by Siena

Appropriate goals?

3 July, 2008

This July the Guardian Katine website is focusing on health issues

The lead page on health issues includes a list of AMREF’s health goals. There are also links to AMREFs goals in other sectors: Water, Education, Livelihoods and Governance.

The health goals are:

• Train vaccinators
• Distribute malaria nets
• Train birth attendants
• Improve labs
• Diagnosis training
• Improve drug supply
• Train and supply volunteer village health teams

These are very activity focused objectives, to do with what AMREF , government staff and volunteers will be doing.

For a development program there should be much more focus on the intended outcomes, concerning changes in Katine peoples lives that are expected to happen as a result of these activities, within the three year period of the Katine project. Richard Kavuma’s recent posting on the Guardian website, about Katine people’s expectations about what needs to change in their lives, shows how powerful these sorts of people focused statements can be.

And will any level of improvement in people’s lives be sufficient? Normally it would be expected that some realistic targets would be set, ideally through the involvement of local stakeholders

For example:

By doing these activities…
...achieve these outcomes (examples only)
Train vaccinators Increase the percentage of children under 5 years who have been immunised, from x to y by the end of 2010
Distribute malaria nets Increase the percentage of children sleeping under bed nets, from x to y % by the end of 2010
Train birth attendants Increase the percentage of babies delivered by trained birth attendants, from x to y % by the end of 2010
Improve labs Increase the range of diagnostic tests that visitors to health centers are able to receive on the day of their visit, from x to y % by the end of 2010
Diagnosis training Reduce the number of “false negative” malaria test results of visitors to health centres, from x to y % by the end of 2010
Improve drug supply All listed “essential drugs” are available to health centre visitors at least 26 days in every month, by the end of 2010
Train and supply volunteer village health teams 90% of all village health teams are providing services to households in their villages at least x days per month, by the end of 2010

If these activity oriented health goals have been poorly summarised by the Guardian, then AMREF should be asking for appropriate changes to be made. If they are a direct copy and paste from AMREF documents then AMREF needs to review and revise them, and then notify the Guardian to make the necessary changes.

In my July 2008 visit to Katine I will be looking at what goals and targets have been set, and by whom (see my ToRs). If these are not clearly defined then it will be very difficult to assess the achievements of the Katine project, other than to say there have been a lot of activities. The people of Katine, and individual donors in the UK are likely to be expecting more than this.