Tiriri health centre: Questions still unanswered

Sarah Bosley’s post on the Guardian Katine website on December 2nd has raised some very important questions about why this health service is not delivering the basic services that it should be delivering: providing anti-malarial drugs, drugs for preventing mother-to-child transmission of HIV and other essential services. Instead there are empty shelves in the dispensary, the ambulance is elsewhere and there has not been an assigned doctor present for months. In her view the health centre “was in a worse state than when the Guardian/Amref project began, 12 months earlier.”

She then questions the role of AMREF, and whether there was not something the AMREF staff could have done to address these problems, in addition to their work of installing clean water supplies, supplying mosquito-nets and the training of village health teams and clinic staff.

Visitors to the website have been both sympathetic and critical of these comments. Some emphasised that these problems are not unique to Katine, or Uganda even, and that their systematic causes need to be addressed. Not the provision of stop gap supplies from other local NGOs or AMREF itself. But meanwhile the problem of lack of drugs and a doctor is a very real and pressing one for the people of Katine, regardless of what is happening elsewhere in Africa.

AMREF have responded, and there are now some interim supplies on their way. But these are recognised as stop gap solutions. They have commented:

Given the limited scope of our resources, AMREF must make strategic, sustainable investments in health. Until the national health system in Uganda is robust enough to reach and truly serve rural communities, we must build the system from the bottom up, empowering communities to take care of their own health.

This means focusing on long-term but essential activities such as training communities in disease prevention and early detection, improving health information systems so that a true picture of community health needs is understood by the district health authorities, and for better planning.

The other key aspect of our work is building the health system from the top down. We continue to advocate at the national, district, and local levels for improved drug delivery and improved health care delivery and resources (doctors, nurses, equipment, and so on). Changing government policy is slow, difficult, and necessary work. The alternative is a perpetually weak health care system based on short-term fixes.

To me the debate that Sarah has promoted raises important questions of what should donors, and other stakeholders, expect of AMREFs work in Katine on health issues at the end of three years. What sort of outcomes should be expected? This question is closely tied into a similar question, about what the Katine community should expect from the Tiriri health centre.

Simply listing the inputs that have been provided (e.g. number of people trained, meetings held with government officials etc.) will not be very satisfactory.  AMREF have already been very active, but to outsiders the health centre seems to be going backwards.

What would be much more useful are some realistic performance objectives for the health centre as a whole. These need to be ones that local officials agree with, and which local communities could understand and relate to. They would be largely about the availability and quality of services that should be provided. Once established, there then needs to be some transparent public reporting on how the Tiriri, and other local health centres, are performing on these criteria. If AMREF wants to empower people then providing public information is essential, both about what they can officially expect from their health centres and about what is actually being provided. Ultimately it is the people’s evaluation that matters and which should be heard.

One possible way is to move forward with a local adaptation of the Yellow Star program, already trialled elsewhere in Uganda. Fortunately, one member of the AMREF Katine staff have had some experience with this program. In doing so it would be important to see this not just as an improved health information system, but also as a means of community empowerment, to demand better services.

The evaluator evaluated

Here are…

AMREF REFLECTIONS ON 2^ND EXTERNAL EVALUATION OF KCPP

tabled at the Quarterly Governance meeting, Barclays, 1 Churchill Place, 15^th October 2.30-4.30 pm

along with my responses in red

Strengths

• Mobilised different stakeholders in AMREF to critically reflect on the progress, key priorities over next 12 months – key development questions on what, when and why
• Acknowledged the different pieces of work going on in Katine and the sheer volume of the work
• Participatory and inclusive – got insights from different stakeholders and beneficiary groups
• The process of responding to the report enabled the whole of AMREF to work together and crystallise and document our approach to development with a particular focus on KCPP
• Useful recommendation and learning on how to improve our M&E processes

Areas for improvement

• Very long/heavy report – really difficult to analyse especially when English is not a first language
  • There is a four page summary at the front, and a contents page, and separate annexes
  • The next report will aim to be 20 pages max (excluding Exec Summ and Annexes)
    • But this proposal is subject to discussions re plans for the AMREF Mid Term Revew (MTR)
• Intensive process/time consuming for different stakeholders in KCPP (PIT, country office team, AMREF HQ, AMREF UK, Farm Africa).
  • Following discussions with Guardian in August my aim now is to progressively reduce the frequency of visits and, where possible, to integrate these with AMREF review processes.
• The evaluation visits are too regular to enable staff on ground to learn, take action before participating in another evaluation (So far we have had 2 evaluations in 7 months). Most 3 year projects have 2 – midterm and end of project). KCPP will have 8 over the 3 years (6 + midterm and end of project).  This is not feasible if we want staff to value the evaluation  and deliver on the specific targets in their workplans
  • Following discussions with Guardian in August my aim now is to progressively reduce the frequency of visits and, where possible, to integrate these with AMREF review processes.
• Analysis of KCPP did not fully take into account the wider AMREF context, its policies and procedures)
  • Examples of important missing contextual information would be useful
  • Not sure how this gap could be addressed while also reducing the size of the report
• Analysis of KCPP did not fully take into account the wider development context of Katine, Soroti district and Uganda
  • Examples of important missing contextual information would be useful
  • Not sure how this gap could be addressed while also reducing the size of the report
• The evaluation focussed a lot on the processes. To the staff, who are working under difficult circumstances, highlighting both the interim tangible and intangible outcomes could have been more motivating
  • In the early stages of a project it is the work processes that are most visible and important, and outcomes tend to be less visible. Future visits should focus progressively more on outcomes and impact.
• In some instances the report  did not take into account key sensitivities about staff and impact on the relationships that AMREF has with different stakeholders
  • Details are needed here before I can respond. But on reflection it seemed as though  it was sections of AMREF who were the most sensitive, and sections of government who were quite robust (in wanting their views expressed).

Suggestions for the future

• Make the report shorter and in simple, user-friendly language (maximum 10 pages)
  • 10 pages is too short. 20 pages is more realistic. But final agreement here will depend on the ambit of the next visit, which is under discussion.
• Consider the implications of the evaluation recommendations on the capacity of the PIT and practical realities on the ground – deliver initial targets, address new issues within very tight deadlines
  • Noted: Recommendations should be limited in number
• The report has multiple target audiences.  It would be good to have a summary which is user friendly and targeting our stakeholders in Uganda, especially the district officials (i.e beyond the UK donors, UK public through the website).  Otherwise we stand to be accused of using extractive evaluation processes
  • Noted: There is a need for an Executive Summary that can fulfil this function (in addition to the existing list of Recommendations). But I am not sure if I should produce customised versions for different stakeholders. This might be better done by AMREF.
    
• Place analysis of findings and recommendations within the contextual realities of Katine sub-county, Soroti District and Uganda development context.  Co-evaluation with someone from Africa/Uganda would add a lot of value
  • Need explanation of what “contextual realities” are being referred to, and how they would be covered within a 10 page, or 20 page, report
  • Co-evaluation with “someone from Africa/Uganda” will happen if we integrate my next visit with the AMREF mid-term review
• Extend the time and if possible limit the number of evaluations bearing in mind that we shall also have midterm and end of project
  • Agreed, as noted above
• Next evaluation should focus more on interim outcomes and the foundations we are making for sustainability
  • Sounds appropriate at that point in time in the lifespan of the project. As part of this next review, I would like to see some systematic documentation of what has been done / happened in all the villages of Katine sub-country. A starting point would be a spreadsheet of villages x activities (including non-AMFREF activities).
• Clarify roles and scope of the evaluation – technical and programmatic; operations and management.
  • The Terms of Reference (ToRs) for each visits should be where this is done. Draft ToRs were shared with AMREF, Barclays and the Guardian for comment prior to each of the two visits to Katine so far. The same will be the case with visits in the future.
    

Learning from other projects?

In October I visited and reviewed the progress of UNICEF’s Women and Children’s Health Program in Papua (WCHPP), as part of a wider team including staff from UNICEF, AusAID, GTZ and specialists in maternal and neonatal health. This is the third of a series of such reviews conducted annually since 2006.

How is that relevant to Katine, a very different community in a very different country, assisted by a very different organisation, funded by a very different donor?

Well, it does seem that the two projects do share some common issues, which I will list here in the hope that someone might react to them (including the good staff of AMREF Soroti).

• Both projects involve the assisting agency working closely with government structures. In both project designs there is an important role for community mobilisation and empowerment, especially in relation to health matters. In Papua UNICEF wants to increase public awareness and use of maternal and child health services. It sees increasing demand for services as an essential complement to increasing the supply of such services. I think AMREF has a similar view in Katine. But the progress with this part of the project in Papua has been slow. UNICEF has led, but the government has not been that enthusiastic about this part of the program, compared to others. Questions were raised by the review team as to whether UNICEF should continue working in this area, especially given the fact that their staff were already overburdened with work. It might be better to find or encourage an independent NGO to work on the demand side, especially one that was more independent of the government.

• Both projects are engaged in many activities, so much so that it was not easy at first glance to appreciate the strategy guiding these activities. In the WCHPP there were two overlapping strategies. One could be described as humanitarian assistance. This involved funding support for staff training, repair of health facilities, procurement of supplies, etc. All of this was useful, and appreciated, but it was essentially gap filling, filling in where government funds should have been at work. Government funds were available, but they were being invested elsewhere, in new hospital buildings and in sectors other than health. The other strategy was the introduction of new and improved ways of doing things (computerised health information systems, partnerships between midwives and traditional birth attendants, etc). This is where UNICEF was arguably doing more development work, potentially adding value to the way in which government health systems were already working.
  • With both the humanitarian and development work there were some important limitations, that needed to be addressed. With the humanitarian work, there were no agreed performance targets or milestones that UNICEF would help the government reach, and then phase out its assistance. For example, at what point do you stop supporting the operations of a sub-district health centre, and move your assistance elsewhere? With the development work, the introduction of new or improved methods was not consistently associated with assessment of existing methods, and then a follow up comparison of the value of the new method. Nor was there any systematic packaging and promotion of the results of studies that were carried out. De factor “experiments” were being carried out, but they were not being systematically assessed and results publicised. This is despite the fact that the idea of development projects as “policy experiments” has been around for decades (See Rondinelli, 1993).

• There was a third feature that seemed to be shared by both projects, relating to decentralisation. Both Uganda and Indonesia have decentralised government administration, Indonesia probably the most radically. If ever there was a need for decentralised administration, it would be in Papua, which is culturally and geographically a long way from Jakarta. Yet, UNICEF’s response to providing assistance to its Papua office was remarkably centralised. A KAP (Knowledge Attitude Practice) survey was centrally commissioned for the whole of Indonesia as one entity, and this will be used to measure the impact of health education and awareness raising activities in Papua (and elsewhere), where the issues facing women and children are not carbon copies of those found elsewhere in Indonesia. A M&E framework was introduced, exactly as used by similar UNICEF projects elsewhere in Indonesia. Yet the capacities of local government, and associated health services in Papua bear no comparison with those found in Java. All this raises questions of how much can we expect a highly centralised aid agency to help a highly decentralised government?

• The fourth feature related to the behaviour of UNICEF and GTZ staff as members of the annual review teams I mentioned above. While they have now participated in reviews of each others projects for three years in a  row, it still surprises me how much they are still very wedded to their view that their own approach is the best possible under the circumstances, and how little acknowledgement (and even less follow up action) they have given to potentially positive features of each other’s projects. This experience resonates with what we have seen on the Guardian blog, in the way that AMREF has responded to comments by another NGO that it is possible to build schools significantly cheaper that the way AMREF has done. That is, with a little interest and enthusiasm to learn, and a considerable amount of defensiveness.

In my report on my second visit to Katine in August 2008 I raised a very similar set of issues

• The risks of overly centralised management of the project, within AMREF
• Questions about to what extent AMREF should do community empowerment, and whether it might be better done by another party more independent of government
• The need to set targets for what AMREF was trying to achieve, in consultation with its local partners
• The need to take the idea of generating development models seriously, by building operational research into all new activities that were trying to do things differently

The issues raised here also relate to my argument in a recent posting: How well is the KCPP doing? Compared to what? In that posting I argued that comparisons of AMREF’s performance with other projects could be as important as comparisons of with its own stated intentions and objectives

August 2008 Visit Report by the external evaluator

The August 2008 visit report is accessible here

• Main report
• The annexes

1. Summary of issues and main recommendations

1.      Recommendations about changes almost by definition do not identify the good that is present and which needs to be maintained. With this in mind the following points should be noted:

• The KCPP staff are committed to their work, and appear to work well together as a group. The management style is team oriented, which is essential if there is to be effective integration of the project components.
• There is a breadth of experience and expertise within the team e.g. the Health component Project Officer’s experience with the Yellow Star programme, and the Project Manager’s experience with training in rights- based approaches.
• There are good relationships with the district and sub-county authorities. Although the district authorities have made critical comparisons with NUSAF they are also openly appreciative of the good working relationship that they have with AMREF
• Both the material aid and training provided by the different components are appreciated by the communities concerned within Katine. There have been no substantial criticisms of their quality or relevance.

2.      This section summarises some major recommendations. The main text of the report lists specific recommendations by number, where the issues arise. There are 14 sets of recommendations (28 in all). And a further seven recommendations relating to the work of CARE and UWESO in Annex E. Please read them all. Search for: Recommendations*

3.      There have been significant delays in the project implementation, with the possibility of more than a quarter of the planned activities not being completed by the end of the first year (and possibly a third of the year 1 budget remaining unspent). However what is not necessary is a rush to complete activities by the end of September, simply for the sake of completing as much of the original plan on time as possible. What is important is attention to the cause of these delays.  The results of this examination should be evident in more timely implementation in the October 2008 – March 2009 period. See section 3.

PS1: AMREF have since pointed out that the project workplans outline “a sequence of activities as opposed to specific deadlines” (through the workplans do assign specific activities to specific time periods). In their view delays are to be expected because AMREF is involved in community development, and thus AMREF does not set the pace on its own. However, as noted later in this report, delays have also arisen from inefficiencies of process within AMREF. The question now is how big do the delays need to be before the project donors can legitimately express concern?

PS2: Re reviewing the causes of delays, AMREF’s view is that their existing planning and review processes are adequate

4.      The importance of particular project activities depends on their place within a wider strategic view of how the project will achieve its ends. A project with many activities but little strategy could easily be accused of gap-filling, and little more. This is a serious criticism that can be made of many aid projects, and one that AMREF needs to be able to respond to KCPP. The issue of KCPP’s strategy is discussed in section 8, on the project’s theory-of-change (ToC).  One high level statement of KCPP’s ToC is that the Development of Models + Creation of Demand = Development impact. This report argues (in 3.5 and that the project’s strategy for the encouragement of public demand (e.g. for access to quality health, education and water services) is underdeveloped and needs attention including greater use of communication channels outside of the government administration (e.g. via mass media and elected representatives at different levels).  The approach to the “development of models” also needs to be articulated more explicitly, because the options here are more complex than appear at first glance. In section 8.4 a number of different types of model creation have been identified. What would now be useful is clarification from AMREF on which of these are important in the KCPP. See section 8.

PS: AMREF’s view here is that its corporate (10 year) and Uganda country strategy provide a sufficient strategic guidance to the KCPP

5. Another high level statement of KCPP’s  ToC is that the Community Partnering + Capacity Building + Operations Research and Advocacy = Development Impact. There is ample evidence that Community Partnering and Capacity Building are already taking place. What is not yet present is sufficient investment in the analysis of the efficiency and effectiveness of the work being by the KCPP, which could then be used by staff and other stakeholders to advocate the use of certain development approaches (aka models). Doing so would be consistent with AMREF’s stated support for evidence-based advocacy (versus “assumption based advocacy”).  Progress with operations research and advocacy would be evident in (a) dedicated activities and budget allocations for this work within each of the components’ workplans, as well as for the M&E officer, and (b) documentation of progress made with advocacy efforts. See section 8.

PS: AMREF has commented that “AMREF is in the process of developing an advocacy strategy for the Katine project. The advocacy strategy will provide guidelines on how the project will gather the evidence and utilise it to influence policies and best practices at the local and national levels through the various foras that AMREF participates in such as the health policy action committee.”

6. Two important sources of risk for the project have been identified during this July visit. One is external, and one internal. The external risk is that the KCPP will face significant competition for attention from other development projects, such as NUSAF[1]. NUSAF is more decentralised in its management, and seen to be more cost-effective by Soroti district officials[2].  With the beginning of NUSAF Phase 2 District and sub-county officials may be less open to the adoption of KCPP models, unless the KCPP engages in serious evidence-based advocacy about the value of what it is doing.  There are also other development projects elsewhere in Uganda which will be competing for national and international attention along with the KCPP[3]. AMREF needs to identify how the KCPP approach is different and what difference these differences make (i.e. talking about its theory-of-change in practical terms) (See section 8.3). It needs to be able to show how it is adding value in ways that would not be possible if the same aid funds were to flow through different channels (either government or non-government).

PS: Re comparisons with other projects, AMREF have said “However we would like to desist from direct comparison with NUSAF because no formal comparable review has been done. ” Although AMREF may not wish to have comparisons made, other stakeholders are likely to persist in doing so, throughout the life of the project, and perhaps that reality needs to be acknowledged and responded to. This accommodation may have already begun, in that AMREF have also said that it “acknowledges issues raised by the district and sub-county authority and aims to take their comments on board. As part of the project steering committee agenda, AMREF will aim to do a comparative financial analysis of the project’s interventions vis a vis funds channelled through Katine sub-county and Soroti district.”

7. A significant internal risk to project success is the degree of centralised management within the Uganda country program.  Expenditures over UGX 100,000 (£33.50) .need to be approved by the Kampala office, whereas the Sub-county chief can authorise up to UGX 30 million. Quarterly workplans by all KCPP staff need to be approved by the Kampala office. Both processes take more time than more localised decision making. Strategies for communications and advocacy apparently need to be developed at the country level first. Country office staff represent the KCPP in both the monthly Project Management Committee and the quarterly Project Steering Committee meetings held in Katine and Soroti respectively. All these processes have implications for the efficiency, effectiveness and credibility of AMREF’s work in Katine.  It should be noted that concerns about the degree of centralisation have been made by members of the District Steering Committee, Sub-county Project Management Committee members, KCPP staff and Guardian visitors. There is, in my view, a strong argument in a favour of AMREF HQ undertaking a review of existing practices  Where it is decided that specific processes do need to remain centralised then there should be clear and transparent performance targets for their completion times. See section 3.7.

PS: AMREF’s response is that they use “standard management processes for Katine and all the other 23 projects in Uganda. These provide efficiency through scale as well as accountability that would be expensive and burdensome to replicate (sic) separately for every project. Moreover, these processes have been developed to support implementation across AMREF, based on our 51 years of work in Africa.” They also note that “The PMC and SC were established through participatory discussions with all the stakeholders and are subject to review according to local needs.

8. Within the project there are two areas of strategy which need particular attention. In the health component the investment in upgrading of health centre facilities and training of village health teams has involved a de facto assumption that the necessary drug supplies will be available, especially anti-malarials. Reports from the VHTs and Health Centre staff indicate otherwise, than in fact shortage of anti-malarials is a major problem. This is in the context where the AMREF’s 2007 needs assessment identified malaria as the single main cause of morbidity and mortality. This is clearly one area where a research and advocacy strategy needs to be developed as an integral part of the KCPP approach. See section 3.1.

PS: AMREF has commented that “Drug stock outs are a common problem in Uganda and AMREF has taken this as an advocacy issue.  In another project in Uganda, AMREF is undertaking an operations research into the issue of drug supply chain. The results of this research will be used as evidence for wider advocacy and policy influencing and will benefit KCCP. AMREF also has other malaria specific projects in Uganda that are addressing similar issues.”

9 The governance component also faces some challenges. On the ground in Katine the IEC[4] component has not yet started to address the issues of “the community’s rights to basic services…”, instead pre-existing conventional IEC materials are being re-used.. The piloting of community partnering “from a rights based approach” seems to be on hold. Discussions in Kampala suggested that this office favours a conservative interpretation of how issues of empowerment and rights can be dealt with. In Katine staff seem to have a more open view. AMREF should clarify its position by developing a strategy statement saying how its existing (or revised) governance activities in Katine will help achieve the governance objectives as stated in the project proposal. This strategy statement should be shared and discussed with its donors. See section 3.5.

PS: AMREF have confirmed that “Rights Based Approach (RBA) is prioritised as a core implementation strategy in the AMREF global and country strategic plans. The RBA model is being developed and rolled out across Africa since this is a relatively new area for AMREF.”…”Based on the agreed project design, the community empowerment component is broader than RBA. Progress on governance is based on the capacity of communities to engage with their decision and policy makers. AMREF’s role is to catalyze community action since taking on that role would disempower the community”.

10 Both of the two problems identified above may be reflections of what could be called a strategy gap in the current design of the KCPP. That is the absence of documented thinking about how activities in each component should connect causally to the project objectives of that component. The recent introduction of a new planning format for proposed activities may help fill this gap, if it asks the right questions. If not, they could be a form of micro-management.

11.  The project has its own objectives and indicators, and data on some of these indicators is now being collected, in addition to the work done on the baseline survey. However further steps are needed. The definition of specific levels of achievement on the various project indicators is essential, if the success of the project is to be evaluable. In the longer term, the sustainability of all project aided activities will be dependent of capable monitoring of those activities by the relevant community groups and government structures (e.g. PTAs, SMCs, PDCs, VSLAs, etc). This is more likely to happen if KCPP objectives (especially the associated targets) are closely aligned with those of its local government and community partners. Although some progress is being made with identifying targeted levels of achievement on indicators of shared concern, this work does need to be finalised soon, preferably by the end of this year. See section 4.

PS: AMREF believes the existing comprehensive M & E framework for the KCCP is adequate, but that “it will be updated periodically on the basis based on new project information and in response to community priorities”.

12. Regarding transparency, AMREF has been open in its dealings with local government, though local government would like to see more information on costs of activities. District and sub-county authorities have in turn expressed willingness to share information on their budgets, an offer worth pursuing. Although the importance of transparency seems to be well recognised in the KCPP office more attention does need to be given to improving transparency with the public at large in Katine, both about AMREF activities and the work of its partners. Simple measures could be useful, such as robust public notice boards at schools, health centres, markets and sub-county offices showing meeting minutes, survey results, annotated maps, and AMREF’s own plans. The use of talk-back radio has also been suggested[5]. Internationally, progress with greater transparency via the AMREF websites has been slow, but with a major improvement during my visit, hopefully now to be sustained. See section 7.

13. Transparency will not be painless.  It will almost inevitably bring with it some critical judgements of how well the KCPP is doing, when compared to others. For example, decentralised projects like NUSAF, and activities by smaller NGOs with lower overheads. Ideally AMREF will respond positively to these challenges, and provide in turn a demanding comparator for other development projects. This expectation is in effect the external evaluator’s theory-of-change: about the relationship between transparency, the value of comparisons and increased aid effectiveness.

PS: AMREF has replied that it is “confident that our development model is rigorous enough to enable us to engage in informed development debates with other likeminded organisations.”

Footnotes:

[1] Northern Uganda Social Action Fund. NUSAF Phase 2 is about to start in January 2009

[2] But not without its own problems e.g. corruption. See Sh2.5b NUSAF funds missing New Vision, Thursday, 10th July, 2008 http://www.newvision.co.ug/D/8/12/638475

[3] Including those announced by President Museveni, in his recent visit to Soroti.

[4] Information, Education and Communication

[5] There are three radio stations in the area, and many households have radios.

How well is the KCPP doing? Compared to what?

One conventional approach to the evaluation of aid projects is to identify the project’s objectives, and then see what has been achieved on the ground and how it compares to the stated objectives on paper.

This approach has its own difficulties. For example, the objectives might not be stated very clearly. Or if there are clear objectives, there may not be any useful indicators of progress with these objectives. Or if there are useful indicators, there may not be any targeted level of achievement on those indicators. I have already discussed these types of problems in the KCPP, both directly with the AMREF KCPP staff, and via this blog. (e.g. “Appropriate Goals“)

But this approach may be irrelevant to many people. What may be more important is how the KCPP seems to be progressing, when compared to other development projects that they know of.

Yesterday’s article on the Guardian Katine website (by Richard Kavuma) on President Museveni’s visit to Soroti mentioned a number of different development initiatives that have been promised, and in some cases already happened. Richard’s article mentions Northern Uganda Social Action Fund (NUSAF), the National Agricultural Advisory Services (NAADS), Savings and Credit Cooperative Societies (SACCOs), and now “model farms”

“Presenting highlights of his anti-poverty campaign, Museveni announced a fund of 98 billion Ugandan shillings ($60 million) to support six homesteads in each of Uganda’s 5,000 parishes to set up model farms. The idea is that the remaining villages will learn from the model homesteads how poverty can be overcome and will – hopefully – start their own journeys.” **

The KCPP project is also about developing convincing models of good practice (re health, education, water, sanitation, livelihoods, governance), that government, local communities and individual households will hopefully adopt.

However, the NUSAF (Phase 2), the SACCOs, and the model farms will all be in competition with the work AMREF is doing via the KCPP. In competition for the attention of local elected leaders and administrators, who need to decide how to best use their scare resources (both their own time and the budgets they are responsible for). And in competition for the attention of households as well.

Competition could be useful. But it will depend on who sets the rules, about what kinds of development achievements are really important. That will be an important arena of competition itself. AMREF could well argue that transparency (about how a project is managed) is most important of all, because without open access to information how can people really tell what is going on? Have the billions of shillings really been spent as intended?

AMREF could also argue that sustainability is very important, important enough to sometimes justify its own development costs being higher than others (e.g. for installing a borehole and establishing a working water source committee).

But to do this it will need some hard facts, not just optimistic assumptions. For example about the percentage of KCPP boreholes still functioning after two years, compared to those established by NUSAF.

And it will also need a strategy, a sense of what facts are the most important, amongst the many that could be collected. What are the most important differences between how AMREF is doing things in the KCPP, and how others (government and non-government) are doing them? And what difference is it expected that these differences make to people’s lives?

These questions can be asked at two levels: (a) about AMREF’s work generally, and (b) much more specifically, about individual project activities. Such as providing furniture and books for schools, or training Village Health Teams, or working with farmers groups to improve their incomes. Facts and findings about concrete activities like these are probably going to be of most interest to the people of Soroti district.

If it can successfully compete for people’s attention, then the impact of the KCPP will have a better chance of spreading, beyond the currently limited number of schools with new equipment or tthe few villages with new boreholes. It will also be in a better position to convince its current (and prospective) donors that there is good reason to fund organisations like AMREF, rather than channeling money directly to poor communities, or to local government, or to other NGOs.  It will be able to show how its particular way of working adds value in a way that justifies the cost involved.

—————————

Footnote**: $60 million = $2,000 per model farm. There could be 30 model farms set up in Katine sub-county, which has five parishes.

Distributed accountability

In my recent visit to the Katine project I raised the issue of whether administrative processes in AMREF’s Uganda Country Office might present a significant project risk, increasing the likelihood that the KCPP might not achieve its stated goals by the end of 2010. In particular I was concerned with the efficiency and effectiveness of their procedures for procurement, the approval of quarterly workplans, and some other related management practices. Compared to many other development organisations I have worked with in the past, these procedures seemed to be unusually centralised. My concern here was not with AMREF’s internal management as an end in itself, but with its effects on development activities in Katine: the ability of AMREF field staff in Katine to implement planned activities on schedule, with some efficiency, effectiveness and (cumulatively) with some credibility.

In discussions in Kampala it became clear that many of the requirements for these processes (but not all) were set further up the organisational hierarchy, outside of Uganda. There are some upcoming opportunities for proposing changes (an Auditors review, and a Board meeting), but changes may not happen simply through lobbying done on behalf of one project, or even one country programme. Nevertheless, they would be worth proposing. Of particular concern to me was the Katine Project Manager’s very limited authority to buy goods and services locally (UGX 100,000 maximum), in surprising contrast to the Ugandan government’s emphasis on local procurement (the Katine Sub-country chief can spend up to UGX 30 million).

In parallel to discussions about possible changes in these requirements I also suggested the use of some management process indicators, with some demanding targets. Ideally information about monthly performance on these process indicators would be made known to all AMREF staff, and be included in the subsequent six months progress reports that are sent to the donors and made available to the Project Steering Committee in Soroti. This would provide a more distributed form of accountability, in contrast to the current progress reporting that focuses on the implementation of activities in Katine by AMREF field staff. Of course the country office could fairly say that they are already accountable, as representatives of the whole country programme. But this is not so true for the specific contributions they make to the KCPP and other projects in their country programme, through procurement and workplan approvals. If these supporting activities were given the same amount of monitoring attention as project activities on the ground then improvements in performance might make it easier to justify the share of KCPP money spent on country office salaries, to not only the donors, but more importantly to the stakeholders in Soroti.

Whether there will be any progress either through devolution, and/or more intensive monitoring, remains to be seen. Ideally, when I next return to Uganda it will be the country office staff who first alert me to any internal bottlenecks and delays. I will not have to discover them by working back from implementation delays as seen in Katine.

AMREF makes progress with its Open Information Policy

When I first met with AMREF staff in London in September(?) last year, at the very beginning of the Katine project, I raised the issue of transparency, and whether AMREF would consider developing a Disclosure Policy, which would spell out what documents would automatically be made publicly available, what would remain confidential and where case-by-case approval might be needed.

I was pleased to see some months later, in November 2007, that the AMREF Board had approved an Open information Policy (possibly already under development before my discussion in September). This was followed by an Implementation Plan, which is expected to be completed by the end of 2008. While I was impressed by the range of documents to be made available, I was a bit concerned at how slow this process was (the project runs from Sept 2007 to the end of 2010), and that the disclosure of project documentation was not explicitly mentioned in the Implementation Plan.

However, in the last few days AMREF have taken a major step forward, with the listing of eight project documents, on both its AMREF UK and AMREF HQ websites. These include four documents making up the first six months progress report, and four background project documents (relating to design and baseline data collection).

In my next visit I will be checking to see how many staff, especially in Katine, are aware of the Open information Policy. And their understanding of how it is to be operationalised. Often policies get developed, but then sit on the shelf, unknown to most people, and therefore only partially implemented at best.

It will also be important to identify how many local stakeholders are aware of this policy, especially in the Project Steering Committee at the district level, and the Project Management Committee level, in Katine sub-county. Issues of access to information, regarding project costs, have already been the focus of discussion in meetings held so far.

Confusing mobilisation and empowerment?

Taken as it stands, the recent article on the Guardian Katine website: “Katine community welcomes Amref’s empowerment drive” reinforces the concerns I expressed to AMREF in my January 2008 visit to the KCPP (the Katine project).

That was about the risk of confusing mobilisation with empowerment. These two processes are at the opposite end of a continuum, they are not the same kind of thing. Mobilisation is about pursuasion, getting people to do things you want them to do, or what you think they should be doing (e.g. getting their children immunised). Empowerment is about enabling people to do what they want to do, whatever that might be. Diversity of people’s behavior is often a good indicator of successful empowerment. For example, increased diversity of ways of earning income in a local economy, as an indicator of economic empowerment. Related to this diversity definition is the willingness to express differences of opinion with others. To argue, not just to cooperate. For example, to demand a health service which has essential drugs in stock, or to demand a school where teachers are present in their post.

It could easily be argued that because there can be place in a development project for mobilisation, as well as empowerment, that these two roles should be the responsibility of two different people in an organisation. Not the responsibility of the same person.

Unfortunately in Anne’s article Richard is introduced both as “the Amref officer for empowerment,…” and later quoted as saying “I lead all ‘mobilisation’ processes” If I was a UK reader of Anne’s article I would be wondering if there is risk that one of Richard’s tasks will end up being neglected, and whether that might be the most troublesome one: empowerment

Janet Anyango: unaffordable healthcare

“Janet Anyango’s son Yuventino suffers from cerebral malaria, and Janet has faced a constant struggle to care for him. But with little or no income and six other children, it’s been virtually impossible for her to cope. As Janet recounts her story, further tragedy strikes” See this video on the Guardian Katine website.

This video is gutwrenching…there is no other word for it.

It must prompt some hard (evaluation) questions in the minds of many visitors to the Guardian website. Such as:

• Why are the necessary drugs not available in the health centre?
• Why cant AMREF help her directly, with the money the Guardian has raised?

In the field of maternal health it is common to carry out a detailed analysis of the causes behind deaths during childbirth (called maternal mortality audits) and in cases of “near misses“, where the mother almost dies. This makes a lot of sense – if some one has died, when they should not have, then every effort should be made to learn as much as possible from this tragedy, so it will not happen again. Their lives deserve respect.

Janet Anyango’s story deserves the same kind of attention. While here in Uganda on my second visit to Katine I will be discussing Janet’s story with AMREF and others. AMREF staff in Kampala have already pointed out to me that there are a number of points in Janet’s story where good decisions could have made a big difference. And where AMREF’s support to the health services in Katine could increase the chance that such good decisions will be made in the future. For example:

• the distribution of treated bednets. Did the family receive any from the health services?
• the availability of early diagnosis. Early treatment is better than late
• the correct diagnosis of malaraia, versus a fever of another kind
• the appropriate prescription of drugs
• the availability of those drugs
• the correct use of those drugs

All this considered, it is still hard not to feel angry at someone for the empty shelves in the health centre. More hard questions need to be asked about what is wrong with the supply chain. There are likely to be both local and national reasons. I will follow up the local reasons next week, in Katine. Some of the national level reasons have already been explored by others in Uganda, incuding recently by the mass media (see the weblink at the end).

The video has already prompted me to ask how AMREF is responding at the national level to this problem of lack of availability of essential drugs. One of the criticisms that has been made of area-based projects like the KCPP is that they ignore the importance of national policies and practices. That is where changes are needed, it is argued. From my discussions with AMREF staff in Kampala yesterday it is clear that AMREF are engaging with national health policies via a number of channels, including on the issue of malaria drug supplies. I have encouraged AMREF to document that strategy and publicly report the progress being made (by AMREF in association with other important actors in Uganda). That work needs to be seen, including all the difficulties involved. Because this is a public issue that needs continuing public visibility.

On that theme, I hope to include some links here on related actions already being taken by others in Uganda. Here is one

• Govt hospitals full of expired drugs. Daily Monitor, 14th July, 2008

Crowdsourcing and international aid programs

In October 2007 Paul Bradshaw wrote an enthusiastic post (on his Online Journalism Blog) about the Guardian’s involvement in what could be seen as a crowdsourcing experiment with AMREF, an African NGO working in Katine sub-country in Uganda, and supported by the Guardian. In that post Paul quoted Guardian editor Alan Rusbridger:

“We’ll need money obviously. But, just as importantly we need advice and involvement. Among our readers are water engineers, doctors, solar energy experts, businessmen and women, teachers, nurses, farmers. We absolutely don’t need a stampede of volunteers, but we would like a technical know-how bank of people who are prepared to offer time and advice. We’ll let you know how to get involved as we go.”

Paul then emphasised that this was “In other words, crowdsourcing – but not crowdsourcing as seen so far in newspapers, where the focus is on asking readers to help gather or analyse information for a story: this is crowdsourcing to help address the actual issues identified by the story.”

After I made contact with Paul earlier this week he asked me if there was any evidence of crowdsourcing actually taking place. My initial reaction was reserved.

As described by Alan Rusbridger above, I think that crowdsourcing via the web may be a solution to a problem that is not necessarily recognised. I suspect AMREF might well argue that they have the necessary expertise, and where they don’t have it, they know how to find it. Certainly an early post by AMREF on the Guardian website was cautious about the possibilities. There has however been one important positive example of unsolicited help that AMREF has welcomed. That was the offer of solar panels for the new AMREF sub-office in Katine sub-county, where main supply electricity is intermittent at best. Something closer to crowdsourcing will be evident if and when AMREF seeks out specific types of help via the Guardian Katine website.

One more immediate challenge is the person power needed to “harvest” the ideas that might come via the Guardian website. The project has only recently appointed a communications officer, who will be paying attention to the Guardian website and other equally important more local communication tasks. The solar power offer was mediated via the Guardian, who told AMREF. AMREF did not have to find it after sorting through many less useful offers. There are of course pros and cons to that mediating role.

One relates to what I thought was a conflation of the roles of the Guardian and AMREF, as present in Alan Rusbridger’s statement. In practice they cannot and should not be doing the same thing. This is one of a number of reasons why I have argued that it is important to more clearly define what the Guardian’s specific role is, especially the role of its website. I think more clarity here could lead to more appreciation and use of the website by AMREF.

The other point about Rusbridger’s statement is that it does seem to assume that the main problems are technical when in fact it could be argued that they are really more social and institutional. For example, how do you get decentralised government to work effectively in countries like Uganda, and how do you aid agricultural innovation in a context where private ownership of land is not the norm? I suspect these problems are less amenable to crowdsourcing solutions, especially when the contributors are from other cultures. But I could be wrong.

These comments should not be read as an argument against using crowdsourcing in this context. In my view it is still early days with this experiment of linking a media organisation (as donor/intermediary), an aid organisation and a local community. There are some discussions underway about engaging the local community with the website, via localised internet access. If that happens the results could be very interesting. Local people are likely to have their own views about what information and ideas they want to access from the world that becomes more open to them this way.

regards, rick davies