Information about the Katine baseline survey can be found here:

Development aid projects don’t always start with baseline surveys. More widespread use of baseline surveys would be a good thing. They can be useful for a number of reasons:

  • The results can be used to engage stakeholders in discussions, at the beginning of a project, about what needs to be changed, and how
  • The results of subsequent re-surveys can be compared to the baseline survey to see what has changed (and not). This information can be useful both during project implementation and at the end of a project: to help improve the effectiveness of a project, and to help show its overall impact to other parties e.g. policy makers or donors.

Nevertheless, I have some major concerns about the household baseline survey that was carried out in Katine in January this year. I am concerned that it may not be able to serve the second of the two purposes I have listed above. It may not be of much use.

There are two reasons why I am concerned. The first is about what is missing. The household survey randomly sampled 95 households from six parishes in Katine sub-country. AMREF will be proving assistance to all six parishes over the next few years. The sample did not include any similar parishes nearby that could be used as a comparator, otherwise known as a control group. So, even if the re-survey in 2010 does show significant improvement in people’s lives it will not be clear what this means. It could be a reflection of the fact that conditions have improved across the district, and across the country even. “A rising tide lifts all boats”

The lack of a control group is not necessarily a disaster. Often, especially in large development projects with a significant emphasis on decentralised planning, interventions will vary across locations. If this is the case then we can also expect that the desired outcomes (i.e. changes in peoples lives) to vary across locations. We could make some predictions, then test these against observations, to find out what kinds of interventions are associated with what kinds of outcomes. For example, does investing across all sectors (health, water, education), make more of a difference than investing heavily in just on sector (e.g. water). Or is it simply a matter of how much is invested, with bigger investments making more of a difference than small investments?

Will this internal comparison be possible in Katine? The random sample of households was designed to make statistically valid comparisons between the six different parishes. However, from the information I have seen so far, it is not expected that AMREF’s interventions will vary substantially between these parishes. The parish is not an important planning unit, in the way that AMREF is working in Katine. The most common unit of planning seems to be the village. There are health committees, water committees, farmers groups and credit groups at the village level, and AMREF (and its partners) will be working with all of them. In addition, there are some larger planning units: the 13 schools (and their associated School Management Committees and Parent Teachers Associations) and the three major health centres.

Fortunately, AMREF has begun to develop a database on all villages, and on the schools and health centres. In my next visit I will be asking about the kinds of data being kept in those databases, and how well it is being maintained, and used. This is where the investment in baseline data collection and regular monitoring thereafter, will be most crucial.

PS1: There may be some voices within AMREF who think this sounds too much like research, too removed from the practicalities of improving peoples lives. I would describe it as a kind of action research, that can help ensure that AMREF’s interventions are as effective as possible, and replicable by others.

PS2: This more recent posting is also relevant, because it talks about comparisons between Katine sub-county and other sub-counties in Soroti district

In the Katine Chronicles (May 23rd, 2008), Madeline Bunting has written on the state of education in Uganda

Her article highlights the problems caused by the pursuit of inappropriate objectives. In this case Universal Primary Education, simply interpreted as having all primary school age in school. The government of Uganda was required to prioritise this objective, as one of the conditions associated with the provision of debt relief by international donors.

“The pressure to meet the MDG [millennium development goal] on education has forced Uganda into a desperate overexpansion of the education system, argued some of those I met. Stephen Ochola, for example, the district chairman of Soroti, believes that the MDGs have brought the Ugandan education system almost to its knees. Huge classes of 75 children are common and they rarely have enough books, let alone desks and chairs. Teachers have to share blackboards and eke out chalk. Ochola argues that children are sitting in school for years, learning next to nothing.”

This development begs the question of what AMREF’s objectives are for the development of education in Katine. And whose objectives are these? Are they those of the Minister of Education in Kampala, those of the Soroti District authorities, or the communities associated with each of the 13 schools in Katine sub-county? It is clear that there are differences in views about what is needed between the Soroti and Kampala, and its possible there may also be differences of views between various communities in Katine, and Soroti.

Madeline’s article prompted me to revisit the AMREF project documents, to see how the education objectives had been specified. In the Monitoring and Evaluation Plan (Nov 2007and Jan 2007) there are three indicators for desired changes in children’s education status (and many others about the education process). These are:

  • Primary school completion rates
  • Percentage of children aged 6 to 12 attending school
  • Percentage of orphans age 6-12 years attending school

As far as I am aware there are not yet any targets for expected achievements on these indicators, so it is not yet possible to identify how closely these objectives relate to national targets versus what the district, sub-county or local communities think is realistic and desirable. The issue of appropriate targets needs discussion with stakeholders in Katine and elsewhere. The issue of quality of education also needs some attention,… is completion of primary school an adequate indicator, capturing the type of changes that AMREF and its partners want to see happen?

On this last point it was interesting to see that in the Minutes of the Katine stakeholder meeting on April 3rd 2008, under “Key questions/ issues raised by stakeholders”, there is the following bullet point:

  • Education focused on attendance. There was a need to look at literacy levels and include out of school youth also. Education kept being mentioned as a key priority by district officials.

This page will provide links to:

AMREF UK on Katine

21 May, 2008

This page will provide links to:

A baseline household survey was carried out in January 2008. The full text of the report of the survey is available here

The results are discussed on the Katine blog

My comments to AMREF on the baseline survey are available on the Guardian website, or here on the Evaluating Katine blog

Contents of the report

Executive Summary ………………………………………………………………………………….5

I. Background ………………………………………………………………………………………..10

2. Household Characteristics……………………………………………………………………14

3. Household Water Sources …………………………………………………………………….15

4. Household Sanitation and Hygiene………………………………………………………..20

5. Reproductive Health ……………………………………………………………………………23

6. HIV/AIDS Knowledge …………………………………………………………………………..26

7. Fever/Malaria …………………………………………………………………………………….31

8. Home-based care and community sensitization — Village Health Teams……34

9. Household Food Security and Livelihood ………………………………………………..35

10.Conclusions and Recommendations ……………………………………………………..39

This report is available online, on the Guardian Katine website.

I have made reference to it in the Summary of Recommendations section of my January 2008 Visit Report

This AMREF document is available on the Guardian newspaper Katine website, in an article titled “Stakeholders question timescale of Katine project

I posted the following comments:

“I was pleased to see the article about the recent stakeholder meeting held in Katine (“Stakeholders question timescale of Katine project”) and to be able to access online a record of the meeting (“Katine stakeholder meeting and Preliminary project steering committee meeting April 3rd 2008″)

I would hope that further meetings like this are also shared on the Katine website. In the process, I hope that some more detail is provided on the participants who are not government representatives. It seemed that in this meeting almost all participants were government officials or representatives. If that was meant to be so, it would be useful to make that clear. Similarly, records of future meetings should be very clear on who they were meant to include.

My main reason for emphasising this point is that one of the evaluation criteria I will be using (as the external monitor/evaluator) is equity. This means fairness of process as well as fairness of result. There is a second dimension here as well, that of transparency. If a record is kept of the participants in such meetings it will be possible for myself, the Guardian journalist (Richard Kavuma) and anyone else, to make follow up contact with the meeting participants later on, both to hear their views of the meeting, and of what has happened thereafter.”