Will the Katine baseline survey be of much use?
28 May, 2008
Information about the Katine baseline survey can be found here:
Development aid projects don’t always start with baseline surveys. More widespread use of baseline surveys would be a good thing. They can be useful for a number of reasons:
- The results can be used to engage stakeholders in discussions, at the beginning of a project, about what needs to be changed, and how
- The results of subsequent re-surveys can be compared to the baseline survey to see what has changed (and not). This information can be useful both during project implementation and at the end of a project: to help improve the effectiveness of a project, and to help show its overall impact to other parties e.g. policy makers or donors.
Nevertheless, I have some major concerns about the household baseline survey that was carried out in Katine in January this year. I am concerned that it may not be able to serve the second of the two purposes I have listed above. It may not be of much use.
There are two reasons why I am concerned. The first is about what is missing. The household survey randomly sampled 95 households from six parishes in Katine sub-country. AMREF will be proving assistance to all six parishes over the next few years. The sample did not include any similar parishes nearby that could be used as a comparator, otherwise known as a control group. So, even if the re-survey in 2010 does show significant improvement in people’s lives it will not be clear what this means. It could be a reflection of the fact that conditions have improved across the district, and across the country even. “A rising tide lifts all boats”
The lack of a control group is not necessarily a disaster. Often, especially in large development projects with a significant emphasis on decentralised planning, interventions will vary across locations. If this is the case then we can also expect that the desired outcomes (i.e. changes in peoples lives) to vary across locations. We could make some predictions, then test these against observations, to find out what kinds of interventions are associated with what kinds of outcomes. For example, does investing across all sectors (health, water, education), make more of a difference than investing heavily in just on sector (e.g. water). Or is it simply a matter of how much is invested, with bigger investments making more of a difference than small investments?
Will this internal comparison be possible in Katine? The random sample of households was designed to make statistically valid comparisons between the six different parishes. However, from the information I have seen so far, it is not expected that AMREF’s interventions will vary substantially between these parishes. The parish is not an important planning unit, in the way that AMREF is working in Katine. The most common unit of planning seems to be the village. There are health committees, water committees, farmers groups and credit groups at the village level, and AMREF (and its partners) will be working with all of them. In addition, there are some larger planning units: the 13 schools (and their associated School Management Committees and Parent Teachers Associations) and the three major health centres.
Fortunately, AMREF has begun to develop a database on all villages, and on the schools and health centres. In my next visit I will be asking about the kinds of data being kept in those databases, and how well it is being maintained, and used. This is where the investment in baseline data collection and regular monitoring thereafter, will be most crucial.
PS1: There may be some voices within AMREF who think this sounds too much like research, too removed from the practicalities of improving peoples lives. I would describe it as a kind of action research, that can help ensure that AMREF’s interventions are as effective as possible, and replicable by others.
PS2: This more recent posting is also relevant, because it talks about comparisons between Katine sub-county and other sub-counties in Soroti district