In 2016, the Enhancing Community Resilience Programme (ECRP) in Malawi – managed by Christian Aid, and funded by UK aid, Irish Aid and the Norwegian Ministry of Foreign Affairs – has invested time and resources in two innovative monitoring and evaluation approaches that go beyond numbers, data and formal reports to uncover the real stories of the people and communities being reached.
The first, Picture Power has provided communities with the skills and equipment to monitor and evaluate the project by taking their own photographs.
The Qualitative Panel Assessment (QPA) has provided an in-depth, long term view of how lives and households have changed.
Sabine Joukes, ECRP's former chief of party, explains the thinking behind the two approaches:
‘We have all our quantitative data. We really get quite comfortable with this. The numbers are huge on a programme like ECRP and if you just look at the numbers they are all looking very positive and you think you are doing a great job.
‘Picture Power and the Qualitative Panel Assessment (QPA) give us this.’
ECRP aims and approaches
ECRP aims to eradicate extreme poverty and hunger, reduce vulnerability and strengthen the resilience of more than 355,000 people in Malawi. Christian Aid leads the consortium delivering ECRP across seven districts. The £15m programme, running until 2017, enables people to move beyond survival and subsistence, to enjoy thriving and dignified lives.
Picture Power: the aims
The aim of a community-based, participatory photo project is to provide communities with the skills and equipment to conduct their own project evaluation and/or monitoring.
Participants use photography as a tool for gathering qualitative data on the changes and challenges that have been important to them during the lifecycle of a project. This Picture Power project was implemented in two districts of Malawi – Machinga and Mulanje – with five participants chosen from a community in each location.
Those who took part were encouraged to take photographs of the activities, changes and challenges that have been important to them and their communities during their involvement with ECRP.
The participants had two days of training in camera use and visual storytelling, and then a week to take photographs. After this, the images were captioned and edited with the participants selecting their favourite images to present to their community in an exhibition.
Developed and managed by Christian Aid’s media, content and PR unit, Picture Power draws on the ‘most significant change’ story-gathering approach and can be tailored to suit specific programmes across a range of themes.
'The results are fantastic' says former ECRP chief of party Sabine Joukes. 'As a development programme, as programme staff or researchers, we have our own way of explaining resilience, a way of explaining why a household has taken up a particular activity.
'With Picture Power we took this completely out of the equation – we completely walked out of the picture.
'They have taken pictures of things that we would not ever have thought to take pictures of, and each one has a unique and important meaning to that person. The participants themselves have noticed different things going on – they have an awareness of what is important and interesting.
‘That’s what is so enriching about Picture Power. The participants are left to their own devices – quite literally. The images are not only a great source of information and learning for us, but they resound with meaning and fresh perspectives. They are really beautiful too.’
Picture Power: the exhibition
After a week of taking photographs, Picture Power participants, with support from Christian Aid staff, wrote captions describing the subject and indicating why the photograph was significant. Each participant selected between 15 and 25 images that they felt represented the most important outcomes of ECRP from a personal perspective and from the point of view of the wider community.
The chosen images were then printed and each participant presented their work to members of their community in an exhibition. Once all of the participants had presented their photographs, community members were asked to select the issues in each individual’s presentation that they felt were the most important. They did this by placing a coloured sticker, representing their gender, on the photograph they agreed represented the most significant ECRP outcome from their own perspective.
When all members of the community had voted for their photographs, the results were recorded by Christian Aid monitoring and evaluation staff. The choices were compared with results from more traditional evaluation exercises to gain a more holistic view of the effect of ECRP on the target communities.
Qualitative Panel Assessment (QPA)
An annual component of ECRP is a Qualitative Panel Assessment (QPA) documenting stories showing the impact of the programme at household level. Using a 'most significant change' method, this important and in-depth approach adds a human face to reports and figures, supplementing more quantitative milestones and helping to explain and validate findings.
Through the QPA, various examples have emerged of ECRP households making real strides towards resilience and even prosperity. Certain activities have been more successful than others, in some districts more than others, and there are different degrees to which households have embraced the programme.
A minority of households have not seen progress, sometimes for reasons that may be important in an assessment of the programme as a whole. The QPA highlights these examples too.
In 2016, 25 households were interviewed – eight in Nsanje, nine in Chikwawa and eight in Kasungu. Where possible interviews were conducted with two members of each household, with efforts made to gather a variety of perspectives from women and men, the old and the young, and people from different sections of the communities. The interviews were supplemented by GIS (geographic information system) mapping of households and fields. This computer system can show many different kinds of data on one map, enabling people to more easily see, analyse and understand patterns and relationships.
'What we want to see with the QPA is an in-depth view of the lives of the households. It’s a much more qualitative perspective.
‘It is a longitudinal view of the programme – how are people’s lives changing, to what degree? What are the good changes and what are the not so good changes? Why have things changed.'
The QPA will provide longitudinal qualitative data which will be used in the end of project evaluation in 2017.
Most significant change
Most significant change (MSC) is a specialised method of data collection aiming to identify change stories. It requires an open ended style of questioning with carefully structured questions.
One of the key components of the MSC is defining ‘domains of change’, which establish clear lines of questioning. The following major domains were identified for ECRP:
1. Changes in income sources.
2. Changes in assets.
3. Changes in food availability.
4. Changes in farming practices.
5. Changes in vulnerability to disasters and climate change.
The next steps
From the QPA, it is clear ECRP has made a huge difference to the lives of many beneficiaries. There are a number of extraordinary stories that have been documented, where people’s lives have been transformed as a direct result of being reached by the programme.
There are also households where this is not the case. In these instances, the learning is perhaps even more crucial and much can be gleaned from community members' responses and attitudes.
It is the hope of the research team that this process and these findings will be taken on board and have a genuine influence on ECRP's approaches in the future.
The QPA was conducted by C12 Consultants, an environmental and development consulting firm dedicated to providing international standards enriched by local insight.