Development initiatives and projects are as good as the social impact they can demonstrate.
NGOs and entrepreneurs continue to burst at the seams with innovative ways to promote social good, tackle poverty, enhance global health and preserve the environment. But precisely how to measure the social impact of their initiatives remains a recurring question.
Last August, I teamed up with two fellow MPA students and the Global Livingston Institute (GLI), a non-governmental organization that seeks to improve communities globally and disseminate innovative approaches to international development, to tackle this question for GLI’s Sustainable Farming Collaborative project.
GLI launched the Sustainable Farming Collaborative following results from a previous needs assessment conducted by an earlier team of MPA students acting as consultants. Working in partnership with Development in Gardening (DIG) and Children of Peace Uganda (CPU) in Lira, Uganda, the Sustainable Farming Collaborative seeks to impart trainees, many of whom are former child soldiers (FCS), with the agricultural knowledge needed to cultivate, manage and sell cash crops sustainably. By fostering farmer collaboratives and extending the newly trained farmers’ access to external markets, new farm enterprises are an opportunity for the farmers to create an income, revel in new possibilities and unfold a new part of their social identity. Acting as consultants, our team was tasked with creating data collection tools that can be used to evaluate the initiative’s social impact on its participants over time.
It took four months of reviewing past evaluation reports for similar agricultural training programs, meeting with academics, specialists and practitioners, drafting and redrafting before creating a baseline and follow-up survey for the formative evaluation and mid-term review of the program.
From this experience, I want to share three takeaways that can optimize social impact measurement efforts for your development project.
Ensure all key variables used in your theory of change have operationalized definitions
A well-outlined theory of change is vital to the development of data collection tools capable of measuring an intervention’s impact. Although our team had an understanding of what the Sustainable Farming Collaborative’s mission and vision was, no formalized theory of change had been outlined. Our early discussions with the project’s implementers were dedicated to mapping out the inputs, activities, and outputs that the initiative entailed. Critical thought was also poured into figuring out the ‘how,’ that is the causal relationships that underpin these elements and facilitate social impact.
In our discussions, we also made it a priority to ensure that the six social impact outcomes (agricultural proficiency, economic advancement, food security, social capital, empowerment and women’s empowerment) included in the theory of change had clearly stated, operationalized definitions. This was vital to avoid ambiguity. Empowerment, for example, can be defined in several ways, which can each be measured in several different ways. Defining each of the six outcomes proved to be helpful in selecting indicators capable of measuring the actual specific intended outcome, which were ultimately weaved into the final data collection tools.
Newly trained farmers share the impact of the Global Livingston Institute (GLI)’s agricultural training program in a focus group held in the Lira District of Northern Uganda. Photo by Laurie Miller, Cornell Institute for Public Affairs.
Think critically about timing and evaluability: Is it too early to see impact?Time is important. When our team stepped in as consultants last August, the Sustainable Farming Collaborative’s implementers initially planned to administer the follow-up survey three months after the agricultural training ended.
To improve the prospect of the follow-up survey producing useful data, we recommended conducting the follow-up survey six months after the training ended. At the half-year mark, we theorized, social impact outcomes like an increase in social capital and economic advancement will have become more notable. Moreover, we predicted that extending the time frame would help control for variation in factors external to the training program such as crop cycles. This would help strengthen the correlation between the training program and subsequent increases in crop production and farmer income.
To ensure cultural relevance, turn to focus groups in the formative stages of research
Focus groups can be used in the formative stages of research to elicit information on how respondents think about outcomes that are less quantifiable.
For example, although ‘women’s empowerment’ generally refers to the betterment of women, how to measure this is largely dependent on the culture a project is situated in. In northern Uganda, where the Sustainable Farming Collaborative is located in, most households are female-led as a result of the war. In this context, common indicators like ‘increases in household decision-making power’ would not translate well.
Rather than assuming which indicators best illuminate changes in women’s empowerment, our team recommended that GLI facilitate a semi-structured focus group discussion with female farmers that allows for the exploration of their experiences since participating in the program. Transcripts of these focus groups can be analyzed to generate candidate domains and items for an amended baseline and follow-up survey.