All posts
Every organization that aims to improve the lives of others has to confront some basic questions: Who should we help with our limited resources? How should we find the people we鈥檇 like to help? And how should we make sure that the people we鈥檙e actually helping meet the criteria we鈥檝e established?
Surprisingly, this isn鈥檛 something that NGOs talk about very much. As a rough indicator, think about how many NGOs you know that report representative, quantitative measures of how poor the people they serve are. We haven鈥檛 found many.
At 91精品 we鈥檝e worked hard to quantify and report transparently on our performance in this area. (Historically we鈥檝e been reaching people living on around US $0.65 nominal per person per day.) We鈥檝e also become known for using some innovative techniques to accomplish this. For example, we鈥檝e used the material that people鈥檚 roofs are built from as an eligibility criterion: people with thatch roofs turn out to be much poorer than those with metal roofs in areas where we鈥檝e been working, and it鈥檚 even possible to confirm roof type from space using satellite imagery.
But this isn鈥檛 the right criterion for Homa Bay County, where we recently started working. Fewer than 3% of households in Homa Bay have thatched roofs (compared to around 40% where we operated previously, in Siaya County). Using thatched roofs and homelessness as selection criteria was no longer going to work 鈥� we had to think of new approaches.
To inform and structure this decision, we tested out several different targeting methods and eligibility criteria to evaluate their pros and cons, and the circumstances in which we thought they would be most and least useful. For example, we tested: a variety of proxies, such as per-capita housing space and housing materials; community-based targeting, where members of a village nominate, through various means, which they think are the poorest households; points-based systems such as the Progress out of Poverty Index (PPI) and the Multidimensional Poverty Index (MPI); subjective assessments like our field officers rating on a 1-5 or 1-10 scale the poverty of a household or the quality of their house; and various blends of these different approaches.
We wanted to evaluate how well different techniques worked along several dimensions. We looked for accuracy (did the method actually identify poor households), perceived fairness (did community members think that the method was fair), gameability (how easy was it to cheat the system), and cost (how expensive was the method to administer). We will write more about the targeting project on our blog over the coming months.
Besides testing and evaluating different techniques, the project allowed us to settle on usable eligibility criteria for our new home in Homa Bay. We found that there was no single, objective replacement for thatched roofs as a criterion (e.g. mud floors or the presence of a latrine) in our new location that also met the bar for accuracy, perceived fairness, gameability, and cost. We also found that some methods that might work well in other contexts didn鈥檛 work well in Homa Bay. For example, community-based targeting was perceived as fair and was cost-effective, but it was not particularly accurate, in part due to the high prevalence of clanism in Homa Bay 鈥� villagers were sometimes nominating those in their own clan over the poorest households in the community.
In the end, balancing our four criteria, we decided to use an algorithm that takes into account several factors, including housing (e.g. house size), assets (e.g. presence of a latrine), vulnerable recipient status (e.g. homelessness), and other criteria.
There are major differences between this method and the thatch and homeless criteria we used in Siaya. Because the new criteria take into account many factors, it鈥檚 harder to game the system. Also, the new criteria include a broader range of vulnerable statuses than the criteria in Siaya by adding widows and child-headed households into the algorithm. This is likely to increase the perceived fairness of our eligibility criteria. The new criteria also have some weaknesses: it鈥檚 slightly more expensive from an operational standpoint, because the criteria involve more questions, and it is more difficult to explain to community members why a given household was or was not eligible. This may counteract the perceived fairness of enrolling more vulnerable-status recipients.
Taking all of these dimensions into account, our new eligibility criteria will allow us to identify and serve the poorest of the poor in Homa Bay. Even though the new criteria make this system slightly more expensive, that cost will likely be mitigated by a decrease in gameability and an increase in accuracy. And although the new algorithm may be perceived as harder to understand, we hope that by accounting for vulnerable-status groups the criteria will be perceived as more fair in the communities where we work.
In the process we also learned a lot about how to identify the poor 鈥� about other criteria, how to implement them, and what works and what doesn鈥檛. We expect to keep adjusting our eligibility criteria and targeting methods as we bring our program to communities in more places since no two places are the same. We will continue to post updates about our innovations in targeting and eligibility criteria and our ongoing adjustments to our new operations in Homa Bay.
