Pursuing riskier opportunities for impact in global health and development

Here we show how we aim to maximize the impact of our grants by pursuing higher-risk opportunities that are neglected by other funders.

How can we have the biggest impact?

Considering the scale of the challenges we face — the billions who can’t support their kids all the way through school, the millions in extreme poverty who suffer from hunger and disease, and the thousands of preventable deaths each year — it’s natural to seek opportunities to make lasting and transformative changes. However, sometimes it feels like even our highest-impact recommendations are not enough.

At Founders Pledge, we often compare impact-per-dollar produced by global health and development funding opportunities to the impact-per-dollar produced by GiveDirectly’s unconditional cash transfers. We do this because it seems reasonable to believe that if a program improves the wellbeing of its beneficiaries less than a cash transfer equal to its costs would, you should just give the beneficiaries cash instead. It’s rare to find programs that are more impactful than cash transfers because recipients can use cash to buy whatever is most useful to them in their unique circumstances.¹ But can it really be that even with all our knowledge and connections, the best we can do is just send cash?

We suspect that the answer to this question is no. We already know of some interventions that seem to be more cost-effective approaches to saving lives than cash transfers, such as providing cheap medical supplies like antimalarial bed nets or vitamin supplements. Our research also suggests that some anti-poverty programs that give people a productive asset alongside a complementary set of supports, like Bandhan’s Targeting the Hard Core Poor program, are a bit better than cash transfers in terms of improving wellbeing.

But we suspect the best opportunities might be programs that are less direct, harder to study and neglected by other funders. One reason is that the biggest “wins” in global health and development have come about through policy changes rather than direct service provision. As we discussed in our previous blog, governments in low- and middle-income countries spend 10 times more on development programs than development aid agencies and philanthropists combined. Funding programs that improve the efficiency of this spending or improve other policies that drive development like education, trade and migration can be highly leveraged opportunities for impact. Examples include things like providing policy advice to a government that’s instituting economic reforms² or advocating for innovative new programs like Advanced Market Commitments for vaccines.³ Yet such opportunities are time-sensitive and highly contextual. Identifying them is challenging for researchers, practitioners and funders alike.

Assessing high-risk opportunities

At this time we’re not sure which organizations are best-placed to influence policy or how they compare to our tried-and-true recommendations. We expect figuring this out will be difficult. The feasibility, timing and impact of these organizations’ interventions are likely to be much less certain than the relatively straightforward programs implemented by our current set of recommendations, which are more direct and testable.

While assessing the potential impacts of opportunities to influence policy is challenging, we view it as an important part of our commitment to maximizing impact for those facing the challenges of poverty. To maintain our rigorous standards while evaluating funding opportunities with high uncertainty, we will make greater use of new sources of information to inform expected value calculations. This approach allows us to account for uncertainty by multiplying the benefits of an intervention in different scenarios by the probability of that scenario coming about.

To illustrate, imagine that a researcher comes to us with a proposal for a research project on an innovative anti-malaria treatment. She needs $1 million to fund the work required to develop the treatment and distribute it among the target population. She hasn’t been able to find other funders, so if we don’t fund it she’ll have to shelve the project. If the new treatment works, it will save 5,000 lives that would otherwise be lost.

$1 million is a lot of money. GiveWell’s current best estimate is that if we just used that money to fund the best antimalarial treatments available today, we could save the lives of almost 300 people with much higher certainty. How do we know whether it’s worth taking the risk and funding the research instead? This is where expected value calculations come in: since the research saves so many lives if it succeeds, we only need it to work 6 percent of the time for it to be as impactful as our current top recommendation.⁴ Other funders might balk at funding a project that they think has a 94 percent of having no impact. But if we fund many of these projects, we would expect this higher-risk approach to save just as many lives as one that funds smaller-scale, but more certain, interventions. And if we could find projects that succeed much more than 6 percent of the time, we could save many more lives over time.

The biggest successes in policy change or research have saved hundreds of millions of lives, created trillions of dollars of wealth, and lifted hundreds of millions of people out of poverty. If we could find more opportunities like these, it would be well worth funding such projects even if their chance of success was 1 percent or lower.

Since the probabilities we assign are subjective, expected value calculations only work if we are reasonably accurate. That is to say, if we give a few different projects a 50 percent chance of succeeding, roughly half of those programs should actually work. Fortunately, we don’t have to just intuit these probabilities. We could instead survey experts or use a “reference class” of similar projects for which we already know the results. For the malaria example above, we might assess the project’s chance of success by checking what proportion of similar medical research projects have resulted in new therapies in the past or asking a sample of well-informed, objective experts to assess the project.

We plan to track our predictions so we can improve them over time. At the same time, we do recognize that this process is more prone to biases and mistakes than research that relies on external experimental evidence. We’re going to be realistic and transparent about these risks, the expected payoffs, and our major uncertainties so that our members can make informed decisions about their philanthropy.

Breaking the mold

Since taking on more risk means funding projects and organizations that may sometimes fail to achieve their outcomes, we might expect potential impact and risk of failure to be correlated. It would be strange, of course, if there were extremely impactful and low-risk funding opportunities that leaders in this space like GiveWell and the Bill and Melinda Gates Foundation had missed. But we don’t think this is a strict trade-off. Instead, we think it’s likely that we can find gaps in this funding ecosystem by identifying dimensions along which the Global Health and Development Fund may have a comparative advantage. We have a few ideas about what these dimensions could be.

First, we suspect that some funders have a natural bias towards interventions with high “attributability”, i.e. for which an individual donor can easily see the change their grant caused. Providing general support that allows outstanding research and advocacy organizations to carry out their work, but which may not be linked to highly specific activities or outcomes, could be a good bet for the Fund.

Second, we will leverage our experience in evaluating programs which are difficult to evaluate experimentally. While we still think it’s important to be thoughtful and rigorous when assessing claims to impact, we’ve worked to expand the scope of our recommendations beyond direct interventions. For example, our report on Evidence-Based Policy assesses the impact of funding technical assistance work by measuring the impacts of a policy change and quantifying an organization's contribution. We’ve also made several methodological changes behind the scenes. For this Fund, we’ll look to apply these new tools and consider a wider range of evidence for effectiveness in order to identify impactful organizations whose work might not be considered by other funders.

Third, other funders may be reluctant to fund programs where the probability of success is low or unknown, like policy advocacy or the hypothetical malaria research discussed above. This helps funders in the public eye avoid embarrassing failures and negative impacts. But it also risks missing out on a lot of value. By funding a portfolio of such options, we can mitigate downside risks and increase our expected impact.

Concluding thoughts

Overall, it seems likely that there are gaps in the existing ecosystem that Founders Pledge members, being entrepreneurial and impact-focused funders, may be especially well-suited to filling them. By evaluating and funding interventions which have lower attributability, non-experimental evidence of effectiveness, or a low probability of winning a huge pay-off, we could achieve larger and more lasting impacts for our beneficiaries. That’s part of the mission of the Global Health & Development Fund, and one we’re excited to pursue together with our members and partners.

In our fourth and final post in this series, we’ll discuss our plans for sharing what we learn over the first year of managing the Global Health & Development Fund and how we plan to measure the Fund’s success. Read part four of the series

Notes

1. “While there are sophisticated arguments for in-kind versus cash as mechanisms for transfers, there is little or no empirical evidence that we should expect in-kind programs to have enormously larger impacts on well-being (say, a factor of two) than just transferring cash. And there are inevitably administrative costs of any program and hence the cost to transfer a dollar to a person is inevitably going to be larger than a dollar” (Pritchett, Lant. “Alleviating Global Poverty: Labor Mobility, Direct Assistance, and Economic Growth”, Center for Global Development, Working Paper 479, March 2018, p. 13) ↩

2. “Where things are that screwed up, helping countries move toward effective economic management will be the most effective thing that we can do for poverty. You can't easily do that as an outside donor. I'd say that Ethiopia at the moment is going through tremendous reform, and we really ought to be focusing attention on helping Ethiopia through that transition. There's tremendous potential for growth, and they're fundamentally changing policy there in ways that could be really beneficial to the poor. So jump on those opportunities when you see them; we can't make them happen, that's something the developing country has to decide to do themselves, but we should help them as much as we can” (Glennerster, Rachel, “Fireside Chat with Rachel Glennerster” (EA Global talk), 15 April 2019, https://www.effectivealtruism.org/articles/fireside-chat-with-rachel-glennerster/ ↩

3. See, for example: https://www.cgdev.org/impacts-and-influence/advance-market-commitments-vaccines ↩

4. To obtain this result, we calculate probability p such that (p * 5000)/ $1,000,000 = 1 / $3400 ↩