top of page
png8 copy.png

OUR RESEARCH PROCESS 2023-2024

This is a summary of our general research process for finding cost-effective interventions. The process inevitably varies across different cause areas that we evaluate. This version has minor updates from the 2021-22 version available here.
 

This process is also fluid and ever-changing; we test each stage as we go and adapt accordingly.

OUR GOALS

Our primary goal is to identify the most high-impact intervention opportunities.

Our secondary goals are:

Providing support to future incubatees through high-quality research reports and implementation advice.

Seeking additional wins (supporting other organizations, keeping track of high-impact interventions, identifying potential program participants).

Ongoing improvement of the research process.

OUR DECISION CRITERIA

We determine policy interventions to be particularly high-impact based on the following 5 targets (applied in priority order):

1. Substantially better than our bar to beat – E.g., for global health policy we set a bar of 5x more cost-effective than AMF and other top GiveWell evaluated charities.

2. Evidence quality – Evidence should be high quality and robust to uncertainty.

3. Limiting factors –  There should be minimal ways that a new charity could fail.

4. Variation – The overall set of opportunities identified should vary in type.

5. Maximally impactful – Highest expected value.

THE PROCESS

Our research process comprises seven stages, which were preceded by strategic process design. We time-cap each stage to produce two-three high-quality recommendations that will be implemented through our yearly Incubation Program.

STAGE 0:  PROCESS DESIGN AND TOPIC LEARNING

Goal 1: Designing the decision-making process for the cause area.
Time spent: ~30 Hours
Implementation:

  • Defining crucial parameters for decision-making

  • Research into available metrics

  • Creating/updating templates for each step and method

  • Establishing a benchmark idea to compare against

Goal 2: Gaining background knowledge

Time spent: ~30 hours per person

Implementation: 

  • Broad reading and research discussed with the team

  • Cross-applicable research presented to the team

STAGE 1: GENERATING IDEAS

Goal: Creating a list of ~150-400 intervention ideas in our target cause area

Time spent: ~ 50 Hours

Implementation: 

  • Generating ideas based on:

    • Contacting experts

    • Contacting key organizations

    • Broad reading and cross-applicable research

    • CE Team’s input

  • Put ideas into key categories

  • Remove any obviously low-quality ideas

STAGE 2: QUICK PRIORITIZATION

Goal: Narrowing down ideas list to 25% (~50) most promising interventions

Time spent: ~120 Hours

Implementation: 

  • Researching top considerations that might eliminate sets of ideas

  • Organizing ideas (e.g., merging similar ideas)

  • First independent rating, taking 2-12 minutes  (Researcher 1) 

  • Second independent rating, taking 2-12 minutes (Researcher 2)

  • Additional research 

  • ​​Review meeting and final decision-making

STAGE 3: SORTING IDEAS

Goal: Narrowing down ideas list to 25% (~15) most promising interventions

Time spent: ~130 Hours

Implementation: 

  • Assessing strength of evidence for the intervention (40 min. per idea)

    • Looking for evidence (e.g., RCTs, recommendations from effectiveness-focused organizations, macro/country-level data on effects, consensus among experts, existing effective implementations)

    • Assessing quality of evidence

    • Cutting 20% of least promising ideas

  • Paths to failure weighted factor model (15 min per idea)

    • Each team member individually scores each idea on a number of factors related to how likely the idea is to fail to make a successful charity

  • Quantitative modeling (30 min. per idea)

    • CEA/BCR estimates 

    • Comparing to the benchmark interventions

STAGE 4:  EXPERT INTERVIEW AND CRITICAL UNCERTAINTY ANALYSIS

Goal: Arriving at a ranked list of the top ~10 ideas for the most complex research stage

Time spent: ~150 Hours
Implementation: 

  • At least 1 expert interview on each idea

  • Identify 1-6 crucial considerations or cases against each idea

  • Spend 1.5h evaluating each of the identified crucial considerations.

STAGE 5: DEEP DIVE

Goal: Selecting top 2-4 charity ideas that we recommend to be started through our 2022 Incubation Program, by producing high-quality research reports on the top 5-10 ideas.

Time spent: 700-900 Hours

Implementation: 

  • Background reading

  • Mapping a causal chain of the intervention and theories of change

  • Geographic assessment weighted factor model

  • Stakeholder mapping

  • Interviewing experts

  • Quality of evidence assessment (academic evidence, case studies, historical data, macro-level data trends, theoretical evidence, assessing the strength of evidence for each step in the causal chain)

  • Assessing the potential for a new charity (neglectedness, limiting factors, ease of implementability, scalability)

  • Cost-effectiveness analysis

  • Externalities/crucial considerations

  • Expert review

  • Final decision meeting

STAGE 6: WRITE-UP, IMPLEMENTATION REPORTS AND FEEDBACK

Goal: Creating polished reports and 2-3 implementation notes on the most promising interventions in order to get new charities up and running.

Time spent: ~5-30 Hours

Implementation: 

  • Work with the outreach team to polish the reports ready for publication

  • Write up additional thoughts and reading lists and contacts lists and any additional research that could be used to inform new charity founder working on the top ideas.

  • Give feedback on and improve the research process.

If you are interested in seeing a longer, internal, unpolished version of the process, please contact Morgan at morgan@charityentrepreneurship.com​

bottom of page