png8 copy.png

OUR RESEARCH PROCESS 2021

The focus of our research in the year 2021/2022 is finding the most cost-effective interventions in the area of international development policy.

OUR GOALS

Our primary goal is to identify the high-impact intervention opportunities.

Our secondary goals are:

Providing support to future incubatees through high-quality research reports and implementation advice.

Seeking additional wins (supporting other organizations, keeping track of high-impact interventions, identifying potential program participants).

Ongoing improvement of the research process.

OUR DECISION CRITERIA

We determine policy interventions to be particularly high-impact based on the following 4 targets (applied in priority order): 

Substantially better than current top global health charities – e.g., 5x better than AMF (Satisficing target).

Variation – The overall set of opportunities identified should vary in type.

Evidence quality – Evidence should be high quality and robust to uncertainty; there should be minimal ways that a new charity could fail.

Maximally impactful – highest expected value (maximizing target).

THE PROCESS

Our research process comprises 7 stages, which were preceded by strategic process design. We timecap each stage to produce 3-6 high-quality recommendations that will be implemented through our yearly Incubation Program.

STAGE 0: PROCESS DESIGN

Goal: Designing the decision-making process for the international development policy cause area.
Time spent: ~150 Hours
Implementation:

  • Defining crucial parameters for decision-making

  • Research into available metrics

  • Creating templates for each step and method

  • Establishing a benchmark of recommendation

  • Test run

STAGE 1: TOPIC LEARNING

Goal: Gaining background knowledge

Time spent: ~ 100 Hours

Implementation: 

  • Broad reading and research discussed with the team

  • Cross-applicable research presented to the team

STAGE 2: GENERATING IDEAS

Goal: Creating a list of ~200 intervention ideas in our target cause area

Time spent: ~ 50 Hours

Implementation: 

  • Generating ideas based on:

    • Contacting experts

    • Contacting key organizations

    • Broad reading and cross-applicable research

    • CE Team’s input

  • Put ideas into key categories

  • Remove any obviously low-quality ideas

STAGE 3: QUICK PRIORITIZATION

Goal: Narrowing down ideas list to 25% (~60) most promising interventions

Time spent: ~190 Hours

Implementation: 

  • Researching top considerations that might eliminate sets of ideas

  • Organizing ideas (e.g., merging similar ideas)

  • First independent 10-minute research rating (Researcher 1) 

  • Second independent 10-minute research rating (Researcher 2)

  • Additional research 

  • ​Review meeting and final decision-making

STAGE 4: SORTING IDEAS

Goal: Narrowing down ideas list to 25% (~15) most promising interventions

Time spent: ~130 Hours

Implementation: 

  • Assessing strength of evidence for the intervention (30 min. per idea)

    • Looking for evidence (e.g., RCTs, recommendations from effectiveness-focused organizations, macro/country-level data on effects, consensus among experts, existing effective implementations)

    • Assessing quality of evidence

    • Cutting 20% of least promising ideas

  • Identify implementability/execution difficulty and limiting factors (30 min. per idea)

  • Quantitative modelling (30 min. per idea)

    • CEA/BCR estimates 

  • Comparing to the benchmark interventions

STAGE 5: SHALLOW INVESTIGATIONS

Goal: Arriving at ~5-10 top ideas for the most complex research stage

Time spent: ~150 Hours
Implementation: 

  • 5-10 hour research reports on each intervention

  • Expert interviews

  • Identify and assess crucial considerations

  • Look for evidence against intervention

STAGE 6: DEEP DIVE

Goal: Selecting top 2-4 charity ideas that we recommend to be started through our 2022 Incubation Program and producing high-quality research reports.

Time spent: 600-800 Hours

Implementation: 

  • Background reading

  • Mapping a causal chain of the intervention and theories of change

  • Geographic assessment weighted factor model

  • Stakeholder mapping

  • Interviewing experts

  • Quality of evidence assessment (academic evidence, case studies, historical data, macro-level data trends, theoretical evidence, assessing the strength of evidence for each step in the causal chain)

  • Assessing the potential for a new charity (neglectedness, limiting factors, ease of implementability, scalability)

  • Cost-effectiveness analysis

  • Externalities/crucial considerations

  • Expert review

  • Final decision meeting

STAGE 7: IMPLEMENTATION REPORTS

Goal: Creating 2-3 implementation reports on the most promising interventions in order to get new charities up and running right after the program.

Time spent: ~80-120 Hours

Implementation: 

  • Additional research used to inform a successful start and implementation by Incubation Program alumni.