
OUR RESEARCH PROCESS 2021
The focus of our research in the year 2021/2022 is finding the most cost-effective interventions in the area of international development policy.
OUR GOALS
Our primary goal is to identify the high-impact intervention opportunities.
Our secondary goals are:
Providing support to future incubatees through high-quality research reports and implementation advice.
Seeking additional wins (supporting other organizations, keeping track of high-impact interventions, identifying potential program participants).
Ongoing improvement of the research process.
OUR DECISION CRITERIA
We determine policy interventions to be particularly high-impact based on the following 4 targets (applied in priority order):
Substantially better than current top global health charities – e.g., 5x better than AMF (Satisficing target).
Variation – The overall set of opportunities identified should vary in type.
Evidence quality – Evidence should be high quality and robust to uncertainty; there should be minimal ways that a new charity could fail.
Maximally impactful – highest expected value (maximizing target).
THE PROCESS
Our research process comprises 7 stages, which were preceded by strategic process design. We timecap each stage to produce 3-6 high-quality recommendations that will be implemented through our yearly Incubation Program.
STAGE 0: PROCESS DESIGN
Goal: Designing the decision-making process for the international development policy cause area.
Time spent: ~150 Hours
Implementation:
-
Defining crucial parameters for decision-making
-
Research into available metrics
-
Creating templates for each step and method
-
Establishing a benchmark of recommendation
-
Test run
STAGE 1: TOPIC LEARNING
Goal: Gaining background knowledge
Time spent: ~ 100 Hours
Implementation:
-
Broad reading and research discussed with the team
-
Cross-applicable research presented to the team
STAGE 2: GENERATING IDEAS
Goal: Creating a list of ~200 intervention ideas in our target cause area
Time spent: ~ 50 Hours
Implementation:
-
Generating ideas based on:
-
Contacting experts
-
Contacting key organizations
-
Broad reading and cross-applicable research
-
CE Team’s input
-
-
Put ideas into key categories
-
Remove any obviously low-quality ideas
STAGE 3: QUICK PRIORITIZATION
Goal: Narrowing down ideas list to 25% (~60) most promising interventions
Time spent: ~190 Hours
Implementation:
-
Researching top considerations that might eliminate sets of ideas
-
Organizing ideas (e.g., merging similar ideas)
-
First independent 10-minute research rating (Researcher 1)
-
Second independent 10-minute research rating (Researcher 2)
-
Additional research
-
Review meeting and final decision-making
STAGE 4: SORTING IDEAS
Goal: Narrowing down ideas list to 25% (~15) most promising interventions
Time spent: ~130 Hours
Implementation:
-
Assessing strength of evidence for the intervention (30 min. per idea)
-
Looking for evidence (e.g., RCTs, recommendations from effectiveness-focused organizations, macro/country-level data on effects, consensus among experts, existing effective implementations)
-
Assessing quality of evidence
-
Cutting 20% of least promising ideas
-
-
Identify implementability/execution difficulty and limiting factors (30 min. per idea)
-
Quantitative modelling (30 min. per idea)
-
CEA/BCR estimates
-
-
Comparing to the benchmark interventions
STAGE 5: SHALLOW INVESTIGATIONS
Goal: Arriving at ~5-10 top ideas for the most complex research stage
Time spent: ~150 Hours
Implementation:
-
5-10 hour research reports on each intervention
-
Expert interviews
-
Identify and assess crucial considerations
-
Look for evidence against intervention
STAGE 6: DEEP DIVE
Goal: Selecting top 2-4 charity ideas that we recommend to be started through our 2022 Incubation Program and producing high-quality research reports.
Time spent: 600-800 Hours
Implementation:
-
Background reading
-
Mapping a causal chain of the intervention and theories of change
-
Geographic assessment weighted factor model
-
Stakeholder mapping
-
Interviewing experts
-
Quality of evidence assessment (academic evidence, case studies, historical data, macro-level data trends, theoretical evidence, assessing the strength of evidence for each step in the causal chain)
-
Assessing the potential for a new charity (neglectedness, limiting factors, ease of implementability, scalability)
-
Cost-effectiveness analysis
-
Externalities/crucial considerations
-
Expert review
-
Final decision meeting
STAGE 7: IMPLEMENTATION REPORTS
Goal: Creating 2-3 implementation reports on the most promising interventions in order to get new charities up and running right after the program.
Time spent: ~80-120 Hours
Implementation:
-
Additional research used to inform a successful start and implementation by Incubation Program alumni.