“Without reflection, we go blindly on our way, creating more unintended consequences, and failing to achieve anything useful.”  Margaret J. Wheatley What are flowthrough effects? Your charity will have a direct impact, but it will also have many indirect flowthrough effects, also known as secondorder effects. For example, an organization distributing bednets might cause a reduction in malaria, but its secondorder effect might be increased economic growth due to less loss of productivity from disease. One direct effect can set off an infinite chain of second, third and fourthorder effects, which might even surpass the direct impact of a charity. However, flowthrough effects aren’t always positive. That’s why it’s so important to consider them beforehand, to minimise your chances of encountering a nasty surprise further down the line. For example, while a charity that gives out bednets to prevent malaria likely has very valuable shortterm effects, could it be creating harmful overpopulation in the mediumterm? Or could it be creating a net population loss, because richer people have fewer children? How to account for flowthrough effects What cause area and intervention you pick will determine how much research into flowthrough effects will be necessary. We strongly recommend trying to identify and understand as many short, medium and longterm effects as you can before implementing that intervention. Calculating direct effects is hard enough without taking second order effects into account. Good giving opportunities are scarce, good science is difficult, errors can go uncorrected for long periods of time, and it’s hard to even figure out which metric to focus on. It’s even more challenging to predict medium and longterm effects, which might not emerge for another 50 or 100 years. While flowthrough effects will still require significant analysis, it is important to make sure they do not paralyze your decisions. We think weighted quantitative modeling is usually the strongest strategy for incorporating flowthrough effects into impact predictions because it strikes the right balance between time and the ability to make progress in a generally intractable area. However, there is large disagreement in this area and many ways to deal with the issue. Below is a list of the different methods along with their advantages and disadvantages. Ignore flowthrough effects The simplest strategy, and possibly the most common one in the charity sector, is to ignore potential flowthrough effects entirely. Some charities believe flowthrough effects are too complicated to include in their impact calculations. Some take positive shortterm effects as a proxy for good medium and longterm effects. Some underestimate the magnitude, importance or likelihood of flowthrough effects. We don’t think burying your head in the sand is a productive solution, however. If you neglect to consider indirect impact, you are very likely to miss large, clear, and important effects that will change your expected cumulative impact. Make an educated guess In the absence of certainty, you might do well to incorporate a best ‘guestimate’ of a particular flowthrough effect in your impact calculation. You can adjust your impact estimate accordingly, perhaps using a “many weak arguments” framework. Even if each individual argument is weak, if there are a sufficient number of them all supporting the same conclusion, that can be more reliable than one very strong argument supporting a different one. Making an educated guess, based on the limited information you have available, will at least give you a ballpark figure to work with. Create a weighted quantitative model You could create a weighted model based on your confidence in the direction (positive vs negative) and magnitude of each potential flowthrough effect. For example, secondorder effects (which are harder to measure) would usually have less weight compared to firstorder effects (which are easier to measure), because they are less predictable. This is similar to having a bayesian prior which is described elsewhere. If you were only 60% sure of the size of an intervention’s first order effects, you would give the figure a 60% weighting in your calculations, to account for that uncertainty. Similarly, if you were even less confident about the size of the secondorder effects, with only 30% confidence in your guess, you would give the secondorder effects a weighting of just 30% within your calculations. This model prevents very speculative flowthrough effects from impacting your calculations too much; the lower your confidence in a figure, the weaker its effect on your end result. Thus, fourthorder effects would have a far greater impact on standard expected value calculations than in a weighted quantitative model. Generally, for expected value calculations (if the uncertainty is equal on both sides which is often assumed) there is no discount applied for weakness of evidence or uncertainty of the flow through effect. Instead, the probability you put on a value is taken at face value. This is strange, because putting a 50% chance on flipping a coin and getting heads is very different from putting a 50% chance that your partner will say yes if you propose to them. In the former case, if you flip the coin a thousand times, you can be very confident you will get heads approximately five hundred times. However, you will (hopefully) not propose to your partner one thousand times. You might say that you put it in the same category as a multitude of other things that you put a 50% chance on, of which 50% will be accurate. That does not solve the problem though, because it is well established that people are very poorly calibrated, and if they put a 50% chance on something, that is usually not the actual probability that it will happen. You can improve your calibration skills by practicing, which we recommend, yet there’s quite high odds that calibration is domain specific. Practicing estimating distances will not crossapply to estimating the release of a new technology. Additionally, some areas might be impossible to calibrate on because the feedback loops are too slow, such as making predictions one hundred years in the future. A weighted quantitative model could be described as an expected value calculation that takes into account more factors than naive ones normally do, specifically strength of evidence or your metaconfidence in the estimate. In other words, the weighted quantitative model is more weighted towards ‘no effect’ than are typical models. As a result, the more uncertain you are in an effect (even if your best guess is it's the same effect), the more you regress the effect towards ‘no effect’. Conduct indepth research You could list all the potential flowthrough effects and try to research them as much as is feasible. Some questions may be much harder to research than others. You may, however, be able to examine historical analyses of similar situations, or metaresearch into the ways others have dealt with flowthrough effects in the past. Some researchers have offered proxy variables you could attempt to measure and connect causally with various interventions. Choose robust interventions Choosing an intervention that you’re confident has net positive consequences under many different conditions would minimize the margin for negative flowthrough effects. For example, it seems very unlikely that improving education in developed countries, promoting international cooperation, or advocating for philosophical thoughtfulness will be net negative, even if they do have a sizable risk of being net neutral. 9 E.g. many developed world education initiatives have reduced risk of negative flowthrough effects, yet end up producing little to no results. Areas with large potential flowthrough effects This is by no means an exhaustive list. Rather, this is a list of the indirect effects that have been discussed most commonly in the effective charity community. Flowthrough effects and compounding It has been argued that the effects of doing good now compound. For instance, if you inspire one person to donating to effective charities, that person will continue inspiring new people, and those people will inspire more people in a chain reaction, thus “earning interest on your interest”. We believe this is an oversimplification. For one, say you start a direct poverty charity and that inspires approximately one new charity per year, and those charities have the same “inspiration rate”. This won’t go on forever until everybody in the world is starting direct poverty charities. It’s not an exponential curve, but rather an sshaped one. There are the initial low hanging fruit, exponential growth for a period of time, then a diminishing amount. However, this isn’t the end of the story. After all of these charities start, they don’t last forever. People retire, charities shut down, problems are solved, etc. So really after the tapering, there is a probably relatively linear comedown. Additionally, compounding benefits apply to doing good later as well. It’s not like if you start a charity 10 years from now, nobody will care anymore. However, there is still a penalty for starting later. If you spent 39 years researching, then spent 1 year doing, you’d only have 1 year of inspiring, so only one extra charity started because of you, rather than if you had spent 1 year researching and 39 years doing, at which point you’d have far more charities inspired by you. Furthermore, this is probably an overly optimistic scenario. There are many more ways to deviate from doing good through doing rather than giving. If you are earning to give and you inspire another person to earn to give, if they donate to the same charity or set of charities as you, it’s easy to see how much good they are doing by your standards. Starting a charity or working for another is much more complicated because of the diversity of options. Depending on how pluralistic your values and epistemics are, inspiring others is more or less good. How to deal with low probability but highly harmful flowthrough effects This lesson will discuss what to do if there is a chance that your intervention may have a very harmful flowthrough effect, but your research suggest that this is unlikely. Perhaps there have been studies rejecting the hypothesis, but the studies still have some room for error. To illustrate our point, we will use the example of iron supplementation, an intervention that may possibly increase the risk of contracting malaria. Lisa wants to start an iron supplementation charity and her research found that metaanalyses drawing on over 20 RCTs (Shankar, 2000; Gera, 2002) found no significant effect on malaria. Therefore, while Lisa should not ignore this potential flowthrough effect, she should discount its weighting in her calculations according to the strength of the evidence. There is also strong evidence suggesting that iron and folic acid fortification has some positive flowthrough effects, such as depression reduction (Beard, et. al., 2008; Ballin, et. al., 1992; Rahn, et. al., 2008; Luca, et. al., 2008). It is generally a good idea to err on the side of caution, and use pessimistic figures in costeffectiveness estimates. If, after comparing the costeffectiveness of implementing the intervention in different countries, Lisa finds that the top few locations happen to be malariafree zones, that’s fantastic. If there are several malariafree zones and several malariaprone zones with similar costeffectiveness estimates, then it makes sense to avoid highmalaria areas, in order to minimize the risk of negative flowthrough effects. If, however, the locations in which the program would be most costeffective happen to be malariaprone, and implementing in malariafree locations would be much less costeffective, then she will have to think seriously about whether this tradeoff might be worth the risk. Thinking deeper about this topic Lisa would have to consider that among countries that are malariaprone, some have better measures in place to control the disease than others (e.g. a more thorough bednet distribution program.) Having said that, no country’s preventative measures have 100% coverage. After comparing the best options that are both malaria risks and malaria free she would have to estimate which one would have a better net effect on the world.
0 Comments
Leave a Reply. 
Categories
All
Archives
May 2020
