9 Evaluation-aversion and obstacles to considering charity impact
Connection to terms/concepts in other work
- Caviola, Schubert, and Nemirow (2020) (literature review), “Motivational obstacles”, “Prioritization aversion”; comes closest to our ‘taboo tradeoffs’
9.1 General relevance to effective giving
For people to choose one product over another product on the basis of some characteristic (e.g., safety, taste, or durability), they presumably must be aware of these differences before purchasing. Economists note the difficulty of investing in producing, marketing and selling high-quality products and services when consumers have difficulty distinguishing these products from inferior ones ( Dulleck and Kerschbamer (2006); Nelson (1970), Shapiro (1983)). This is especially difficult when the quality of a product is not known before purchase (an ‘experience good’ such as a ticket to a particular stage show, see ibid), or when it is not known until a long time later, if at all (a ‘credence good’ such as a health remedy or investment advice Dulleck and Kerschbamer (2006))
Considering a charitable donation as a ‘product purchased by the donor’, it seems to fall into the latter category of ‘credence good’. If donors value their ‘marginal impact on outcomes’ as discussed in our earlier definitions of impact, they may need to do extensive research (or at least know about and visit web sites such as GiveWell) to have some estimate of the value-for-money they are getting.1
A typical charitable donor, particularly one who donates towards a geographically-distant intervention, will never directly see or experience the consequences of her donation. Thus, for people to systematically choose to donate to the most effective charities, presumably…
They must understand and value the idea of effectiveness.
They must either:
- already know how effective charities are relative to one another,
- have reliable information on this presented to them by the charities (or other entities),
- or they must want to and find it appropriate to seek information on this, and be able to obtain reliable information.
- (For B or C to have the desired effect…) The act of learning about effectiveness must not substantially decrease their willingness to donate.
In this section we consider the evidence for
People’s aversion/willingness (or sense that it is appropriate/inappropriate) to evaluate effectiveness in a charitable giving context
and other obstacles tied to this.
9.2 General cost-benefit analysis (CBA)-aversion or reluctance
9.2.1 Description
People may be reluctant3 to consider the cost and benefits of the actions they are funding through their charitable donations (or they find this less appropriate/normal). This contrasts with a much greater willingness to consider these and other domains such as consumption, investment, and public policy. People also seem to avoid accessing/buying/seeing information (particularly information that may be likely to make them feel compelled to give.)
9.2.2 Conceptual discussion
For this to be considered a bias, the relevant individuals must intrinsically value the usefulness of their charitable activity at least to some extent.4 However, they may consider it very costly or distasteful to actually do this evaluation, or it may clash with other motivations and tendencies.
This aversion also must be distinguished from a lack of ability to do CBA in the charity domain; the latter would instead be considered a quantitative bias.
The reluctance to engage in this evaluation process may relate to the idea of “taboo trade-offs”; if these tradeoffs are taboo, considering them may involve great emotional distress.5
(Berman et al. 2018) refer to the idea that “believing that charity is a subjective decision licenses individuals to donate in personally gratifying ways.” This perspective plausibly combines partial and conflicted utililitarian preferences with the presence of moral licensing. As these authors note, the belief that CBA is not a natural part of the charitable domain may stem from the lack of direct feedback one gets from donating (in Economics terms, a “credence” good) relative to consumption and investment goods.6
Several papers (Null (2011), Fong and Oberholzer-Gee (2010), Metzger and Günther (2019)) find evidence that (arguably) suggests that people are often reluctant to pay for (and may even actively avoid) certain information about charitable costs and benefits. People are also known to ‘avoiding being asked’ in general (DellaVigna, List, and Malmendier (2012), Andreoni, Rao, and Trachtman (2017)). However, these may reflect motives distinct from CBA, such as a self-serving bias .7
CBA: Roots in Psychological theory
This ‘CBA discomfort’* can be connected to several (potentially overlapping) theoretical frameworks:
Fiske’s Relational Theory (Fiske (1992); also see Aggarwal (2004)), which proposes four basic types of social relationships: 1. communal sharing, 2. authority ranking, 3. equality matching, and 4. market pricing. For more on this, see Heyman and Ariely (2004) on social vs economic markets.
Taboo Tradeoffs and Protected Values: to the extent that CBA requires making taboo tradeoffs (e.g., choosing between saving one life and saving another) that clash with protected values, people may be reticent to engage in CBA for prosocial purposes.
‘Distorted Altruists’: People do care about welfare maximization, *but “without clear information to make comparisons* they rely on their feelings to guide their choice” Berman et al. (2018), citing (Loewenstein and Small (2007); and Slovic (2007)).8
If determining which charity is effective requires cost-benefit analysis (CBA), people may avoid doing so. If effective charities force people to consider CBA then people may avoid these charities in order to avoid having to do these evaluations.
Stated more broadly, effective giving is predicated on CBAs for programs and organizations. Donors must engage with these CBAs and carefully evaluating which sources are trustworthy and meaningful. To the extent that people are uncomfortable with doing this in the charitable domain, it will be difficult for them to give effectively.
9.3 Evidence: attitudes towards CBA in the charitable domain, CBA-avoidance
Evidence for claim: “People sometimes actively avoid information about charity effectiveness…”
Tangentially related: many people tend to ‘avoid being asked or pressured to donate’ (DellaVigna, List, and Malmendier (2012), Andreoni, Rao, and Trachtman (2017)).
(DellaVigna, List, and Malmendier 2012) provide field experimental evidence that people will pay costs to avoid being asked and avoid social pressure. However, for this same case if they are asked they then to respond by giving. While “avoiding the ask” is not avoiding cost-benefit analysis, it suggests that people are in fact strategic in avoiding things that may make them feel compelled to donate.
Andreoni, Rao, and Trachtman (2017) find a related result in a distinct field setting.
Also somewhat tangentially, Dana, Weber, and Kuang (2007) find, in a modified laboratory dictator game, that about 40% of subjects choose not to learn (at no cost) about the payoffs of the recipient subjects, in a context where this affords them ‘moral wiggle room’ not to donate.
Evidence for: “People rarely seek out effectiveness information and are reluctant to purchase it”
Null (2011)
In the final stage of an experiment from Null (2011)…
Subjects were given the option to spend USD 5 of their total gift to the development charities in order to find out which of the three would receive a matching rate of USD 3 (the other two would receive matching rates of USD 1.50). Altruistic subjects whose donation was at least USD 20 and gave to all three charities, or whose total gift is greater than USD 35 and gave to two charities, would find it profitable to purchase the information. … (84%) met these criteria on gift size and number of charities supported.
…only 40% of subjects were willing to give up a small portion of their endowments in order to find out which charity would receive the highest rate; the rest preferred to allocate their gifts without knowing what they would be worth to the charities.”
…These subjects who chose not to purchase the information forfeited matching funds ranging from 30-150% of the value of their unmatched gifts, with the median donor sacrificing matching funds exactly equal to the value of her unmatched gift, a truly staggering sum.
Null attributes this failure to buy information either to subjects who “simply did not care about the potential to substitute into the charity with the highest matching rate”, perhaps driven by some form of simplistic warm glow motive, or to simple misunderstanding or fatigue (in an incentivized elicitation, she found some evidence of incomplete comprehension). To the extent this is not a misunderstanding, it might be seen as evidence of CBA aversion; participants did not want to purchase evidence that would require them to do calculations in this domain.’9
Null’s result is interesting but it involves unique, somewhat unusual, and somewhat artificial contexts. The participants were aware that they were in an experiment.
Other papers involving laboratory ‘real charitable giving’ experiments also find a relative lack of willingness to pay for effectiveness-related information.
Metzger and Günther (2019)
Metzger and Günther (2019) run a laboratory experiment, where participants have the opportunity to donate (from their lab income) under
one of three information settings: aid impact, recipient type, or administrative costs… Within each information setting, participants were randomly assigned to a “high-” or “low-performing” NGO and randomly assigned to the control and treatment groups. Participants in the treatment group had the opportunity to buy detailed information about whether they were matched with the high- or low-performing NGO.
In the most relevant ‘aid impact’ setting, the difference in impact was explained as
Depending on which project, the NGO will make the following impact:
Project 1 will employ an additional teacher, which would, for every franc donated, enable one student to remain in school for 1 week longer.
Project 2 will provide a meal to students at school, which would, for every franc donated, enable one student to remain in school for 1.5 weeks longer.
Project 2 (providing a meal at school) is 50% more effective than project 1 (financing an additional teacher). The efficiency of an additional teacher and the efficiency of providing a meal at school has been assessed in several scientific studies.
This does indeed, seem like the sort of ‘real quantitative impact information’ we are considering in this context, although the very sparse description might not make this as credible as we would like.10
Overall, they find a relative lack of interest in purchasing this information.
Our main results are the following. First, while only 28.73% (se = 0.03) of the participants in the treatment group buy the offered information, 57.09% (se = 0.02) donate to charity. That means, about 28% of the participants make an uninformed donation. Second, demand varies considerably across information types. Demand for information on aid impact is the lowest with 22.33% (se = 0.04) and highest for information on the recipient type (37.81%, se = 0.05). …
Last, we find that non-buyers do not respond to information: they hardly change their donation behavior when they receive information for free. This suggests that individuals, who decide against acquiring additional information do not understand, do not trust, or do not care about the information provided in this laboratory experiment.
Does this demonstrate ‘participants did not buy information because they don’t care about effectiveness?’ (Discussion in fold)
The average donation of ‘aid non-buyers’ was about 2.5 Swiss franks (CHF). (Figure 6). The information costs 0.3 (CHF) If the information is reliable, it could increase the value of the actoin by 25% on average; the 50% more efficient charity is assigned half the time. If the individual prefers to donate only to the more efficient charity, the relative value might be worth about .625 CHF total, or 0.325 more than its cost. Perhaps
Note this may be an underestimate; this ‘average donation’ already reflects uncertainty about the value of the donation.
(Note: I would need to think about this decision problem in more detail; this is an attempt at an order of magnitude.)
The comparison to the purchases of information in the other context also seems overstated. The ‘information on the recipient type’ may itself be seen as (and may actually be, in this case) an important measure of impact. The ‘recipient type’ determines whether the charity ‘focuses on promoting the education of children’ versus whether it ‘focuses on promoting the education of young artists’. As the authors admit, participants may believe “that donating to children will generally have a larger development impact than donating to artists, considering the recipient type as a proxy for the impact of a donation”.11
Fong and Oberholzer-Gee (2010) also finds some loosely related evidence (see fold).
Fong and Oberholzer-Gee (2010) run dictator game experiments involving payments to real-life welfare recipients living in 178 public housing in Pittsburgh; each subject is matched with a particular recipient of their potential donation. In their “Choice treatment”, a subject can choose to pay $1 to learn about a recipient’s drug use or disability, information meant to suggest the deservingness of the recipient. “We find that a third of the dictators are willing to pay money to learn more about their recipient. Dictators who acquire information mostly use it to withhold resources from less-preferred types, leading to a drastic decline in aggregate transfers.”
But this needs to be interpreted carefully: those who decide not to buy information appear less generous than the average!
9.3.1 Evidence for “people accept and value subjectivity in the charitable domain more so than for other choice domains”
(Berman et al. 2018) provide evidence from a series of five survey/vignette experiments; unlike those mentioned above, these (mostly) involve hypothetical choices among multiple causes. All experiments use standard subject pools (behavioral lab subjects or Mturkers) with reasonably large samples. All ask for hypothetical (Likert-scale) responses involving fictional charities, investments, and other scenarios; they mostly rely on between-subject responses, and their statistical analyses report reasonable tests on the relevant comparisons.
Study 1 showed that, relative to personal decisions, individuals view charity as being relatively subjective. Study 2 showed that individuals prioritize emotional connection over welfare maximization when comparing charity options. Study 3 examined choice processes and found that people are more likely to override welfare maximization when choosing a charity than when choosing a financial investment. The final two studies found that individuals are less likely to license themselves (Study 4) and others (Study 5) to select an ineffective option when a decision maker assumes a position of responsibility.
Their “Study 1: Perceived Subjectivity of Charity” found that, in rating statements such as
“it is important that the ______ I choose reflects my personal tastes or values”
and
“It is more important to rely on objective measures rather than personal feelings when choosing ______” …
they found people agreed more with the subjective/taste approach when assigned a treatment where the blank was “Charity”, relative to those assigned treatments involving medical treatments, investments, and cell phones. (But less than some other things like art, and similar to restaurants in some tests!)
Participants were asked to evaluate how they believe they should make decisions across six decision domains (choosing a charity, an investment, a cell phone, a restaurant, a piece of art, and a medical treatment). Specifically, they evaluated the extent to which they agreed with the following statements: “It is important that the _____ I choose reflects my personal tastes or values,” “It is more important to rely on objective measures rather than personal feelings when choosing _____,” and “Objective measures are the best way to choose _____.” Responses were made on a scale from 1 (not at all) to 7 (very much so).
Their “Study 2: Personal Feelings Versus Welfare Gains” presented participants with “Mary” and a pairing of fictional domestic (homelessness) and international (micronutrient) charities, presenting effectiveness information on both (clearly favoring the latter).
The treatment—changing ‘which charity Mary felt an emotional connection to’—had a significant impact on the response to “Which charity should Mary donate to”, in the predicted direction.
They were also asked: “Which option does the greatest good for the greatest number of people?”; here responses favored the international charity for both treatments; but even so, when Mary felt connected to local charity, participants favored her donating there.
(Unfold for the authors’ description of this methodology and results…)
Participants read a scenario involving Mary, who was looking to donate money to one of two hypothetical charities. One charity, Hunger Care in Africa, was described as providing micronutrients for children in Africa, and for each USD 100 donated, this charity could feed five children for a month. A second charity, Jump Start Your Community, was described as training and educating the homeless in Mary’s community, and for every USD 100 donated, this charity could provide 2 hr of job training to one homeless person
Each participant was then randomly assigned to one of two emotional-connection conditions. In the local-causes condition, participants read that Mary feels an emotional connection toward helping people in local communities, whereas in the distant-causes condition, participants read that Mary feels an emotional connection toward helping those in third-world countries
We then presented participants with two different questions on separate pages in random order. One question assessed where they felt Mary should donate (“Which charity should Mary donate to?”), whereas the other question asked which charity was the most effective at maximizing social welfare (“Suppose that Mary wanted to donate to the charity that did the greatest good for the greatest number of people. Which one should she donate to?”). Both questions were measured on a 7-point scale from 1 (definitely Jump Start Your Community) to 7 (definitely Hunger Care in Africa)
Berman et al. (2018) summarized their results from this study as follows:
When participants read that Mary felt an emotional connection with distant charities, they responded that she should donate to Hunger Care in Africa… and also evaluated it as being more effective… However, when Mary felt connected to local communities, they indicated that she should donate to Jump Start Your Community…, despite indicating that Hunger Care in Africa was more effective
In their “Study 3: Charity Versus Investment Choice”, subjects were assigned categories and fictional examples of either charities or investment opportunities, and presented domain categories and effectiveness information for each. Fewer participants in the charity treatment (relative to the investment treatment) chose to sort by effectiveness rating, and fewer chose the highest-rated option.
(Unfold for further discussion of study 3…)
Study 3 built on the ideas from Study 1. Effective altruism suggests that to make the biggest difference, people should try to maximize welfare, analogously investors seeking maximize their financial returns. However, Study 1 suggested that individuals viewed objective information as more important in making financial decisions than in charitable decisions. The third study from Berman et al. (2018) tries to more directly compare the two, using the following method:
Participants were also told that the website organizes options into six different categories (alternative energy, cancer, education, food, housing, and international). These categories were chosen because they could plausibly represent groupings of either charities or investments.
Participants were then given the choice to sort the options either by effectiveness or by category type. Sorting by effectiveness suggested that an individual was primarily concerned with selecting one of the most effective charities or investments, whereas sorting by category suggested that an individual was primarily concerned with selecting a charity or investment that reflected a preferred option type
They found that participants were significantly more likely to sort by effectiveness in the investment condition than in the charity condition. Furthermore, when the options were ranked, “participants chose a less effective charity… than investment”.
Study 4 and 5 both examined how responsibility changes people’s perception of how important objective information is. Berman et al. (2018) hypothesized that “when individuals assume a role of responsibility, they may feel obligated to act in accordance with the welfare of the entire group and will discount their personal preferences to do so”.
In Study 4, they find that, in rating research departments for funding, participants pay more attention to charity effectiveness ratings when the are given the “role” of a “president of a local medical research center” rather than a donor. Similarly, in Study 5 participants assess someone who allocates funds to a research department; participants respond to the effectiveness of the department chosen more when rating the decision quality and altruism/selfishness of a “president…” than rating a “donor”.
(Unfold for further details of study 4…)
In Study 4, participants were assigned to a donor condition or a president condition, and then participants were either given information on effectiveness ratings, or there was no mention of effectiveness ratings.
Participants assigned to the donor condition imagined that they lived near a local research center. Participants assigned to the president condition imagined that they were the president of a local medical research center. Participants in the donor condition were then told that “because the market has done well, you have more cash than usual. You would like to help by making a donation to medical research.” Participants in the president condition were told that “Because the stock market has done well, the research center has more cash than usual. As president, it is part of your job to allocate these funds to conduct additional medical research.” Each participant was then given his or her choice between allocating funds to one of three departments, with effectiveness ratings provided for each department… These ratings were purposefully selected so that the most intuitively appealing choice was rated as the least effective (cancer), and the least appealing choice was rated as the most effective (arthritis), thereby creating a tension between the option that would be most personally gratifying and the option that would maximize welfare
Their results suggested that people paid more attention to the effectiveness ratings when they were in a position of responsibility:
When the effectiveness ratings were absent, the assigned role did not significantly affect the option that participants chose… However, when the ratings were present, participants in the president condition were significantly more likely to select higher rated options than those in the donor condition… Although participants used effectiveness ratings when given this information, significantly more did so when placed in a position of responsibility
Their fifth study used a similar methodology to Study 4, to examine “how individuals judge others who select either a relatively effective or an ineffective option”, using the following method:
The study has a 2 (role: donor vs. medical center president) x 2 (choice: most effective option vs. least effective option) between-subjects design. Participants read a scenario about Mr. Peterson, an individual who was allocating money to a local medical research center. In the donor condition, participants read that Mr. Peterson was looking to donate money from recent stock market gains to a department within the medical center. In the president condition, participants read that Mr. Peterson was the president of the medical research center and that he was looking to allocate surplus funds from recent stock market gains to a department within the medical center In all conditions, participants then read about an independent research firm that evaluates the effectiveness of departments in the same fashion as the previous study … Participants then evaluated the extent to which they thought the decision was responsible, appropriate, and thoughtful
As with Study 4, the results suggested that people thought effectiveness information was more important for people in positions of responsibility:
In both the president and the donor conditions, participants felt that Mr. Peterson’s decision was of lower quality when he selected the least effective option over the most effective option. However, the drop in decision quality was significantly greater when he was president… than when he was a donor In both the president and the donor conditions, participants felt that Mr. Peterson’s decision was less altruistic when he selected the least effective option over the most effective option. However, this difference was greater when Mr. Peterson was president… than when he was a donor
Overall, these suggest that, when considering charitable donations, people tend to favor–or at least to accept–the use of subjective preferences and personal ties, rather than objective information, and they do so more than for more “standard” goods and choices. This is more accepted for “donors” than for people with responsibility for others’ funds.
In their analysis of the studies, Berman et al. (2018) concluded with the following:
Our results suggest that people view charity decisions as being relatively subjective, which inhibits the impact of effectiveness information on welfare maximization. Thus, to persuade people to make donation decisions that maximize social welfare, providing information alone may not be sufficient. Rather, it may require altering how individuals view their role as a donor altogether14
However (as they do note), the effectiveness information still has some (positive) effect on participants’ responses; it is not ignored. Their experiments also do not analyze the avoidance of information or CBA.
Methodological strengths and weaknesses:
Hypothetical nature of choices (a limitation)
Some evidence suggestng these are not taken seriously
Specific context in vignettes allow alternative interpretations…
9.4 Information as an ‘excuse’ not to give; information may allow motivated reasoning
This topic straddles the material in this section and the next section, and the papers presented here inform both of these
Fong and Oberholzer-Gee (2010)
“Dictators [charitable giving] who acquire information mostly use it to withhold resources from less-preferred types, leading to a drastic decline in aggregate transfers”
*Limitations of Fong and Oberholzer-Gee (2010)
- Selection effects
- In their tables, exogenous provision of information seems to increase donations overall.
Exley (2016): Finds greater discounting of ‘less-efficient’ charity in charity-charity decision-making than in charity-self decision-making.
Limitations of Exley study:
Experimenter demand (M-turk focus), not really ‘impact’ information
She considers evidence on the deservingness of the recipients, not on the impact of a charity itself.
Donors may also be uncertain about other benefits they may gain from each donation such as ‘gratitude and long-term sense of fulfillment’.↩︎
Caveat: This is hard to label: ‘aversion’ may be the wrong word: people may finding it less appropriate/normal/virtuous to do CBA in a charitable context, or it may naturally not occur to them to do it.↩︎
To clarify: How is this ‘reluctance’ observed/manifested?↩︎
A Utilitarian would certainly do this. Jason Schukraft notes that “one can reject utilitarianism and still consistently believe that, all else equal, saving 10 lives is 10x better than saving 1 life.”↩︎
Todo: get a reference for this term.↩︎
(they cite: Imas paper?)↩︎
On the other hand … a majority ranked effectiveness [how highly?] as a crucial criterion to select a charity and reported greater happiness when the impact of their contribution was highlighted. (Aknin et al. 2013 ; van Iwaarden et al. 2009)↩︎
We distinguish opposition to CBA from the inability to conduct CBA. The former is treated here while the latter involves a series of quantitative biases discussed later. (To do: Clarify the third bullet point on ‘distorted altruists’. This seems to be the dominant view in the literature.↩︎
In a related experiment, Null (2011) also concludes that people do not respond efficiently to adjustments to the charity matching rates. We return to this below.↩︎
The participant might wonder ’is this a particular framing of the impact for experimental purposes? Can I really trust this information? If one program is known to be more effective than the other, why are both still being run? Still, many of those concerns might indeed come up in the relevant real-world settings of interest,↩︎
Metzger and Günther (2019) also considers the impact of providing information on the donation amounts. We will return to this in the next section, on the “Effect of analytical (effectiveness) information on generosity”↩︎
Berman et al. (2018) also point to Goodwin and Darley (2008) for an examination of ‘objectivism’ generally, and Spiller and Belogolova (2017) for their discussion of objectivity in consumer decisions. ↩︎
Somewhat puzzlingly, Mary’s connection to the charity also affected the stated “effectiveness” response! This may bear a closer examination. Is there a meaningful effect here, or does this suggest a confound? ↩︎
Berman et al argue that their results demonstrate the acceptance of the suggestive preferences is somewhat attenuated by the “role of responsibility”, but it’s not clear what this term means or how this could be relevant to voluntary individual giving.↩︎