6.1.1 Anecdotal Bias

Anecdotal bias refers to the influence of anecdotal evidence over statistical evidence on people’s beliefs. Journalists, for instance, are well aware of the power of anecdotes. An analysis of approximately 29,000 New York Times editorials showed a reliance on anecdotes to drive arguments (Al Khatib et al., 2017). While some studies have concluded that statistics are more persuasive than anecdotes (e.g., Allen & Preiss, 1997; Hoeken, 2001; Hornikx, 2005) and others provided more cautious conclusions (Winterbottom et al., 2008), a number have found evidence for anecdotal bias (e.g., Reinard, 1988; Jaramillo et al., 2019; Ratcliff & Sun, 2020; Reinhart, 2006; Shen et al., 2015). Zebregs et al. (2015) suggest that this disparity in findings might be attributable to statistics affecting beliefs and attitudes, and anecdotes affecting intention. A more recent meta-analysis of 61 studies found that, overall, statistical evidence is more persuasive than anecdotal evidence (Freling et al., 2020). However, even if statistical evidence is more persuasive overall, anecdotes that add no additional information to co-presented statistics may still influence people’s judgement (Jaramillo et al., 2019). Further, the meta-analysis found that people tend to prefer anecdotal evidence over statistical data when the stakes are more emotional, medical, or relevant to the decision-maker. In business, decisions are clearly relevant to the decision-maker.

6.1.2 Anecdotal Bias in Business

It is important to investigate anecdotal bias in business because of its implications for managers’ use of case studies. There are many cases of managers successfully using analogies from anecdotal cases but also of failures to analogise correctly (Gavetti et al., 2005; Gavetti & Rivkin, 2005). There is very little research on anecdotal bias in business, but the existing work finds clear evidence of the effect. In fact, the recent meta-analysis by Freling et al. (2020) included the work of Wainberg et al. (2013) as one such paper.

Wainberg et al. (2013) gave a sample of managers and other professionals a choice between two audit firms, which varied in terms of their audit deficiencies for various clients. The experiment was designed in such a way that the statistical evidence favoured one firm, while the anecdotal evidence favoured the other firm. Participants were allocated to one of five conditions. Participants in the anecdotes only condition were given anecdotal examples of firm deficiencies, while those in the anecdotes & statistics condition were given the same anecdotal examples as well as the number of clients and deficiencies found. However, participants were not explicitly provided with the proportions of these values. Participants in the statistics only condition were given this proportions information as well as the number of clients without deficiencies but no detailed examples of deficiencies. The anecdotes & enhanced statistics condition included both anecdotes and the information in the statistics only condition. The terminology here is confusing because nothing about the way the statistics are presented to the participants is “enhanced” beyond how they are presented in the statistics only condition. However, the anecdotes & enhanced statistics—judgment orientation condition emphasised the importance of proportions and keeping absolute numbers in context.

Wainberg et al. (2013) measured the percentage of participants who chose firms favoured by the statistical data, finding evidence of anecdotal bias. Participants in the anecdotes only and anecdotes & statistics conditions equally chose the firm favoured by statistical data. However, participants in the anecdotes & enhanced statistics condition were less likely to choose this firm compared with those in the statistics only condition, even when the underlying proportions were made explicit. This shows evidence of anecdotal bias because participants ignored contradictory statistical data. The lack of difference between the anecdotes & statistics condition and the anecdotes only condition implies that the anecdotal bias effect was “complete”. That is, the presented statistics did not play a role in influencing participants’ choice of firm. A “partial” effect would have occurred if more participants in the anecdotes & statistics condition had chosen the firm compared with participants in the anecdotes only condition. This would have meant that statistics played at least some role in influencing choice.

The other important finding in this work is that anecdotal bias was reduced by highlighting relevant statistical features and providing an explanation of statistical inference. This is important because it suggests that potential psychological biases can be reduced with a reframing of provided information and an explanation of relevant statistical concepts.

Wainberg (2018) conducted a similar study to that of Wainberg et al. (2013) but with a capital budgeting task as opposed to a binary choice. Participants had to choose between three production line machines for a mid-sized company that prints circuit boards. The statistical data suggested that Machine A was better than Machine B, and Machine B was better than Machine C. Participants were given only statistical information or statistical information along with an anecdote. The anecdote was in the form of an email from a colleague who recommended against Machine A (the best option). Similar to Wainberg et al. (2013), participants were assigned to anecdote & statistics and statistics only conditions. In the judgement orientation I and judgement orientation II conditions, participants were told to “think like a scientist” and received either a short or a long explanation, respectively, of the importance of statistical inference.

Wainberg (2018) found evidence for anecdotal bias. Including a contradictory anecdote alongside statistical evidence (the anecdote & statistics condition) reduced the proportion of participants who chose Machine A. The study also found that the addition of instructions that emphasised scientific thinking reduced this bias. Unlike Wainberg et al. (2013), Wainberg (2018) could not determine whether the anecdotal bias was a complete or partial because there was no anecdote only condition. Further, neither work considered the effect of the anecdote’s similarity to the target problem.

6.1.3 Effect of Similarity

Arguably, the extent of one’s reliance on an anecdote should depend on its similarity to the target problem. Previous work has examined the importance of weighting previous cases according to their similarity to the present situation (Gilboa & Schmeidler, 1995; Lovallo et al., 2012). For instance, consider a medical treatment with contradictory statistical and anecdotal evidence; that is, a large-scale aggregated study has found that the treatment has 99% efficacy, while someone reports on social media that they became sick as a side-effect of the treatment. While the decision to use the treatment should be informed more by the aggregated data than by the anecdotal data, an individual may have reason to be concerned if the person who became sick was their identical twin. Therefore, the inference that the individual may also need to be cautious about the treatment arises from a specific causal model based on the shared genetics of the two cases.

There have been mixed results regarding the effect of anecdote similarity on the extent of anecdotal bias. Hoeken and Hustinx (2009 Study 3) found evidence for the effect of similarity on anecdotal bias for a variety of claims. As well as manipulating whether participants received a claim supported by anecdotal or statistical evidence, they manipulated whether the anecdotal evidence was similar or dissimilar to the claim that it was supporting. They found that similar anecdotes were more persuasive than dissimilar anecdotes. Using a student sample, Hoeken (2001) did not find evidence for the effect of similarity to a local government proposal. Similarly, Hornikx (2018) considered the effect of similarity on anecdotal bias in local government policy decision-making. The researchers did not find an effect for similarity or for anecdotes. However, they measured persuasiveness, and it may be that requiring participants to make more concrete decisions will create a more realistic scenario.

Apart from the need to determine the effect of similarity on the anecdotal bias effect, it is important to clarify how such an effect might work. Research on analogical reasoning has distinguished between simple surface similarity and deeper relational similarity (Gentner, 1983). As mentioned above, one’s use of an anecdote should depend on the extent to which it is associated by an underlying causal mechanism or mere surface similarity. Imagine a manager of a multi-divisional company deciding on the allocation of capital between an oil well project and a technology project. Would hearing of a recent failed oil well project at another company influence the manager’s allocation decision? If so, would it influence the manager’s decision because the anecdote has similarities to the target oil well project (surface similarity)? Or would the manager seek out the underlying reason for the failure of the other company’s oil well project to identify whether it is relevant to the target oil project (relational similarity)? The experiments presented in this chapter investigated whether the anecdotal bias effect arose from causal inductive reasoning or merely the surface similarity with the target project.

6.1.4 Experiment Summary

Experiment 1 investigated whether anecdotal bias in a capital allocation paradigm depends on anecdote similarity. Further, it tested whether providing additional statistical information encourages participants to consider the statistics over the anecdote. Experiment 1 used a negative anecdote, which is an example of an unsuccessful case. This kind of anecdote has been shown to produce anecdotal bias in both medical (Jaramillo et al., 2019) and business (Wainberg, 2018) decision-making. However, Jaramillo et al. (2019) found less bias in positive anecdotes, which are examples of successful cases, and Wainberg (2018) did not consider these at all. Therefore, Experiment 2 attempted to replicate the effect of similarity on anecdotal bias using a positive anecdote. Further, Experiment 2 provided participants with information about the sample distribution of the anecdote, whereas Experiment 1 did not. This allowed for an informal test of whether people are sensitive to such information.

6.2.1 Method

6.2.1.2 Materials

6.2.1.2.1 Instructions

All participants were first shown general instructions explaining the task. Subsequent instructions shown to participants depended on their experimental condition. Those in the anecdote only condition were told that they would be shown a case study of a failed project and an analysis of why it failed. Those in the statistics only condition were told that they would be shown NPV and reliability information for two focal projects. They were told that these values were sourced from a study with a large sample. Those in the anecdote & statistics condition were shown both of these instructions and were also told that the information in the anecdote had been included in the aggregated study data. Those in the anecdote & enhanced statistics condition were shown the same instructions as those in the anecdote & statistics condition, but were also provided with the explanation of scientific thinking used by Wainberg (2018). Appendix C.1.1.2.1 shows the instructions used in Experiment 1.

6.2.1.2.2 Allocation Task

In the allocation task, participants were asked to allocate a hypothetical budget to one of two projects from two different businesses. In this chapter, these projects are referred to as the focal projects, with one being the target project and the other the comparison project. The target project was used as the reference for the similarity manipulation. That is, the anecdote was either high or low in similarity to the target project. Further, the data analyses presented in Section 6.2.2 used allocations to the target project as the dependent variable. The comparison project was simply the other focal project to which participants were allocating. It was always a different type of project to both the target and anecdote projects.

Participants were presented with information about the name, location, integration (vertical or horizontal), and organisational structure (centralised or decentralised) of each business (see Appendix C.1.1.2.2 for an explanation of these terms). Further, they were presented with information about the features of each project that are typically available to managers prior to investment. Participants in the anecdote only condition were shown only this information (see Figure 6.1). Those in the anecdote & statistics, anecdote & enhanced statistics, and statistics only conditions were shown this information along with measures of NPV and overall reliability rating (see Figure 6.2). Participants entered their allocation data beneath this table in two text boxes labelled Project A allocation and Project B allocation, respectively.

Figure 6.1: Focal project display for the anecdote only condition in Experiment 1. Here, Project A was the target project and Project B was the comparison project.

Figure 6.2: Focal project display for the statistics only, anecdote & statistics, and anecdote & enhanced statistics conditions in Experiment 1. Here, Project A was the target project and Project B was the comparison project.

6.2.1.2.3 Anecdote

Participants who were presented with an anecdote (those in either the anecdote only, anecdote & statistics, or anecdote & enhanced statistics conditions) were shown a description of another business project and an accompanying analysis. Figures 6.3 and 6.4 show the anecdotes for those in the high and low similarity conditions, respectively. The project description had a similar layout to that of the focal projects. That is, it contained information about the business name, location, integration, and organisational structure of the business. It also detailed several predicted features of the project. Beneath this description was a paragraph presenting an analysis of why the project had failed. This paragraph referenced each of the features in the description to justify the failure of the project.

Participants in the high similarity condition were shown a description of a project from a business with the same type of investment as the target project (Project A). All categorical attributes were identical to those in Project A, but all numerical attributes were lower. The analysis explained that the numerical attributes had failed because they had not reached certain cut-offs. Critically, these cut-offs were all higher than the matching values in Project A. This was done to ensure that the numerical attributes in the anecdote appeared more relevant than those in Project A. For instance, in Project A, oil extraction was set at 2,200 L/hr, and in the anecdote it was 2,000 L/hr, while the cut-off was set at 3,000 L/hr. Thus, the failure of the anecdotal project arising from insufficient oil extraction would appear more relevant because the oil extraction in both the anecdotal project and Project A was lower than the cut-off value. Note, however, that there was uncertainty about the generalisability of these cut-off values because the participants did not receive an explicit indication of whether these values were meant to generalise to other cases.

Figure 6.3: Anecdote for participants in the high similarity condition in Experiment 1.

Figure 6.4: Anecdote for participants in the low similarity condition in Experiment 1.

6.2.1.2.4 Follow-up Questions

Participants who were shown the anecdote were subsequently presented with follow-up questions. They were asked about how similar they believed the anecdote was to the target project, how relevant it was to their allocations and how relevant it would be for their judgements about other projects of that type (see Appendix C.1.1.2.3).

6.2.2 Results

6.2.2.1 The Effect of Similarity on Anecdotal Bias

Anecdotal bias was tested by comparing the statistics only condition with both the high- and low-similarity anecdote and statistics conditions (see Figure 6.5). The omnibus one-way ANOVA of these three conditions was significant, \(F(2, 118) = 4.19\), \(p = .018\), \(\hat{\eta}^2_p = .066\). Planned comparisons show that participants in the statistics only condition allocated a higher percentage of their budget to the target project compared with participants in the high-similarity anecdote with statistics condition, \(\Delta M = -12.31\), 95% CI \([-21.53,~-3.09]\), \(t(118) = -2.64\), \(p = .009\); but not the low-similarity anecdote with statistics condition, \(\Delta M = -1.48\), 95% CI \([-10.75,~7.80]\), \(t(118) = -0.31\), \(p = .753\). These findings provide evidence of anecdotal bias in the high similarity condition only.

Figure 6.5: Mean allocation to the target project for the statistics only condition and the two anecdote & statistics conditions. Error bars represent 95% confidence intervals. Raw data are plotted in the background.

6.2.2.2 The Effect of Enhanced Statistics

The effect of enhanced statistics was investigated by testing the interaction of anecdote similarity and evidence type (anecdote & statistics and anecdote & enhanced statistics conditions, excluding the anecdote only and statistics only conditions). As shown in Figure 6.6, the two-way interaction was not significant, \(M = 3.89\), 95% CI \([-8.86,~16.65]\), \(t(238) = 0.60\), \(p = .548\). Further, the difference between the anecdote & statistics condition and the anecdote & enhanced statistics condition (averaged over similarity conditions) was also not significant, \(\Delta M = -0.12\), 95% CI \([-6.50,~6.26]\), \(t(238) = -0.04\), \(p = .971\). This suggests that providing participants with information about how to think statistically is not sufficient to facilitate a focus on statistics.

Figure 6.6: Mean allocation to the target project, by anecdote similarity and evidence type conditions (excluding the statistics only condition). Error bars represent 95% confidence intervals. Raw data are plotted in the background.

6.2.2.4 Relevance Ratings

Regression analyses were conducted to determine the relationship between allocations and the follow-up relevance ratings. As shown in Figure 6.7, the specific relevance ratings interacted with similarity condition, \(b = -2.84\), 95% CI \([-4.80, -0.87]\), \(t(240) = -2.85\), \(p = .005\). It appears that specific relevance ratings were related to allocations, but only in the high similarity condition. Further, there were no significant associations with the general relevance ratings. This suggests that participants applied reasoning to the connection between the anecdote and the target project as opposed to simply reacting to the failed project and associating that with that project’s industry.

Figure 6.7: Mean allocation to the target project, by specific relevance rating and similarity condition. LOESS method was used for smoothing over trials and the shading represents 95% confidence intervals. Raw data are plotted in the background.

6.2.3 Discussion

Hypothesis 6.1 was supported. Participants in the anecdote & statistics condition allocated less capital to the target project compared with those in the statistics only condition. However, this effect depended on anecdote similarity because this only occurred in the high similarity condition, not in the low similarity condition. Thus, while anecdotal bias was evident when the anecdote was similar to the target project, participants were not influenced when the causal mechanisms did not align. Contrary to Hypothesis 6.2, despite being influenced by the anecdote, participants still made some use of the statistics. This is different from the findings of Wainberg et al. (2013), who found no difference between the anecdote only and anecdote and statistics conditions, indicating a complete anecdotal bias effect. Hypothesis 6.3 was also not supported because the added enhanced statistical instructions used to encourage participants to use the statistical information did not reduce participants’ reliance on anecdotes.

Experiment 1 was limited because it only considered a negative anecdote; that is, a failed project. In real life, however, case studies often have a positive valence; that is, the story of a successful company. In fact, in business, it is possible that the anecdotes used are more likely to be positive because of survivorship bias. Jaramillo et al. (2019) found an anecdotal bias effect for negative but not positive anecdotes. This may be because the stimuli consisted of medical decisions and, in this domain, the loss of health may be more strongly noted than an equivalent gain in health. In Experiment 2 (discussed in the subsequent section) a positive anecdote was added to investigate whether anecdote valence would affect anecdotal bias.

It is unclear whether the effects found in Experiment 1 were related to participants’ perceptions of the type of sampling used to select the anecdotes. The instructions in Experiment 1 did not explain how the anecdote displayed to participants was chosen. Whether sampling is believed to be intentional or random has been shown to affect people’s decision-making (e.g., Hayes et al., 2019). In the present experiments, participants’ sampling assumptions may have changed the extent to which they used the anecdote in their decisions. For example, it may be rational to choose the anecdote over the aggregated data if (a) the anecdote was not sampled randomly from a pool of anecdotes, and (b) the anecdote had a greater similarity to the target project compared with other anecdotes in the pool in relevant ways. That is, if the anecdote were chosen because of its high relevance to the target project, it would be irrational to ignore it. In Experiment 1, it was unclear whether participants may have held these beliefs. To control for these assumptions, in Experiment 2, the instructions further clarified that the anecdote (a) was sampled randomly from a pool of anecdotes, and (b) was not significantly more similar to the target project than any of the other anecdotes in the pool.

6.3.1 Method

6.3.1.2 Materials

6.3.1.2.1 Instructions

Participants were shown similar instructions to those in Experiment 1 (see Section 6.2.1.2.1). The general instructions page included a test of the basic information expressed in the instructions. This test also functioned as an attention check. As in Experiment 1, participants were also shown instructions that were specific to their condition. These were shown on the same page as the rest of the project display, above the case study and focal projects. The instructions clarified that the anecdote had been randomly sampled and that all anecdotes in the pool were equally similar to the target project. Appendix C.2.1.2.1 shows the instructions used in Experiment 2.

6.3.1.2.2 Allocation Task

As in Experiment 1, the allocation task included a table describing the two focal projects and (apart from the statistics only condition) a description and analysis of an anecdote. Figures 6.8 and 6.9 show the anecdote and focal projects, respectively, for the negative valence, low similarity condition. Figures 6.10 and 6.11 show the anecdote and focal projects, respectively, for the positive valence, high similarity conditions. In the statistics only condition, participants were only shown the focal projects display. Appendix C.2.1.2.2 details the counterbalancing and randomisation used in the experiment.

Figure 6.8: An example of the anecdote display in the negative valence, low similarity condition of Experiment 2.

Figure 6.9: An example of the focal projects in the negative valence, low similarity condition of Experiment 2. Here, Project 1 was the target project and Project 2 was the comparison project.

Figure 6.10: An example of an anecdote display in the positive valence, high similarity condition of Experiment 2.

Figure 6.11: An example of the focal projects in the positive valence, high similarity condition of Experiment 2. Here, Project 2 was the target project and Project 1 was the comparison project.

6.3.1.2.3 Interstitial Page

Prior to the display, participants were shown an interstitial page, which was used to (a) introduce the display and (b) check the participant’s attention (given that no input was required, participants could easily skip the page without reading the text). See Appendix C.2.1.2.4.

6.3.1.2.4 Follow-up Questions

Participants were shown similar follow-up questions as in Experiment 1, except that in Experiment 2, rating scales were 1–7 instead of 1–6. See Appendix C.2.1.2.3 for a sample display of the follow-up questions.

6.3.2 Results

This section reports only the data relevant to the Experiment 2 hypotheses. See Appendix C.2.2 for manipulation check analyses and analyses of the follow-up rating data.

6.3.2.1 Overall Effect of Manipulations

As shown in Figure 6.12, the similarity \(\times\) valence \(\times\) evidence type interaction (excluding the statistics only condition) was not significant, \(F(1, 94) = 3.42\), \(p = .067\), \(\hat{\eta}^2_p = .035\). However, the similarity \(\times\) valence interaction was significant, \(F(1, 94) = 76.41\), \(p < .001\), \(\hat{\eta}^2_p = .448\), as was the evidence type \(\times\) valence interaction, \(F(1, 94) = 10.11\), \(p = .002\), \(\hat{\eta}^2_p = .097\). The analyses below elaborate on the specific hypothesised effects.

Figure 6.12: Mean allocation to the target project, by evidence type, similarity, and valence conditions. In mixed factorial designs, error bars cannot be used to make inferences by “eye” across all conditions. Therefore, error bars are not included. Raw data are plotted in the background.

6.3.2.2 Anecdotal Bias Depends on Anecdote Similarity

To investigate whether anecdotal bias depended on anecdote similarity, the differences in budget allocations between the statistics only condition and the two anecdote & statistics conditions (high and low similarity) were calculated. The values of the statistics only condition were different for each valence condition to create equivalent comparisons. For the negative valence condition, participants allocated more money the high-NPV project; while for the positive valence condition, participants allocated more money to the low-NPV project. As shown in Figure 6.13, the similarity \(\times\) valence interaction was significant, \(F(1, 47) = 30.66\), \(p < .001\), \(\hat{\eta}^2_p = .395\), as was the main effect of valence, \(F(1, 47) = 9.85\), \(p = .003\), \(\hat{\eta}^2_p = .173\). The main effect of similarity was not significant, \(F(1, 47) = 0.53\), \(p = .469\), \(\hat{\eta}^2_p = .011\).

The effect of the anecdote is represented differently for each valence condition. As such, the interaction was further analysed by comparing the two similarity conditions for each valence condition. For negative anecdotes, the statistical values (e.g., NPV) associated with the target project were higher than those for the comparison project. If participants were influenced by the negative anecdote they would therefore allocate less to the target. For negative anecdotes, a lower allocation to the target project is represented in Figure 6.13 as a positive value—the difference in allocation from when the participant did not see an anecdote. For positive anecdotes, the statistics were lower for the target project, so an influence of the anecdote is seen as a negative value in Figure 6.13. The hypothesised effect of negative anecdote similarity on anecdotal bias would suggest a higher difference score in high similarity than in low similarity. That is, more influence of the anecdote when it is similar than when it is dissimilar. For positive anecdotes a the hypothesised effect would suggest the reverse: a higher difference score in low similarity than in high similarity.

For negative anecdotes, the allocation difference was greater when the anecdote was similar to the target project than when it was dissimilar, \(\Delta M = -18.17\), 95% CI \([-26.17,~-10.17]\), \(t(93.80) = -4.51\), \(p < .001\). For positive anecdotes, the allocation difference was greater when the anecdote was dissimilar to the target project than when it was similar, \(\Delta M = 14.10\), 95% CI \([6.10,~22.11]\), \(t(93.80) = 3.50\), \(p = .001\). This provides evidence that anecdotal bias depends on anecdote similarity for both negative and positive anecdotes. Participants appeared to be sensitive to the relevance of the anecdote to the target problem.

Figure 6.13: Mean allocation difference between the statistics only condition and the anecdote & statistics condition, by similarity and valence conditions. The horizontal dashed line shows the point in which the two allocations were equivalent. Values above this line show the higher allocation to the target project when participants were shown statistics only compared with when they were shown statistics with an anecdote. Error bars represent 95% confidence intervals, calculated from the within-subjects standard errors using the method from Cousineau and O’Brien (2014). Raw data are plotted in the background.

6.3.2.4 Relevance Ratings

Regression analyses were conducted to determine the relationship between allocations and the follow-up relevance ratings. Figure 6.14 shows these data. While the specific relevance ratings for negative anecdotes showed the same trends as in Experiment 1, the interaction was not significant. Similarly, the ratings trends for positive anecdotes were as hypothesised, but their interaction not significant. It appears that specific relevance ratings were related to allocations, but only in the high similarity condition. Further, there were no significant associations with the general relevance ratings. This provides limited evidence that people were explicitly reasoning about the connection between the anecdote and target.

Figure 6.14: Mean allocation to the target project, by specific relevance rating, similarity condition, and valence condition. LOESS method was used for smoothing over trials and the shading represents 95% confidence intervals. Raw data are plotted in the background.

6.3.3 Discussion

Hypotheses 6.5 and 6.6 were supported because participants showed a stronger anecdotal bias effect when both positive and negative anecdotes had greater similarity to the target project. Further, as per Hypotheses 6.7 and 6.8, participants incorporated statistical information in their judgements, for both negative and positive anecdotes. Unlike in Experiment 1, the relevance rating data did not provide as clear indication that participants were using only the specific project information rather than merely its industry.

Therefore, Experiment 2 found that, unlike in the medical domain, the effect of anecdotes in financial decision-making does not depend on anecdote valence. Further, similar to the findings of Experiment 1, and unlike those of Wainberg et al. (2013), the anecdotal bias effect does not appear to be complete, with statistics still playing some role in participants’ decisions, despite the effect of the anecdote.

6.4.1 Theoretical Implications

The findings presented in this chapter add to the current understanding of the way in which people use different types of evidence in their decision-making. Previous research mostly investigated the relative influence of statistics and anecdotes by comparing anecdotal with statistical conditions. The current work shows that comparing a joint anecdote & statistics condition with both an anecdote only and statistics only condition enables a more specific investigation of participants’ anecdotal bias. The influence of anecdotes can be seen in the comparison of the statistics only and the anecdote & statistics conditions, while the effect of statistics can be seen in the comparison of the anecdote & statistics condition and the anecdote only condition. These two effects enable the determination of the independent influences of anecdote & statistics. Use of such a design in future research may help to further the understanding of conditions under which these types of evidence are used.

Some of the anecdotal bias literature is based on the assumption that using anecdotal evidence over statistical evidence is necessarily irrational. This is likely to have arisen from examples in the medical domain in which such decisions are indeed irrational (e.g., believing that vaccines cause certain disorders, despite the available evidence). In such cases, people over-rely on anecdotes and should be relying more on aggregated data. However, a case could be made for the rational use of an anecdote based on its similarity to the target problem. For instance, there are times when an anecdote is so similar to the target situation (e.g., the identical twin example discussed in Section 6.1.3) that it would be unwise not to consider it. That is, the use of anecdote should depend on both (a) the extent of underlying structural similarity to the target problem and (b) the distribution of this similarity across the pool from which the anecdote was sampled. People should use anecdotes if their casual structures are significantly more relevant compared with other cases in the available data.

However, similarity can also be misleading. For instance, if a case appears highly similar but differs in terms of a key hidden dimension that is the real causal mechanism, then using the anecdote may be the wrong thing to do. What appears to be important is being sensitive to relational rather than surface similarity. Future research should investigate how varying participants’ assumptions about sampling from a data set of anecdotes influences their anecdotal bias. Such assumptions can include the size of the sample, the shape of the distribution, and where in the distribution the anecdote came from. Prior work found that people are sensitive to distributional properties when generalizing (Carvalho et al., 2021), but it is not clear if this will replicate with descriptive cues such as in the experiments in this chapter.

6.4.2 Practical Implications

The current work contributes to managerial decision-making by providing insights into how managers make better decisions when using case studies and statistical information. Managers of large companies are often in a difficult position; they have incomplete information and are in an uncertain environment. Despite this, different biases and responses to those biases may be anticipated for different levels of uncertainty. For instance, a manager may be presented with both a convincing case study that suggests a certain course of action as well as aggregated data. The manager needs to be able to weigh the evidence accordingly.

The work in this chapter suggests that there are three elements to consider: (a) the quality of aggregated data (determined by factors such as sample size), (b) the relative similarity of the cases in the data pool to the target situation, and (c) the similarity of the anecdote to the target problem. For instance, an anecdote that is similar to the target situation in terms of relevance and is significantly more similar than other cases in the data set should carry more weight than an anecdote that comes from a pool of cases that are all equally similar to the target problem. Lovallo et al. (2012) found that similarity judgements increase prediction accuracy beyond a simple regression model. Taking into account a project’s relative similarity to other cases is likely to further increase predictive validity.

When aggregated data are not available, however, managers should rely more on anecdotes that have greater similarities in terms of causal structure. That is, they should be wary of merely using surface similarities to make inferences and instead consider the underlying relational structures. The present data suggest that laypeople can do this to some extent, with participants not being completely swayed by the mere similarity of type of business project. However, future research should investigate this further to better understand the boundaries of people’s analogical reasoning in capital allocation decisions.