Identifying Confirmatory Bias in the Field: Evidence from a Poll of Experts (with Rodney Andrews and Trevon Logan, under review)
NBER Working Paper no. 18064
Laboratory experiments have established the existence of cognitive biases, but their explanatory power in real-world economic settings has been difficult to measure. We estimate the extent of a cognitive bias, confirmatory bias, among experts in a real-world environment. In the Associated Press Top 25 College Football Poll expert pollsters are tasked with assessing team quality, and their beliefs are treated week-to-week with game results that serve as signals about an individual team's quality. We exploit the variation provided by actual game results relative to market expectations to develop a novel regression-discontinuity approach to identify confirmatory bias in this real-world setting. We construct a unique personally-assembled dataset that matches more than twenty years of individual game characteristics to poll results and betting market information, and show that teams that slightly exceed and barely miss market expectations are exchangeable. The likelihood of winning the game, the average number of points scored by teams and their opponents, and even the average week of the season are no different between teams that slightly exceed and barely miss market expectations. Pollsters, however, significantly upgrade their beliefs about a team's quality when a team slightly exceeds market expectations. The effects are sizeable-- nearly half of the voters in the poll rank a team one slot higher when they slightly exceed market expectations; one-fifth of the standard deviation in poll points in a given week can be attributed to confirmatory bias. This type of updating suggests that even when informed agents make repeated decisions they may act in a manner which is consistent with confirmatory bias.
Does the Hot Hand Drive the Market? Evidence from Betting Markets (with Trevon Logan, revise and resubmit)
This paper investigates how market makers respond to behavioral strategies and the implications of these responses for market efficiency. In particular, we ask whether market makers rationally price out certain strategies at the expense of leaving other strategies profitable, resulting in potential market inefficiency. We answer this question by examining betting market outcomes in an amateur sport, American college football, using data from over 11,000 games from 1985 to 2003. We find that favorites are statistically overpriced while home teams are statistically underpriced. We show that the magnitude of this bias is large enough to generate both economic and statistical inefficiency in this betting market. Furthermore, we provide suggestive evidence for the cause of this inefficiency: betting houses deliberately inflate the betting lines in order to account for previous strong performance against the spread, or the ``hot hand.'' Although eliminating the ``hot hand'' is rational for a betting house since betting on ``hot" teams is a popular betting strategy, tempering the ``hot hand'' results in consistently profitable simple betting strategies.
How Do Experts Use Bayes' Rule? Lessons from an Incentive-Free Environment (under review)
Experts are regularly relied upon to provide opinions in financial markets, but the expert's incentives to truthfully reveal his opinions depend on the environment. I consider how experts report their beliefs in an environment devoid of incentives. Using data from an incentive-free environment-the Associated Press Top 25 Poll for college football-I find that experts reassess data selectively, depending on their prior hypotheses over time. I also find that experts do not fully incorporate all of the information contained in a signal for a given time, as they ``reassess" the quality of information in light of new information. Furthermore, I also find that experts react more strongly to imprecise signals than would be expected if the experts were truly Bayesian. These reactions cause persistance in beliefs over time, since prior beliefs are weighed in current belief calculations. Overall, the manner by which experts respond mirrors confirmatory bias, the idea that individuals interpret ambiguous evidence as being consistent with their prior hypotheses. I conclude that the fact that experts may be biased even without incentives suggests that incentives may exacerbate biases.
Do Social Settings Affect Sexual Assault Prevalence? Evidence from Texas Alcohol Consumption (with Joey Smith, e-mail for a copy)
A large psychology and health literature suggests a relationship between alcohol and sexual assaults, but little is known about the exact relationship. We expand on this literature by decomposing alcohol into both measures of consumption and availability in order to better understand which components of alcohol are important determinants of sexual assaults. To estimate this relationship, we match a unique panel data set which contains the total amount of alcohol sales and total number of package outlets and bars from the Texas Alcoholic Beverage Commission to individual-level crime incident reports from the NIBRS. We find that private settings are substantially more risky; in particular, we find a positive and statistically significant impact of alcohol sales from liquor stores on the number of sexual assaults, but do not find any relationship between alcohol sales from mixed-drink establishments. We interpret our estimates as a lower bound of the relationship, since sexual assaults are substantially underreported.