{"id":4419,"date":"2018-04-27T06:30:43","date_gmt":"2018-04-27T10:30:43","guid":{"rendered":"http:\/\/web.colby.edu\/cogblog\/?p=4419"},"modified":"2020-02-07T11:20:59","modified_gmt":"2020-02-07T16:20:59","slug":"elude-the-illusion-dont-forget-not-to-forget","status":"publish","type":"post","link":"https:\/\/web.colby.edu\/cogblog\/2018\/04\/27\/elude-the-illusion-dont-forget-not-to-forget\/","title":{"rendered":"Elude the Illusion: Understand The Illusion of Validity So You Don\u2019t Fall Victim To This Common Decision Making Bias"},"content":{"rendered":"
\"\"

The illusion of validity will often cause people to make risky bets on a roulette wheel<\/p><\/div>\n

Have you ever placed a bet that a certain number will appear on dice or a roulette wheel? Maybe a number has come up repeatedly so you assume that there is less of a chance that this number will appear in the next roll or spin, even though every number has an equal probability of coming up. Many people fall victim to this bias and end up losing money at casinos. This phenomenon can be explained by the illusion of validity. Defined as a person\u2019s tendency to overestimate their accuracy in making predictions given a set of data, the illusion of validity is one common source of bias in decision making (Einhorn, 1978).<\/p>\n

<\/p>\n

So why exactly are we so bad at making accurate predictions? To answer this question, we must first discuss the role of patterns in the illusion of validity. People love to look for patterns, and when they find these patterns they want to associate a meaning to them. A classic example of this phenomenon is Pareidolia<\/a>,\u00a0where humans see faces in inanimate objects such as the moon, washing machines, or trees. Another relevant example how people find and make meaning of patterns is superstition. Maybe you wear a special shirt when your favorite football team plays because the last two times that you wore this shirt your team won. We all know that washing machines don\u2019t have faces, and the shirt a sports fan wears will not affect the performance of a person\u2019s favorite team. Nevertheless, humans often associate inaccurate meaning to patterns they observe not realizing there is often no meaning behind a pattern, it is just simply a pattern.<\/p>\n

\"\"

The illusion of validity completely disrupts our ability to make accurate predictions<\/p><\/div>\n

In terms of the illusion of validity, when people observe data they often make predictions or conclusions about the data based on patterns that they see with the goal of making sense of their observations. However, as confident as one may be in their predictions, these conclusions often have no scientific grounding (Tversky & Kahnemen, 1974). In his recent book Thinking Fast and Slow<\/em> (2011), Kahneman brings this bias to life in sharing an example from his work as a psychologist selecting leaders in the Israeli Army. They designed a drill where soldiers had to work together in groups of 8 through challenging obstacles. Kahneman was confident that those who succeeded in the drill would become the best leaders over time. But as these soldiers progressed in the Army, Kahneman learned that other things better predicted the selection of leaders.\u00a0 Nevertheless, he still kept using the drill for the early identification of leaders.\u00a0 Kahneman and his colleagues were experiencing the illusion of validity.\u00a0 And this experience contributed to his research on this decision making bias.<\/p>\n

Like me, you might be wondering why, if there is evidence that disconfirms peoples\u2019 predictions, why people still are susceptible to the illusion of validity? Unfortunately, as the example of the Israeli Army above, even when people are well informed on the illusion of validity, they still can be influenced by it. A study by Oskamp (1965) helps us understand why. In this study participants, either clinical psychologists or undergraduate students, were asked to make predictions about patient\u2019s reasons for being in the hospital\u2014were they there for psychiatric problems, or another medical reason? The researchers found that there was not a large difference in the accuracy of predictions made — experienced clinical psychologists were only slightly better at making predictions than untrained undergraduates. Oskamp explained how clinical psychologist participants remembered the instances in the past where their predictions had been correct while ignoring the predictions that they had made in the past that were incorrect (Oskamp, 1965). In other words, when making predictions, people recalled memories of specific events when they made successful predictions of patient type, while ignoring or deemphasizing times that they were incorrect. In essence, these people were making predictions based on selective memories because they failed to remember or address the times when their predictions were incorrect.<\/p>\n

Although we like to think that our memory recalls all kinds of experience with similar accuracy, we all in fact have selective memories<\/a>. Selective memories can help to explain the illusion of validity and provide some rationale for why people seem to be ignoring the times that their predictions were unsuccessful. One reason that people seem to be \u201cignoring\u201d their incorrect predictions is because they simply do not remember that they were incorrect. Another decision making bias may come into play — retrospective bias. With retrospective bias, a person may, for example, predict that a seven will be rolled on a die. However, after the roll reveals a five, that same person may look back and think, \u201cI knew it wasn\u2019t going to be a seven,\u201d even though they obviously did not. As a result, we are making inaccurate predictions because the information that we are basing our predictions on is faulty because we based it on selective or even false memories.<\/p>\n

\"\"

Standardized test scores are not a good predictor of college success<\/p><\/div>\n

How does the illusion of validity impact you in the real world beyond the casino? Well, the decisions that are made about us and for us are often tainted by the illusion of validity. For example, being admitted to college has become increasingly competitive and unpredictable. The admissions officers are basing their decisions off a whole host of things such as high school GPA, extracurriculars, supplemental essays, and, standardized test scores. Other factors that admissions officers have recently paid increasing attention to are diversity, gender, financial need, and athletics. These \u201cexperts\u201d in admissions try to predict who will be successful based on the data provided by each applicant. However, these experts can be overconfident and simply wrong at predicting who will be most successful in college. For example, in the past, standardized tests are often used by large universities as an important factor in admissions because it is easily comparable across candidates.\u00a0 However, more schools are now making reporting test scores optional, based on recent evident by Hiss (2014) that other factors are just as accurate, and even more accurate in predicting which applicants will be successful at their college.\u00a0 So why are so many admissions officers still placing so much confidence in the predictive power of standardized test scores?\u00a0 Perhaps the illusion of validity is one reason.\u00a0 They used it in the past and believe it will be a good predictor in the future, even though the college experience is becoming more complex with issues like diversity and helping disadvantaged students gain access to elite colleges becoming higher priorities.\u00a0 I suggest that admissions officers need to be aware of the illusion of validity and find ways to continue challenge conventional wisdom on the admissions process.<\/p>\n

I hope that by reading this entry you have learned the basics of the illusion of validity. Going forward let\u2019s make sure to question both our own and expert\u2019s predictions so that we all make better decisions whether at the casino or in college admissions!<\/p>\n

References:<\/p>\n

Einhorn, H.J., & Hogarth, R.M. (1978). Confidence in judgement: persistence of the illusion of validity. Psychology Review<\/em>, 85(5), 395-416. doi:10.1037\/0033-295X.85.5.395<\/p>\n

Hiss, W.C., (2014, February 5th) Defining promise: optional standardize testing policies in American college and university admissions. Retrieved from https:\/\/offices.depaul.edu\/enrollment-management-marketing\/test optional\/Documents\/HISSDefiningPromise.pdf<\/a><\/p>\n

Oskamp, S. (1962). The relationship of clinical experience and training methods to several criteria of clinical prediction. Psychological Monographs: General and Applied<\/em>, 76(28), 1-27. doi:10.1037\/h0093849<\/p>\n

Weinstein, Y. & Roediger H. (2010). Retrospective bias in test performance: providing easy items at the beginning of a test makes students believe they did better on it. Memory & Cognition<\/em>, 38(3), 366-376, doi:10.3758\/MC.38.3.366.<\/p>\n

Visuals:<\/p>\n

    \n
  1. https:\/\/giphy.com\/gifs\/southparkgifs-26uflBhaGt5lQsaCA<\/li>\n
  2. https:\/\/betfaq.com\/land\/winreg\/?a=ad666&ac=doorways&sa=mrfeathe<\/li>\n
  3. http:\/\/affinitymagazine.us\/2017\/06\/09\/why-standardized-testing-doesnt-define-you\/<\/li>\n<\/ol>\n

     <\/p>\n","protected":false},"excerpt":{"rendered":"

    Have you ever placed a bet that a certain number will appear on dice or a roulette wheel? Maybe a number has come up repeatedly so you assume that there is less of a chance that this number will appear in the next roll or spin, even though every number has an equal probability of […]<\/p>\n","protected":false},"author":7674,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"footnotes":""},"categories":[266316,80215,80218],"tags":[408652],"_links":{"self":[{"href":"https:\/\/web.colby.edu\/cogblog\/wp-json\/wp\/v2\/posts\/4419"}],"collection":[{"href":"https:\/\/web.colby.edu\/cogblog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/web.colby.edu\/cogblog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/web.colby.edu\/cogblog\/wp-json\/wp\/v2\/users\/7674"}],"replies":[{"embeddable":true,"href":"https:\/\/web.colby.edu\/cogblog\/wp-json\/wp\/v2\/comments?post=4419"}],"version-history":[{"count":3,"href":"https:\/\/web.colby.edu\/cogblog\/wp-json\/wp\/v2\/posts\/4419\/revisions"}],"predecessor-version":[{"id":4489,"href":"https:\/\/web.colby.edu\/cogblog\/wp-json\/wp\/v2\/posts\/4419\/revisions\/4489"}],"wp:attachment":[{"href":"https:\/\/web.colby.edu\/cogblog\/wp-json\/wp\/v2\/media?parent=4419"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/web.colby.edu\/cogblog\/wp-json\/wp\/v2\/categories?post=4419"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/web.colby.edu\/cogblog\/wp-json\/wp\/v2\/tags?post=4419"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}