- Expectation Bias
Fans of critical thinking need to read You Are Not So Smart. With each post, David McRaney thoroughly covers one misconception that humans are frequently guilty of. Over time, he’s amassed quite a compendium. There’s also the Visual Study Guide to Cognitive Biases by efern211 (which I’ve previously mentioned). It’s slick, colorful, and succinct.
Over the coming year, I’ll pull a me too on this front. I’ll be quick, draw upon one or two examples, and efficiently explain stuff. It may be redundant, but more data is better. It’s not as though our culture is awash in critical thinking skills.
Expectation Bias
The tendency for experimenters to believe, certify, and publish data that agree with their expectations for the outcome of an experiment, and to disbelieve, discard, or downgrade the corresponding weightings for data that appear to conflict with those expectations. (VSGCB)
This is when we only agree with the data that we like. My guess is that we rarely notice we’re doing. I’ve seen this tendency manifest when we’re drawn into heated arguments. 
The easiest example is the environmental debate. If I decide to accept one set of data, then disregard another based solely upon my personal conclusions, then I’ve made this error.
I might critique methods. For instance: if data was derived at through slipshod techniques or the reason includes ‘God did it’, that’s one thing. But disregarding data out of hand just because I don’t like what it says is quite another.
We only trust experts if they agree with us
This is not exactly news, but I’m sure we all convince ourselves that we would never do that.That’s what other people do. Ha.
But even among self-doubting types, there are some intellectual matters that we’re willing to concede we’re wrong about and others we aren’t. This is what happens when our personal philosophies and emotions run roughshod over our understanding.
Footnote
I’ve certainly been guilty of this.
I’ve heard about this a few times and you have to be careful in how much credit you give it. Does exception bias exist? Yes. Does everyone do it? No.
Those who we can all agree use critical thinking, tend not to do it. If anything, they overplay the idea of exception bias and apologize for it (even when they’re not doing it). They tend to doubt their ability to rise above this type of bias so much that they wonder if they need to consider every point of view as equally valid.
Those who are guilty of exception bias don’t worry or consider it at all. They doubt every source except those that agree with their opinions.
From a media standpoint, the example would be Fox News versus NPR. Fox News (and it’s viewers) are more likely to attack dissenting viewpoints. NPR on the other hand, is more likely to worry that they are only presenting one side and go out of their way to present a dissenting viewpoint even when it isn’t appropriate or necessary.
I think a lot of what you’re seeing as potential exception bias is your synthesis of the facts. If you conduct analysis and develop an opinion that’s ok. The key is being open to changing it if information changes or if you’re presented with new facts.
Perhaps the risk is not so great because I’m more of a process guy than I am a conclusions guy. I’m certainly not slapping a knee-jerk apology on every conclusion I express. But your essential point stands: Basic intellectual honesty is a bulwark against this particular bias.
Thanks so much for contributing your thoughts.
You said it really well, a little bit of intellectual honesty goes a long way. If more people were willing to listen and occasionally determine if they agree or disagree with an idea based on its merits, life would be a little better and more interesting.
Problem is, this isn’t just about intellectual honesty. People really see things differently when they agree versus when they disagree. It’s a nagging issue for researchers as well. Djerassi’s novel “Cantor’s Dilemma” looks at this issue at length. This is one of the many reasons why any sort of scholarship is fundamentally a group endeavor. We may not like our critics, but the world needs them.