Baloney Detection – How to Distinguish between Science and Pseudoscience [A Short Guide]

Baloney Detection - How to Distinguish between Science and Pseudoscience [A Short Guide]

Introduction

“When we’re growing up (!and, often in adult life) we tend to be pretty credulous. We just believe almost anything that people tell us, especially if it comes from authorities, experts, textbooks, politicians, television, Youtube, and the Internet. I mean, there’s just this sort of sea of information coming at us. And how can you tell the difference between: it’s right or wrong? How do you know?!” Michael Shermer (emphasis mine)

There’s way too much misinformation ‘out there’. And some of it is on purpose – disinformation. Whether it happens on the Internet or within your circle of friends, family and acquaintances, you (and I) are being bombarded with non-sense ad nauseam.

Filtering through, to get some valuable information, whenever that’s possible, is not easy. For this purpose, Carl Sagan and Michael Shermer formulated what they call: a baloney detection kit.

It started with an essay written by Sagan a long while ago [2]. In it, Sagan advises people on how to avoid logical and rhetorical fallacies. Acknowledging the value of this manifesto, Michael Shermer devised a more accessible and easily readable format of Sagan’s words [1].

Pure, raw information is of immense value today. Most information that reaches your senses is coated with personal filters and prejudices; it is distorted, manipulated, or modified in one way or another.

Here, I want to reignite awareness on Sagan and Shermer’s advice, hoping that it will help us better filter through this mountain of non-sense and also help us stave off the bad influence of malicious parties.

The Baloney Detection Kit

According to Michael Shermer (founder of The Skeptic Society and editor of the magazine Skeptic), to detect baloney is to be able to discriminate between science and pseudoscience. To make the process more interactive, he suggests 10 questions we should ask when being presented/confronted with a claim [1]:

  1. How reliable is the source of the claim?

Does the information you are being provided show signs of inaccurate interpretation and/or intentional manipulation?

Human nature is faulty by design. We are creatures of habit more than creatures of reason. Reasoning is a skill that has to be developed. Habits (and habits of thinking) are not. We like to believe in things and we like to seek patterns, even when they do not exist.

But, we aren’t all that bad. Don’t get me wrong. We also posses an automatic internal trigger of non-sense (a bullshit detector, !pardon my parlance) that turns on whenever we think something is not as it is presented to us. Here’s a personal story.

In my childhood I used to buy a local newspaper that came out once a week. It had articles about psy- and paranormal phenomena. It was generically called Dracula. I’m not sure if it’s still in print.

To a large extent I ‘wanted’ to believe most of the stories and articles in it. In retrospect, I could not believe a story about ghosts and disbelieve one about the most recent spotted UFOs. It would have lead to cognitive dissonance (conflicting beliefs). I didn’t know about that back then. But even though I had a tendency to believe in such stories, I also had this persistent feeling (BS detector on, but muted?) that something was not quite right. However, life happened, I grew up, I broaden my activities and interests, and I stopped buying the newspaper.

  1. Does this source often make similar claims?

As Shermer says, pseudoscientists make a habit of going well beyond the facts.

They adopt a claim that has received some scientific support in a certain context, they take it out of context and they fabricate a contorted (often sophisticated) version of it. The gullible, uneducated person tends to believe it.

When nothing bad happens because of the quack’s theories, you could become tolerant of the quack, but remain strong in your disbelief. You may not blame the pseudoscientist/quack for their ideas because you would understand that they may have been poisoned by their own fabricated beliefs. They could be at the point of not distinguishing fact from trivial. To be specific…

Science has shown that in some situations large amounts of focused radiation may be detrimental to human health, in part by causing DNA damage (single strand breaks SSB, double strand breaks DSB, or other DNA lesions).

Pseudoscientist radiophobics (those fearful of all doses of all types of radiation) extrapolate this information in multiple ways, maintaining that all radiation is bad, that cellphones cause cancer (citing from dubious mouse studies and from poorly controlled observational studies), that Wi-Fi is dangerous, and many other similar claims. I was once on the bandwagon.

So far, to my knowledge, rigorous scientific studies do not support these claims.

An example of solid scientific claim, for me, is:

Carefully controlled fecal transplants show an overwhelming success rate (90%+) in curing C. Difficile infections (often caught in hospital settings). Bogus radiation claims are far from receiving such scientific support.

  1. Have the claims been verified by another source?

Arguments become stronger when various sources try to prove them wrong (!not right) and fail to do so.

Arguments are never 100% right. There is always the chance for an argument to be disproved by a carefully conducted experiment. They are always up to scrutiny.

Bad rationale: All swans are white. You completely disprove this argument when you find a non-white swan.
Better rationale: Most swans are white. The argument is not as strong (is nuanced) and it’s least likely to be completely disproved.

Pseudoscientists and quacks make claims in isolation and out of context. They cite other entities who have made the same unverified claims. They are often within the same circle of belief (reinforcing herd mentality). As Shermer suggests [1]:

“We must ask, who is checking the claims, and even who is checking the checkers? The biggest problem with the cold fusion debacle, for instance, was not that Stanley Pons and Martin Fleischman were wrong. It was that they announced their spectacular discovery at a press conference before other laboratories verified it. Worse, when cold fusion was not replicated, they continued to cling to their claim. Outside verification is crucial to good science.”

Not everything that happens in the world of science is always moral, rational, and correct. But it is more testable and verifiable and possibly less dangerous than in pseudoscience.

  1. How does the claim fit with what we know about how the world works?

Some people say that the pyramids were built by an advanced civilization. Others say they were built by aliens, who still live among us and control us from the shadows. I’m not saying that you encounter this in your daily life, but if you do stumble upon this or a similar situation, you could ask yourself (and the claimant) about the context of the supposedly advanced civilization:

Where is the rest of their marvelous artifacts?
Where are their clothes? Where are their tombs?
Where is their trash and where are the tools they used?

As Shermer points out, archeology does not operate like that.

  1. Has anyone gone out of the way to disprove the claim, or has only supportive evidence been sought?

Once again, arguments become stronger as they cannot be disproved. That is why the scientific method is all about verifying, re-verifying, reproducing, replicating, and trying to falsify claims (proven them wrong).

A claim that inherently (by its nature and composition) cannot be falsified is not scientific.

If a theory or an argument cannot make a testable prediction, it is not scientific. Thus, unfalsifiable theories (that cannot be rigorously scrutinized) are not scientific.

One example of unfalsifiable claim is:

There is a giant monster in the lake near my home.

If this is wrong, it is only because of lack of evidence. And this may not convince true-believers (fanatics) against the statement. They could, for their own peace of mind (in the very least sense) keep fabricating statements like:

The monster is invisible to detection. or
It could not be seen because it was hiding in the mud at the bottom of the lake.

Another illustration of falsifiability/non-falsifiability is presented in Sagan’s Demon Haunted World:

We start from the premise:

There is a dragon in my garage“. (yet another monster example – excuse my lack of inspiration)

To prove or disprove the statement you are allowed to ask questions.

Can I see it? No, it is invisible.
Can I touch it? No, it is made of ether.
Can I hear it? No, because it is silent.

Such is a way that all questions can be answered, thus making the claim (the existence of the dragon) unfalsifiable. And, unfalsifiable claims (that cannot appropriately be tested) are pseudoscience.

  1. Where does the preponderance of evidence point?

Shermer explains how the theory of evolution is ‘proved’ by numerous pieces of evidence emerging from different fields [1]:

“No one fossil, no one piece of biological or paleontological evidence has evolution” written on it; instead tens of thousands of evidentiary bits add up to a story of the evolution of life. Creationists conveniently ignore this confluence, focusing instead on trivial anomalies or currently unexplained phenomena in the history of life.”

  1. Is the claimant playing by the rules of science?

Simply put, is the claimant using good reasoning and valid research tools in proposing their claims?

A representative example in this case is: the difference between UFO-logists and the organization called SETI (Search for Extraterrestrial Intelligence).

While UFO fanatic believers start from the premise that UFOs exist (by making anecdotal claims of alien abductions, by presenting obscure pictures of UFOs, by discussing conspiracies, etc), SETI starts from the premise that extraterrestrial intelligence does “not exist and that they must provide concrete evidence before making the extraordinary claim that we are not alone in the universe.” [1]

  1. Is the claimant providing an explanation for the observed phenomena or merely denying the existing explanation?

A strategy that is often employed by many when debating is to criticize opposition instead of providing strong support for their beliefs. Shermer explains this in the context of creationism [1]:

“It is next to impossible to get creationists to offer an explanation for life (other than God did it”). Intelligent Design (ID) creationists have done no better, picking away at weaknesses in scientific explanations for difficult problems and offering in their stead. ID did it.” This stratagem is unacceptable in science.”

  1. If the claimant proffers a new explanation, does it account for as many phenomena as the old explanation did?

Anyone can find a few anomalies or abnormalities that current science does not seem to explain. However, pseudoscientists (isolate) cling to such claims, they take them out of context and fabricate their own theories from them. They often use arguments like:

Newton was wrong!
Darwin was wrong!
Einstein was wrong!
And I am right! My new theory explains the world.

However, can this theory also explain everything else that Newton, Einstein, or Darwin explained, such as gravity, general relativity, quantum physics, etc?

Most often they cannot. Sadly, the one exposed to such claims is most often not very versed in good (rational) thinking to be able to spot the discrepancy. And they tend to fall in love with the shaky/shady theories of the pseudoscientist/quack.

  1. Do the claimant’s personal beliefs and biases drive the conclusions, or vice versa?

Education in rational thinking, learning more about the inherent biases and fallacies we are subjected to, being able to recognize personal biases and those of other people are not part of the repertoire we are born with. Good thinking and a healthy mind are not implicit. You have to actively pursue in developing them…

In the scientific process and in the research community, during peer-review such biases are rooted out. However, and I must add an exclamation mark: this is not always the case!!!

In real-life, you could be conferred fair protection against pseudoscience, manipulation, and self deception through self-education and a healthy dose of skepticism (to be read healthy, not extreme).

Conclusion

Pseudoscientists, quacks and promoters of bad science often argue that human advancement is brought up by geniuses, by wild discoveries, and by people who think outside the proverbial box; that society is forwarded by original thinkers, by those who do not comform and by those who are not willing to commit to the norm.

And I agree, in most part! Human history is the testimony for that.

But, let us remind ourselves that among the few genius minds of our history, there have always existed many quacks and pseudoscientists.

Geniuses may be seen as quacks at first. But the difference between genius and quack is that genius withstands the test of time and the rigorousness of science. Quacks fade.

And it’s not easy to survive with so many quacks and so much pseudoscience around, especially when they have unlimited means of promoting their message. As Michael Shermer says [1]:

“Clearly, there are no foolproof methods of detecting baloney or drawing the boundary between science and pseudoscience.

Yet there is a solution: science deals in fuzzy fractions of certainties and uncertainties, where evolution and big bang cosmology may be assigned a 0.9 probability of being true, and creationism and UFOs a 0.1 probability of being true.

In between are borderland claims: we might assign superstring theory a 0.7 and cryonics a 0.2.

In all cases, we remain open-minded and flexible, willing to reconsider our assessments as new evidence arises. This is, undeniably, what makes science so fleeting and frustrating to many people; it is, at the same time, what makes science the most glorious product of the human mind.”

(Mike, sorry for the long quote! But, it’s too good to be paraphrased or modified in anyway or another).

If you reached this far and you are still uncertain of the message conveyed, you could watch a video version of baloney detection, presented by Shermer himself:

I hereby make a similar suggestion: Please don’t take my words for granted. Please don’t believe what I say. Educate yourself in rational thinking and in logical fallacies, seek good evidence, and make your own judgments.

Ending note (to self, primarily):

Developing a sound mind and forging a healthy brain is a work in progress, often a frustrating struggle. It is against human nature and it is not something we are born with.


Resources:

  1. Shermer, M. (2001). Baloney Detection: How to Draw Boundaries between Science and Pseudoscience. Scientific American, 285(5), 36.
  1. Sagan, C. (2007). The fine art of baloney detection. Paranormal Claims: A Critical Analysis, 1.
  1. Rational Wikipedia (2016). The Fine Art of Baloney Detection. Retrieved from http://rationalwiki.org/wiki/The_Fine_Art_of_Baloney_Detection

Image Credits: SLC Lyrics


Get on my list of friends
More about my book Persistent Fat Loss
More about my book Ketone Power
More about my book T-(Rx)
More about my book Periodic Fasting

Related posts:

Comments

comments

2 Responses to Baloney Detection – How to Distinguish between Science and Pseudoscience [A Short Guide]

  1. Barbara says:

    Most people speak from their childhood indoctrination and are unaware of their biases. Critical thinking skills are not taught in school. Nice ending phrase as well. Great post!

    • Chris Chris says:

      Barbara, there’s little we can do about our childhood since we’re mostly unaware. However, growing up and become more self-reliant does not excuse not to educate oneself in thinking. Thanks for stopping by 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *