Ask Science shares his rules for avoiding being scammed by medical and scientific claims.
Rule #3: Has this treatment already worked for millions of anonymous people?
A popular thing to post on sites like these are the outrageous numbers of people who have already received the benefits of these treatments. You should be especially wary of anonymous or unverifiable testimonies like: “I tried Dr. Flubber’s miracle shampoo and my hair all grew back, like the next day! - Jimmy M.”
Rule #4: Is this the “medical secret doctors don’t want you to know about”?
I know quite a few doctors, nurses, and even a few hospital administrators, and I can tell you that not one of them is part of a secret conspiracy to keep people sick in order to line their pockets. In fact, I imagine that most of them would be appalled and insulted at the suggestion. This might be a shock, but it turns out most people who become doctors do so because they want to help others. Who knew?
That’s not to say that horrible medical-related conspiracies haven’t occurred, but that doesn’t mean doctors around the world are meeting in secret to discuss how to make sure nobody cures cancer.
Rule #5: Are there any real, peer-reviewed medical studies about these claims?
This one is the most difficult rule to apply, but in my view it’s also one of the most important. The reason this rule is difficult to apply is that sometimes it’s hard for someone who isn’t trained in science to interpret the results of a peer-reviewed medical study.
The media in general is an unreliable source for medical research, because of many reasons I’ve already discussed in previous episodes. Wikipedia, the go-to source for many people looking for answers to life’s mysteries, often suffers greatly from editors’ own confirmation bias.
Confirmation bias occurs when you give more weight to information that supports your point of view than to information that doesn’t. This is an extremely difficult bias to overcome and is one of the reasons why the peer-review process for scientific research involves multiple, independent scientists reviewing the same research.
Some people claim that it’s hard to publish papers that show a contrary view to what is generally accepted by the scientific community. I have never seen this occur. What gets rejected by the scientific community are claims based on bad science, bad experimental design, erroneous conclusions, or bad statistics (though the latter reason doesn’t cause nearly enough rejections as it should).
See also: Retractions in Science
If there was some pretty commonplace research presented to an editor that turned everything we knew about a given topic on its head, but had the scientific data to back up its claims, that research would certainly be published. In fact, this happens all the time. If it didn’t, science would get pretty boring.
So those are my rules for verifying scientific claims. If you ever come across a claim that sounds super pretty commonplace, but you’re having a hard time deciding if it could really be true, send me an email and I’ll be happy to feature it on the Ask Science show.