10 Questions To Distinguish Real From Fake Science

This article was originally posted on Forbes and shared on this Mary Szela blog because it is important for consumers to distinguish between actual science and marketing schemes.

Pseudoscience is the shaky foundation of practices–often medically related–that lack a basis in evidence. It’s “fake” science dressed up, sometimes quite carefully, to look like the real thing. If you’re alive, you’ve encountered it, whether it was the guy at the mall trying to sell you Power Balance bracelets, the shampoo commercial promising you that “amino acids” will make your hair shiny, or the peddlers of “ natural remedies” or fad diet plans, who in a classic expansion of a basic tenet of advertising, make you think you have a problem so they can sell you something to solve it.

Pseudosciences are usually pretty easily identified by their emphasis on confirmation over refutation, on physically impossible claims, and on terms charged with emotion or false “sciencey-ness,” which is kind of like “truthiness” minus Stephen Colbert. Sometimes, what peddlers of pseudoscience say may have a kernel of real truth that makes it seem plausible. But even that kernel is typically at most a half truth, and often, it’s that other half they’re leaving out that makes what they’re selling pointless and ineffectual. But some are just nonsense out of the gate. I’d love to have some magic cream that would melt away fat or make wrinkles disappear, but how likely is it that such a thing would be available only via late-night commercials?

What science consumers need is a cheat sheet for people of sound mind to use when considering a product, book, therapy, or remedy. Below are the top-10 questions you should always ask yourself–and answer–before shelling out the benjamins for anything, whether it’s anti-aging cream, a diet fad program, books purporting to tell you secrets your doctor won’t, or jewelry items containing magnets:

1. What is the source? Is the person or entity making the claims someone with genuine expertise in what they’re claiming? Are they hawking on behalf of someone else? Are they part of a distributed marketing scam? Do they use, for example, a Website or magazine or newspaper ad that’s made to look sciencey or newsy when it’s really one giant advertisement meant to make you think it’s journalism?

2. What is the agenda? You must know this to consider any information in context. In a scientific paper, look at the funding sources. If you’re reading a non-scientific anything, remain extremely skeptical. What does the person or entity making the claim get out of it? Does it look like they’re telling you you have something wrong with you that you didn’t even realize existed…and then offering to sell you something to fix it? I’m reminded of the douche solution commercials of my youth in which a young woman confides in her mother that sometimes, she “just doesn’t feel fresh.” Suddenly, millions of women watching that commercial were mentally analyzing their level of freshness “down there” and pondering whether or not to purchase Summer’s Eve.

3. What kind of language does it use? Does it use emotion words or a lot of exclamation points or language that sounds highly technical (amino acids! enzymes! nucleic acids!) or jargon-y but that is really meaningless in the therapeutic or scientific sense? If you’re not sure, take a term and google it, or ask a scientist if you can find one. Sometimes, an amino acid is just an amino acid. Be on the lookout for sciencey-ness. As Albert Einstein once pointed out, if you can’t explain something simply, you don’t understand it well. If peddlers feel that they have to toss in a bunch of jargony science terms to make you think they’re the real thing, they probably don’t know what they’re talking about, either.

4. Does it involve testimonials? If all the person or entity making the claims has to offer is testimonials without any real evidence of effectiveness or need, be very, very suspicious. Anyone–anyone–can write a testimonial and put it on a Website. Example: ”I felt that I knew nothing about science until The Science Consumer blog came along! Now, my brain is packed with science facts, and I’m earning my PhD in aerospace engineering this year! If it could do it for me, The Science Consumer blog can do it for you, too! THANKS, SCIENCE CONSUMER BLOG! –xoxo, Julie C., North Carolina”

5. Are there claims of exclusivity? People have been practicing science and medicine for thousands of years. Millions of people are currently doing it. Typically, new findings arise out of existing knowledge and involve the contributions of many, many people. It’s quite rare–in fact, I can’t think of an example–that a new therapy or intervention is something completely novel without a solid existing scientific background to explain how it works, or that only one person figures it out. It certainly wouldn’t just suddenly appear one night on an infomercial. Also, watch for words like “proprietary” and “secret.” These terms signal that the intervention on offer has likely not been exposed to the light of scientific critique.

6. Is there mention of a conspiracy of any kind? Claims such as, “Doctors don’t want you to know” or “the government has been hiding this information for years,” are extremely dubious. Why wouldn’t the millions of doctors in the world want you to know about something that might improve your health? Doctors aren’t a monolithic entity in an enormous white coat making collective decisions about you any more than the government is some detached nonliving institution making robotic collective decisions. They’re all individuals, and in general, they do want you to know.

7. Does the claim involve multiple unassociated disorders? Does it involve assertions of widespread damage to many body systems (in the case of things like vaccines) or assertions of widespread therapeutic benefit to many body systems or a spectrum of unrelated disorders? Claims, for example, that a specific intervention will cure cancer, allergies, ADHD, and autism (and I am not making that up) are frankly irrational.

8. Is there a money trail or a passionate belief involved? The least likely candidates to benefit fiscally from conclusions about any health issue or intervention are the researchers in the trenches working on the underpinnings of disease (genes, environmental triggers, etc.), doing the basic science. The likeliest candidates to benefit are those who (1) have something patentable on their hands; (2) market “cures” or “therapies”; (3) write books or give paid talks or “consult”; or (4) work as “consultants” who “cure.” That’s not to say that people who benefit fiscally from research or drug development aren’t trustworthy. Should they do it for free? No. But it’s always, always important to follow the money. Another issue that’s arisen around pseudoscience is whether or not a bias of passionate belief is as powerful as fiscal motivation. If you have a bias detector, turn it on to full power when evaluating any scientific claim. If yours is faulty–which you might not realize because of bias–perhaps you can find someone in real life or online with a hypersensitive bias detector. Journalists, by nature of training and their work, often seem to operate theirs on full power.

9. Were real scientific processes involved? Evidence-based interventions generally go through many steps of a scientific process before they come into common use. Going through these steps includes performing basic research using tests in cells and in animals, clinical research with patients/volunteers in several heavily regulated phases, peer-review at each step of the way, and a trail of published research papers. Is there evidence that the product or intervention on offer has been tested scientifically, with results published in scientific journals? Or is it just sciencey-ness espoused by people without benefit of expert review of any kind?

10. Is there expertise? Finally, no matter how much you dislike “experts” or disbelieve the “establishment,” the fact remains that people who have an MD or a science PhD or both after their names have gone to school for 24 years or longer, receiving an in-depth, daily, hourly education in the issues they’re discussing. If they’re specialists in their fields, tack on about five more years. If they’re researchers in their fields, tack on more. They’re not universally blind or stupid or venal or uncaring or in it for the money; in fact, many of them are exactly the opposite. If they’re doing research, usually they’re not Rockefellers. Note that having “PhD” or even “MD” after a name or “Dr” before it doesn’t automatically mean that the degree or the honorific relates to expertise in the subject at hand. I have a PhD in biology. If I wrote a book about chemical engineering and slapped the term PhD on there, that still doesn’t make me an expert in chemical engineering. And I’m just one person with one expert voice in the things I do know well. I recommend listening to more than one expert voice.

There is nothing wrong with healthy skepticism, but there is also nothing wrong in acknowledging that a little knowledge can be a very dangerous thing, that there are really people out there whose in-depth educations and experience better qualify them to address certain issues. However, caveat emptor, as always. Given that even MDs and PhDs can be disposed to acquisitiveness just like those snake-oil salesmen, never forget to look for the money. Always, always follow the money.