Can We Trust Industry Funded Research?
Much of today’s food and nutrition research is paid for by commercial interests. Nutrition Diva explains how to detect bias—of all types.
The podcast version of this article was sponsored by Stitcher Smart Radio, the best way to listen to podcasts on your smart phone. Enter the promo code DIVA to be entered to win $1000.
Recently, we had a lively debate on my Facebook page over whether we should keep chocolate milk out of schools. There are data to suggest that kids drink more milk—and less soda – when chocolate milk is available. Several readers pointed out that some of the research was funded by the dairy industry—as if to say, “Therefore, the information is false. Case closed.”
Who funds a study is a obviously a very important piece of information—and that’s why researchers are required to disclose their funding sources and any other potential conflicts of interest at every step of the process. The fact that a study has been funded by a private interest suggests that we need to take a closer look at how it was conducted and reported. However, it not automatically invalidate the results.
Is Industry Funded Research Biased?
When a commercial interest invests in scientific research, they’re obviously hoping for results that will benefit their product or industry. And if they get results that put their product in a positive light, you better believe they’re going to try to get maximum mileage out of it. But, at least they’re attempting to provide verifiable data for their claims as opposed to just making stuff up!
And we benefit. Conducting research is quite expensive. Quite frankly, if the food industry didn’t fund nutrition research, there’d be a whole lot less research. And although they don’t always work flawlessly, there are ethical safeguards in place.
In order to get the results published in a peer-reviewed journal, researchers have to follow certain rules and protocols about how studies are designed, how data are collected and analyzed, and how the results are reported. The researchers also have to disclose their funding sources—which may cause the review boards to look even more closely at those study designs to be sure that they are scientifically sound.
However, there’s still room for bias. Industry is more likely to sponsor research that they feel has a good chance of yielding positive results—and this may affect which questions get studied, and how. Commercial interests are also very eager to publish and publicize results that benefit them—and not so eager to promote findings that don’t.
What is Publication Bias?
Indeed, industry-funded research is more likely to be flattering to the funder than not. However, this is true of all research to some extent and not just industry-funded research. It’s called publication bias. If I discover that a particular thing (such as a drug, a diet, or a nutritional supplement) has an effect, my study is three times more likely to be accepted for publication in a scientific journal than a study that found no effect.
Apparently, this is simply human nature: As a species, we find it more significant when something happens than when nothing happens. But when you really think about it, proving that something has no effect is often just as meaningful as showing that it has a positive or negative effect.
What is Confirmation Bias?
There’s another kind of prejudice in the scientific community known as confirmation bias. Although scientists like to believe that they are completely objective, most have certain beliefs or suspicions about how things work. Their ideas subtly and unconsciously influence how they design their studies and how they interpret the results. As a result, researchers are more likely to get results that confirm their underlying beliefs.
Confirmation bias also operates at the larger level of the scientific community. If my results are in line with generally-accepted ideas, they are more likely to be published. As a result, it can be very difficult to challenge the conventional wisdom—even when the conventional wisdom is wrong. It’s been known to happen. For decades we told people that eating too many eggs would give you high cholesterol, for example. Now we know that this is generally not the case.
People who report and comment on research (like yours truly) also suffer from ideological bias. We tend to pay more attention to research that confirms our worldview and to distrust or discount findings that don’t.
Bias May be Easier to Detect in Industry Funded Research
One advantage to industry funded research may be that the bias is a lot easier to detect because it’s obvious what to look for. If I see that a study has been funded by a commercial interest, I’m going be looking for ways that they may have slanted the study design or interpretation of results in their favor. (So is the peer-review board that’s evaluating the study for publication.) The fact that industry funding is involved may even cause a study to be held to a slightly higher standard of objectivity.
But when a study has been funded by a university, a non-profit foundation, or a government grant, it’s not immediately clear what sort of assumptions or prejudices the researchers may have. Usually, the researchers themselves are not aware of their own biases. That’s what makes ideological bias so insidious: We (both as individuals and as societies) are often the last to realize when we are under the influence.
How the Scientific Method Corrects for Bias
[[AdMiddle]Research is never going to be perfectly objective because it’s conducted—and interpreted--by humans. The scientific method is a system we’ve devised to try to compensate for our inescapable prejudices. One of the first rules is that no single research finding is definitive. If I do an experiment and get a certain result, the next step is to see if you get the same result when you do the same experiment. If you don’t, we need to try to explain why. And for every explanation we come up with, we have to ask whether there are other equally plausible explanations besides the one we came up with. If there are, we have to rule one or the other out.
It can get pretty messy. But the more data we collect, the truer the picture that emerges. To the extent that industry-funded research produces more data to inform that big picture, I think we’re better off with it than without it.
How to Evaluate Research
Here’s the bottom line: We shouldn’t assume that all industry-funded research is false and we can’t assume that publicly funded research is always true. No matter where it came from, you need to poke around under the hood a little to see whether the study design makes sense, whether the actual data are in sync with how the study results are presented to the media, and whether there are studies from other sources that either confirm or refute the results. It’s also important to be aware of your own beliefs and ideological biases and how they may be coloring your evaluations.
Finally, scientists and scientific commentators must be ready and willing to adjust or even abandon long-held beliefs in the face of new evidence. And that can be difficult for know-it-alls like us. That’s why I’m always glad to hear from Nutrition Diva readers and listeners with different points of view or research that I may not have considered. So keep those emails and comments and links Facebook posts coming!