If you’re looking to be stronger, faster, feel better, and otherwise be the best version of yourself, you can’t do it alone. After all, if you were confident that you had all the answers and the skill to execute them, you wouldn’t be reading this article. Unfortunately, you can’t just ask Siri for the answers, either. The Internet has afforded us an incredible wealth of information, but we’re still faced with the monumental task of sorting through the sea of data to get to the good stuff, and then translating that raw data into action.
In an ideal world, fitness and nutrition research would light the way. But between the challenges of executing a good study, the distortion-machine that is the popular news media, and our own biases and misinterpretations, a stroll through PubMed can feel like walking through a minefield. Still, with a few pointers on what to look for and hazards to avoid, you may find a useful gem or avoid falling victim to the latest scam product.
The scientific method of inquiry is possibly the greatest engine of discovery on earth, but to believe blindly in science is to miss the point entirely. Accepting without question or critical analysis the conclusion of anything published in an official journal or stated by someone with “PhD” behind their name won’t do you any good. The next time you see a news report on a new study or find yourself digging through PubMed, ask yourself the following questions.
The best research in the world is often misinterpreted or misapplied. [Photo credit: Wikimedia Commons]
Can I Actually Read the Study?
If you only have access to the abstract of a study, the press release summary, or a news-media article on the subject, you aren’t being told the results. You’re being told how someone wants you to think about the results.
Consider this study. It whipped the news media in a frenzy, all sporting a headline like this: “New Study Shows the Female Brain is not Wired for Weight Loss!” The study was well done and interesting, but it shows no such thing, unless you’re already an obese, lazy, diabetic female mouse taking a specific medication that targets your uniquely engineered brain.
The news media is a necessary evil. We don’t have the time to read all the research that relates to us, so we rely on experts to select the relevant research and present it well, but the news media is driven by click-bait headlines and controversy, not accuracy, and there are a few red flags that should trigger your suspicion:
- A news article should link the study itself, or at least the abstract. If they actually read it, and they’re not afraid you’ll check them, why not make it easy to access?
- A news article or press release should ideally be written by a subject matter expert or qualified science correspondent rather than a general reporter. This doesn’t necessarily guarantee quality reporting, but a quality correspondent can often provide perspective on the research whereas a busy journalism major who knows little about the field is more likely to simply copy-paste what others have said on the topic .
- There are numerous resources online that review and summarize research in a particular field. If health, nutrition, and fitness are your passion or profession, it might be worth your time to sign up for one.
Finally, if a study is behind a paywall and you can only read the abstract, don’t ignore it, but take it with a grain of salt. If you get on a soapbox over an abstract don’t be surprised when the soapbox collapses underneath you.
Do the Authors Actually Ask What They’re Asking?
Just because a study claims to have tested a certain effect on a certain population does not mean the study actually tested what it says. In this study, the authors claim to evaluate the effect of their treatment on aerobic and anaerobic performance in elite athletes. A product that improved full-spectrum running performance would certainly be useful, and elite performances are much harder to improve, so a small change for elite athletes would probably translate to greater improvements in your average Joe.
However, the two tests they chose (the Yo-Yo Intermittent Run test and a 1 kilometer run), are both predominantly aerobic tests, meaning anaerobic performance wasn’t really tested. Additionally, although they tested players from a high-level soccer team, these players averaged a mediocre 4:00 1 kilometer run, so they were effectively novices for the purposes of this study.
To avoid getting caught in this trap, be wary of general, vague descriptions of either the subjects or the testing protocol. If a study says their subjects have been training for “4 years” but they are barely able to bench press their bodyweight, assess for yourself whether they qualify as “highly experienced weightlifters.”
Is It Relevant?
To translate a studied result into action, you have to consider whether that result is relevant to you and your training. One recent study has made a dramatic stir in the news for suggesting that you can gain muscle and strength equally well from lifting light or heavy weights so long as you do the same number of sets at a max effort. The testing procedures and statistical analysis appear to be excellent, but their definition of “full body, low repetition, heavy weight resistance training” is 3 sets of 8-12-rep-supersets of various machine exercises and the bench press with only one minute’s rest allowed between supersets.
Even if the weight/rep schemes were correct and the lifters were lifting to exhaustion, the authors effectively compared lightweight machine circuit training to even-lighter-weight machine circuit training, and more studies will be necessary for these results to be meaningful to actual bodybuilders, powerlifters, strongmen, and other athletes who routinely train with (truly) heavy full-body movements.
Is It Complete?
Even when a study is unbiased, accurately assessed, well defined, and relevant, it might still be incorrect. Every study is a data point. Some are more useful than others, but even the best study has to be considered in context of what has come before. Consider the research into BMI (your weight compared to your height) and its impact on mortality. Scores of studies have evaluated the connection, and a meta-analysis (combined review of the relevant research) in the Journal of the American Medical Association revealed something fascinating: people classified as ‘overweight’ and ‘class 1 obese’ (BMIs between 25-35) actually had a lower mortality than those in the “normal” range.
This is still an intensely active field of study, but the authors did us a favor and showed each study they included in a nifty forest plot. Nine studies found statistically significant increases in mortality among those classified as overweight (BMI of 25-30), but when you look at the whole picture, you see that those simply mark one end of a bell curve. Those 9 studies could have been conducted perfectly and still shown those results simply by chance. To avoid being stuck with half the picture, there are a few general rules we can follow.
First, avoid confirming your bias. If you went in looking for studies that showed a BMI of 25-30 was dangerous, you would have found plenty of evidence that ‘proved’ your point. When searching, use neutral search terms like “Correlation BMI Mortality” rather than loaded ones such as “Overweight Obese Increased Mortality Risk.”
Second, look for meta-analyses and systematic reviews of the literature. There’s nothing saying these will be correct, but they often provide a much wider picture of the current state of the research, and the discussion and references sections will often be a treasure trove of information for other places to look.
The Internet is Dark and Full of Terrors
One of the greatest revelations I’ve found in my own training is that fitness isn’t faith, diet isn’t dogma, and there’s nothing holy about becoming healthy. You don’t have to trust a guru’s word or buy an overpriced new product because it’s been “scientifically researched.” With a critical mindset, a touch of skepticism, and some experience under the bar, you can sort the trash from the treasure and maybe end up a little healthier and wealthier in the process.
And while you’re at it, make sure you’re getting your information from actual research:
1 Reason to Stop Reading Fitness Articles Based on Numerical Lists
