When we’re talking fitness and athletic performance, knowing how to research a topic and develop a clear picture of the body of evidence for and against it will help keep you from being a fad follower or, worse, an uninformed trash talker.

So today, you’re going to learn how to make informed decisions about your training.

Research Isn’t Personal

A key concept is the difference between using scientific evidence, typically called evidence-based practice, and using anecdotal evidence.

Anecdotal evidence is purely observational. For example, that enormous, shredded guy at the gym swears by his pre-workout mix. Does that mean the pre-workout is that effective or that the guy has great genetics?

RELATED: A Case Study on How Supplement “Research” Actually Works

In contrast to anecdotal evidence, evidence-based practice is basing decisions on research evidence. That is, looking at a topic without preconceived notions and searching for the answer presented by the evidence.

“As you get better at reading studies, you’ll get better at reading the conclusion and going back to the results section to see if the two really line up.”

Previously, I’ve talked about how research is conducted and what to look for in a study to determine its quality. Now, we’ll look at how to find studies related to your topic and stack them against each other to really get a good picture of that topic.

Compiling Evidence for Your Argument

First, a couple of definitions. In the world of science, you have two levels of evidence, primary and secondary:

• Primary evidence is the original source that provided the evidence, meaning actual scientific studies.
• Secondary evidence is a source that references primary sources to make a statement. This would include things like textbooks, online articles, or news reports.

APPLICATION: Bands Are for Pushing, Not Pulling

If at all possible, you want to stick to primary evidence because secondary sources provide evidence using the author’s opinion of the primary resource. So, skip the middleman and go straight to the source.

Stay Current With Your Research

Assuming you’re not a subscriber to a wide variety of research journals, the best resource out there for primary sources for the general public is PubMed. This is run by the U.S. National Library of Medicine and is an index of research articles posted in every major peer-reviewed journal.

How much research are we talking? PubMed adds an average of 2,000 to 4,000 articles a day from over 5,000 sources. It added over 700,000 articles in 2013. Seriously, we’re talking about a lot of data here!

EXERCISE SCIENCE: How Do Squat Suits Improve Squat Performance?

But with huge variety comes difficulty in searching and finding what you’re looking for. For example, at the time this article was written, a search for the term “creatine” came up with 50,412 articles. You definitely need to have a good search strategy to come up with the data you actually want. Fortunately, PubMed has a great set of tutorials that fully explain how to use the site and will teach you how to build your searches to find exactly what you’re after.

“Objectively reading the evidence and making a decision based on what you read is a critical skill both inside and outside the gym.”

Once you’ve got your search terms dialed in and you have a good result list, take a look at the tool bar on the left under the heading Publication Dates. At a minimum, you should be searching for topics published within the last ten years. Sticking to five years is even better. Seriously, you’re not gaining anything by looking at old, outdated studies unless you’re looking for a specific article you want to read. If an older article is still valid, I assure you it’s being used as a reference in newer studies.

For those without a subscription to the journal containing the article you want, you’ll only be able to read the abstract. Assuming the majority of you aren’t subscribers to a large number of academic journals or are college students with full library access, we’ll discuss the data available in an abstract.

How to Read Research Abstracts

A typical abstract will include a brief overview of each of a study’s sections: the purpose, methods, results and conclusions. While the abstract is a significantly shortened version of the full article (a typical article may be ten to twenty pages while an abstract is generally 100-200 words), the basic information you need is still all there.

MORE BY THIS AUTHOR: How To Read and Analyze Research Like a Pro

The first thing to look for in an abstract is the purpose. Look carefully at the purpose statement to see if the study actually relates to your topic. Researching a topic usually involves reading several studies that are similar, but not exactly the same as your topic. Think about how the body works and if a concept is similar enough to be related back to your topic. Is a study about the increased time to fatigue in repeated sprint trials after taking creatine the same concept as a study about repeat performance of swim sprints or high-rep squats? You bet it is!

The next section is methods. Here is where you should be looking for a lot of the information we discussed in the last article. Is the study an RCT? Were the researchers blinded? How many study groups were there? Was there more than one independent variable? All of this will be found in the methods section.

The results section can be confusing and requires careful reading. This section will be full of statistics and technical jargon. If you see terms you don’t understand, take the time to look them up. Really. Do it. Learning the terms used in measurement and analysis for research will only make you better at learning how to read studies and will help you better understand the results section immediately.

Finally, there is the conclusion. This is where the author will explain what the study found as related to the purpose listed at the beginning. As you get better at reading studies, you’ll get better at reading the conclusion and going back to the results section to see if the two really line up.

Putting Together Your Argument

Now you know how to search and how to read an abstract, so how do you really research a topic? We’ve already covered the first step, which is to collect quality studies conducted as recently as possible and then compare the conclusions. But how much evidence do you need? What about the contradictions?

RELATED: Two New Studies on Static Stretching Conflict

In compiling studies on your own, you need to look for a strong majority of the studies you find to support your position. While there’s not a generally accepted percentage, many researchers look for 80% or more of the studies on a topic to support a position before they consider the evidence conclusive.

“If at all possible, you want to stick to primary evidence because secondary sources provide evidence using the author’s opinion of the primary resource.”

Another tool at your disposal is review studies. Typically titled as a meta-analysis or literature review, these are studies that compile as many studies related to a topic as possible into one article and compare the methods and results to find common conclusions. This is a great way to get a feeling for the whole body of evidence about a topic and can save you a ton of work.

Always Be Open to Learning

Finally, remember to be objective. Keep an open mind and learn about the topic before making a decision. It’s human nature to look for evidence that supports your opinion and to disregard evidence to the contrary. Objectively reading the evidence and making a decision based on what you read is a critical skill both inside and outside the gym.

Keep in mind that the human body is amazingly complicated and we’re learning more about how it responds to exercise and nutrition every day. Don’t get stuck in one set of beliefs because that’s the way you originally learned it. Keep reading. Keep learning. And by all means, keep having productive, informed debates.

Photos courtesy of Shutterstock.

The post Don’t Just Argue, Debate Like a Scientist appeared first on Breaking Muscle.

In this article, I’ll cover some of the basic concepts about how a study is designed and how to separate the good from the bad.

Internal Validity and External Validity

Judging the quality of a study consists of examining two areas: internal validity and external validity. Internal validity refers to the controls of a study. External validity refers to whether or not the results of that study can be projected back onto a larger population.

Internal validity is primarily built through accurate measurement instruments (both the equipment and type of test) and control of confounding factors. A confounding factor is any unaccounted for item that caused the effect you observed in your study. Did that new supplement increase the subject’s the back squat weight or was it something else, like the training program the subject used or a change in diet?

RELATED: A Lesson in Study Design (And the Bench Press)

External validity is where you need to examine who the subjects were in the study. Were they all men? All women? Elite athletes? Novices? Old? Young? The demographic of the subjects is incredibly important. Look closely at who was included in the study and ask if those subjects represent you or your athletes. The less varied the demographic of the subjects, the smaller the population the results apply to.

What Do Studies Do?

Now we know these concepts, but how in the world do we figure out the level of internal and external validity in the study we are reading? To start, let’s go back to the basics of what a study is and what it does.

A scientific study tries to answer a question by creating a situation that can be observed and measured. In health and fitness, we are mostly looking to observe cause and effect. This caused that. Did taking creatine increase your one-rep-max (1RM) back squat? Did having that last beer make you that much more awesome? I don’t know how many beers you had, so we’ll use a fake study about creatine and increasing the 1RM back squat as our example.

“An important part of understanding studies is to never, ever believe that just because a study found such-and-such then that is now a fact.”

The most basic step in designing a study is developing a dependent variable and an independent variable. In our study, the weight of back squat is the dependent variable and the amount of creatine taken is the independent variable. The researcher determines the amount of creatine the participants will take in the study design and then measures back squat weights to determine if they changed. That means the this is the independent variable and the that is the dependent variable in your question of this caused that.

The Role of Bias

One of the most important factors that can affect the quality of a study is bias. Bias is a situation where either the researcher did not account for something that affected the results or subjects were selected that were more likely to have a specific outcome. Bias affects both internal and external validity. Not accounting for something that affected the results is a confounding factor, which reduces internal validity. Selection bias is tremendously important in health and fitness research and affects external validity.

The Importance of the Randomized Controlled Trial

Another important aspect of study quality is the type of study conducted. While there are several types of studies, including observational and case studies, the only true way to determine cause and effect is using a randomized controlled trial, or RCT. Other types of studies are nice for presenting data, but to say this caused that, you have to use an RCT. Yes, that is a must.

“The goal of a good study design is to change just one factor, the independent variable, and keep everything else between the study groups as similar as absolutely possible.”

An RCT is the necessary study design for showing cause and effect because it reduces selection bias. Meaning, every subject in the study has an equal chance of being selected for either group. In an RCT, you collect your test subjects and randomly assign them into two or more groups. At a minimum, you must have a test group and a control group. In our fictional study, the test group would take creatine and the control group wouldn’t take any supplements. The goal of a good study design is to change just one factor, the independent variable, and keep everything else between the study groups as similar as absolutely possible.

The one exception to this is for review studies, which include literature reviews and meta-analyses. In these studies, the author compares as many studies as can be found that relate to a certain topic and analyzes the study designs and results. This is a great way to get a picture of a much larger body of evidence about that topic.

Studies Don’t Prove Anything

It is important to understand that scientific studies don’t prove anything. They suggest. They provide supporting evidence. An important part of understanding studies is to never, ever believe that just because a study found such-and-such then that is now a fact.

“You need to examine everything closely, ask hard questions, and be a difficult sell on any idea.”

Sometimes researchers are biased and interpret results incorrectly. Sometimes there’s a confounding factor that wasn’t accounted for. Sometimes you just can’t figure out why the results were the way they were. Look for multiple studies that support the hypothesis. For extra credit, look for studies that don’t support that hypothesis and compare them with the ones that do.

Guidelines to Help You Get the Most From Reading Studies

• Type of study. As stated above, the study has to be an RCT. An abstract won’t always come straight out and call a study an RCT, so look for the key word randomized. If you can’t find it, it’s not an RCT and the results aren’t worth your time.
• Were the researchers blinded? Single blinding means the researchers did not know whether the participants they were testing belonged to the test or control group and is an absolute requirement for reducing bias and establishing internal validity. Like using an RCT, this is a necessity and the results of a study that doesn’t use it shouldn’t be trusted.
• Who were the participants? Remember, we’re looking for good external validity here. Does the study population represent the general public, you or your athletes, or a group that you don’t even work with? If the participants don’t sound like you or your athletes, the results won’t necessarily apply to you.
• What was the level of training of the participants? One of the golden rules of exercise research is to never, ever use novice athletes. Novice athletes will respond positively to just about anything. Seriously, never buy into the results of a study that used novice athletes.
• What is the difference between the test and control groups? This is going to tell you what the study actually examined, and it’s not uncommon for a poorly designed study to end with the results reflecting something completely different from what was intended. Was the topic of the study the back squat, but the results were determined by testing leg press or leg extension weight? Seriously, this stuff is out there.

This all may sound a bit cynical, but in all honesty, that’s the scientific way. You need to examine everything closely, ask hard questions, and be a difficult sell on any idea. To do anything else is to be a sucker for every fad that comes along. Practice looking at studies, asking these questions, and thinking critically about the quality of the study and meaning of the results. Keep at it and you’ll be crunching through studies like a pro in no time.

READ: Correlation and Causation: What Pubs Have to Do With Your Clean and Jerk

There is much more to truly understanding how to read scientific studies, so this article is only an overview to get you started. Have a question? Ask it in the comments!

Photos courtesy of Shutterstock.

The post How to Read and Analyze Research Like a Pro appeared first on Breaking Muscle.