It seems like everywhere you turn there is a different nutrition story making headlines. Whether in mainstream media or on social channels, consumers are inundated with information on the latest and greatest study often telling you to eat this or avoid that. But how do you know if a study is based on reliable data, anecdotal information, or simply a sensationalized headline? One part of the problem is that nutrition science is not linear: it is complex and often oversimplified when reported because it is ever-changing and involves multiple factors that are sometimes difficult to control and track. Regardless, there are things you can do to unravel fact from fiction and navigate the tide of information. Here are the top eight things you should look for when evaluating nutrition research:
- Source of the Study – First, make sure the study was published in a reputable, peer-reviewed scientific journal. Peer review is extremely important because it involves experts digging into the data to ensure the conclusions are valid. Studies that are not peer-reviewed or unpublished data are not considered trustworthy. To double check, visit pubmed.gov. Valid scientific research can be published by any number of organizations including government agencies, academic institutions, nonprofits, and food and beverage companies. Typical credible scientific websites end in .edu, .gov, and .org. It’s important to notice the date of the study as well.
- Check for References – Look at who conducted the study. Generally, you want credentialed nutrition scientists with a PhD, RD, or DSc, and with experience in the area they are researching.
- Population – Nutrition studies can be done on humans (clinical or population based), in animals (all types), or in cells, sometimes called in vitro, in a lab. It is very difficult and in most cases not appropriate to make assumptions about humans from animal or cell research. It is important to always look for human and/or population studies first.
- Study Type and Size – First, it is important to check the size of the study and study type. Often the sample size of a study is referred to as “n”. Be aware that most nutrition studies are observational in nature. Observational studies are large in size and easier to conduct but are subject to confounding (conflicts in the results) while clinical or control studies are usually smaller, more challenging to conduct, and have less risk of confounding. There are:
- Randomized Controlled Trials (RCT): studies that measure the effectiveness of a new intervention or treatment and are considered the gold standard for nutrition research. However, conducting them is expensive and requires a high level of scientific rigor.
- Cohort studies: a type of longitudinal study that tracks a defined population of research participants over a period of time.
- Cross-sectional studies: a type of observational study that collects and evaluates data from many individuals at a single point in time.
- Case control studies: observational study in which two groups with different health outcomes or disease states (i.e., one with heart disease and one without) are identified and compared related to diet.
- Systematic reviews: summaries of the scientific literature already published on a specific topic.
- Correlation vs. Causation – It is important to note that most nutrition studies are observational in nature and therefore can only point out correlations. Observational studies do not prove causation or cause and effect. Consequently, a certain food or diet may be associated with a specific outcome or disease, but it is unlikely a research study will prove causation. This is largely because many other factors may influence the outcome. If a conclusion is drawn, be sure to identify whether the finding was statistically significant. Please keep this in mind when reading research.
- Data Collection – Nutrition research often relies on 24-hour dietary recalls or food frequency questionnaires. This type of data has a high rate of error because test subjects can pick what they want to report or omit. The best studies look at large populations and control for many factors to minimize error. Be sure the data is comparing similar items in the control versus the intervention/experimental group.
- Communicated without Bias – All studies should evaluate both the potential strengths and weaknesses of the research, which should lead to a further study in the future. Beware that the researcher is not overstating the positive or negative results of a study. Most studies should have a clear next step or research project. Look for research that is communicated with balance and accuracy. Researchers should identify possible study design flaws in their discussion section so readers understand the potential for inaccurate results.
- The Body of Literature – Nutrition recommendations are based on the body of evidence, which includes both observational and intervention studies as well as human (population-based and single studies), animal, and cell data. What does the totality of the literature say? If only one study contradicts the weight of the evidence, carefully review before jumping to conclusions.