Meaning of Scientific Studies: How to Understand What They Are?

Beware of headlines that make definitive claims: “Antioxidants block aging,” “Vitamin C prevents cancer,” and so on.

The media sometimes fail to put a study in context by explaining the type of research reported. Any exciting new finding may be presented as definitive, especially if you just skim the headlines. You may be told little about the quality of the research. Moreover, when it comes to diet and nutrition, each type of study faces a special set of challenges and limitations. Here are some of the main study types.

Laboratory studies are done in test tubes or animals. They can suggest how and why a vitamin, mineral, or phytochemical might work, but the findings don’t automatically translate to people and do not provide strong enough evidence to make changes in dietary habits.

Observational studies (also called epidemiological studies) are done in large populations of people— sometimes 100,000 or more—and can run for decades. Scientists collect data of many types at regular intervals as participants simply live their lives. By comparing those who stay healthy with those who fall ill, researchers try to identify factors that could account for differences. Such studies can be very powerful, since they follow what people actually do in their lives over many years. But they cannot prove cause and effect, only correlations. For example, people who eat blueberries might be healthier because they live healthier lives in general, not because they eat blueberries. Scientists try to adjust for such “confounding” factors, but still cannot draw firm conclusions. Two common observational study designs are case-control and cohort studies, each with its own limitations.

• Case-control studies first identify people with a particular condition (the cases), then select similar usually healthy people who don’t have that condition (the controls), and ask all these people about factors in the past that may be related to the condition of interest. These studies may be tainted by problems called “selection bias” or “recall bias.” Selection bias means that the controls aren’t similar enough to the cases to provide a good comparison. Recall bias means the individuals may not recall their behavior correctly. For example, when ill participants are asked to describe their diet, they often recall a worse diet than the one they actually followed. Healthier folks, meanwhile, tend to see their diets through rose-colored glasses.

• Cohort studies begin when researchers identify a large group of people who are initially free of a particular condition, ask about current factors of interest, and then follow them over time to see who develops the condition and who does not. Because participants are routinely questioned about their diets and lifestyles before developing the condition in question, cohort studies are the strongest observational design. However, because they are very large, they often rely on questionnaires to assess diet—which can be subjective and lead to an underestimate of the true size of the effect. Examples of these studies include the Nurses’ Health Study and the Health Professionals Follow-up Study.

Metabolic studies typically involve a small number of volunteers who eat specially prepared meals for short time periods and are tested at regular intervals. These studies are rigorous and closely controlled, but are too brief to show actual effects on disease over the long term. Instead, researchers use them to track changes in risk factors or biomarkers, such as high blood pressure, high cholesterol levels, elevated blood glucose, or inflammation.

Randomized controlled trials are typically considered the gold standard of scientific research. The researchers randomly assign people to receive either an active treatment (such as a vitamin supplement) or a placebo—a pill or other intervention that looks the same as the active treatment but doesn’t have an effect on the body. These types of studies can directly test whether the treatment makes a difference. But they, too, have their problems where nutrition is concerned. To begin with, they may be too short to observe a nutrient’s long-term consequences. For example, in the Harvard-based Physicians’ Health Study, beta carotene didn’t appear to have any effect on cognition at the 12-year mark, but at 18 years, benefits were detected. Also, these studies may involve participants who are in better or worse health than you are, so they may not yield information that is relevant to you. Some of the research that did not find an effect for vitamin or mineral supplements, for instance, had come from studies that enrolled people who had existing heart disease or diabetes; it’s not clear how well they apply to healthier people. In addition, because of limited knowledge about a disease (for instance, cancer), randomized controlled trials may not introduce a dietary intervention when it could have done the most good; participants may already be so ill that the change may come too late to make a meaningful difference in the outcome. Lastly, these trials can be extremely difficult to conduct because researchers have to control the behaviors of large numbers of people, often for many years. Some people may not take their supplements as prescribed, may try other treatments as well, or may drop out of the study. If participants do not stay with the assigned treatment, findings could be blurred or even eradicated.

Meta-analysis is a statistical strategy that identifies previously published studies containing comparable data and combines the evidence together, looking for patterns that corroborate or contradict the findings from the individual studies. The main output is a single overall estimated effect, based on all of the evidence combined—in effect, it’s a study of studies. A key strength of meta-analysis is the ability to combine data from multiple previously published studies, usually across different populations and countries, to derive a best overall estimate of effects. A potential limitation is that the truth of the final estimate depends almost entirely on the quality of the individual studies that go into the meta-analysis: if good studies are included, a good meta-analysis results, but garbage in, garbage out is another possibility as well. Be sure to look beyond the headlines for more information. Reputable sources of health information try to put key studies in perspective. And check with your doctor before making major dietary changes.

Source: pgs. 14-15, Making Sense of  Vitamins and Minerals: Choosing the foods and nutrients you need to stay healthy.