Skip to content
Tutorials Center
My learning

Topics

Lesson 3 of 47 ยท Getting Started

How we know: the evidence behind food claims

Learn how nutrition studies are designed, why confounding and bad memory complicate them, and how to judge the strength of a food claim.

"Coffee prevents cancer." "Coffee causes cancer." Both headlines have appeared, sometimes in the same year, and both were based on real studies. Understanding how that can happen is the most useful skill in nutrition. It is not that scientists are careless. It is that different study designs answer different questions, and each one can be fooled in its own way.

This lesson introduces the main designs, the problems that haunt them, and a short checklist for reading any claim. You will also run a small simulation in which a food with no effect at all appears to protect against disease.

A five-layer pyramid ranking study designs from systematic reviews at the top to animal and cell studies at the base
Caption: the evidence pyramid. Designs higher up control more of the things that can mislead, but the pyramid is a guide, not a law: a well-done cohort study can beat a poor trial.

What you will learn

  • How randomized trials, cohort, case-control and cross-sectional studies differ.
  • What confounding, bias and measurement error are, with examples from food research.
  • How relative risk and absolute risk can tell very different stories.
  • A checklist for judging a food claim in the news.

Key terms

  • Randomized controlled trial (RCT): participants are assigned to diets or supplements by chance and followed, so groups should differ only by the intervention.
  • Cohort study: a group of people is followed over time and outcomes are compared between those with different exposures, such as high and low fiber intake.
  • Confounding: a third factor that is linked to both the exposure and the outcome and distorts their apparent relationship.
  • Relative risk (RR): the risk in one group divided by the risk in another.
  • Biomarker: a measurable body signal, such as blood level of a nutrient, used to assess intake or health.

The main study designs

Randomized controlled trials are the best design for showing that something causes an effect, because random assignment balances known and unknown factors between groups. Nutrition trials can be run with supplements (a pill against a placebo, ideally double-blind) or with whole diets (a Mediterranean pattern against a low-fat pattern). Their limits are practical: people cannot be blinded to eating a diet, adherence drifts, and trials last months or a few years, which is short for diseases that take decades. Many trials therefore measure risk factors such as blood pressure or LDL cholesterol rather than heart attacks.

Prospective cohort studies enroll people who are healthy, measure diet (often with a food frequency questionnaire), and follow them for years. They can be large and long, which is why most evidence on diet and chronic disease comes from them. Famous examples include the Framingham Heart Study and the Nurses' Health Studies. They cannot prove cause, because people who eat differently also live differently.

Case-control studies start with people who already have a disease, find similar people without it, and ask about past diet. They are quick and suit rare diseases, but memory is imperfect and people with illness may recall differently (recall bias).

Cross-sectional studies and surveys measure diet and health at one moment. They describe, for example, how many children are anemic, but cannot tell what came first. Ecological studies compare countries or regions and are prone to misleading patterns.

Animal and cell studies reveal mechanisms and plausibility. They are essential, but a mouse is not a person, and doses in the lab are often far above those in diets.

Systematic reviews and meta-analyses collect all studies on a question using a stated method and, in a meta-analysis, pool the numbers. They are only as good as the studies inside them.

What can go wrong

Confounding. People who eat a lot of vegetables also tend to smoke less, exercise more, have more money and see doctors earlier. This is the healthy-user effect. If you compare vegetable eaters and non-eaters, those other differences can masquerade as the effect of vegetables. Researchers adjust statistically for known confounders, but unmeasured ones remain. This is why randomized trials matter.

Measurement error. Food frequency questionnaires ask people to remember a year of eating. People under-report snacks, over-report vegetables, and cannot estimate portions well. Studies using urinary nitrogen or doubly labelled water show that self-reported energy intake is often lower than true intake, especially in people with higher body weight. Measurement error usually weakens real associations, so some effects are hidden.

Reverse causation. People often change diet because they are ill: someone with early bowel disease may avoid roughage, making fiber look harmful. Starting follow-up after a lag, and excluding early cases, helps.

Publication and funding bias. Positive findings are more likely to be published, and industry-funded studies tend to favor the sponsor's products. Pre-registration of trials helps.

Surrogate outcomes. A diet that lowers LDL cholesterol has not necessarily been shown to prevent heart attacks; it makes it likely, if the cholesterol pathway is the cause.

Absolute and relative risk

Suppose 2 in 1,000 people who do not eat food X develop a disease in ten years, against 3 in 1,000 who eat it. The relative risk is 3/2 = 1.5: a "50% higher risk", a frightening headline. The absolute risk difference is 1 in 1,000, or 0.1 percentage point. The number needed to harm is 1,000 (1 extra case per 1,000 people exposed). Both are true; only the second tells you how much it matters. Always ask for the absolute numbers and the size of the group.

Weighing a claim: a checklist

Ask these questions in order:

  1. What type of study is it? Trial, cohort, or animal? Humans, or cells?
  2. How many people, for how long? Ten people for two weeks says little about disease.
  3. Compared with what? Replacing butter with olive oil is different from adding olive oil to a poor diet.
  4. How big is the effect in absolute terms?
  5. Is it consistent with other studies and with a plausible mechanism? One study is a hint, many are a pattern.
  6. Who paid, and who gains?
  7. Does the headline match the paper? Many do not.

A good one-line summary of an observational study is: "People who ate more X had a lower or higher risk of Y, after adjusting for Z." It is not "X prevents Y".

Python Confounding: a food that does nothing looks protective

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Relative risk versus absolute risk

Common misconceptions

  • "Observational studies are worthless." They are indispensable for long-term effects and harms you cannot test ethically. They just need careful interpretation and, when possible, support from trials and mechanisms.
  • "A randomized trial is always right." Trials can be too small, too short, poorly adhered to, or test an unrealistic dose. Quality matters, not just design.
  • "A 'significant' result is a big result." Statistical significance says a result is unlikely to be chance; it says nothing about size or importance.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Which design is best at showing that a diet causes a change in blood pressure?

    Random assignment balances known and unknown factors between groups, so differences can be attributed to the intervention.
  2. 2. People who eat more whole grains also exercise more. This is an example of

    Exercise is linked to both whole grain intake and health, so it can distort the apparent effect of whole grains.
  3. 3. Disease risk rises from 2 to 3 per 1,000. What is the relative risk and the absolute increase?

    3/2 = 1.5 is the relative risk, and 3 - 2 = 1 per 1,000 is the absolute difference.
  4. 4. Why do case-control studies suffer from recall bias?

    Diet is reported after the disease is known, so memories may be influenced by illness.
  5. 5. Which is a surrogate outcome?

    LDL cholesterol stands in for heart disease risk; it is a measurable marker, not the disease itself.

Summary

  • Different designs answer different questions: trials test causes, cohorts follow people over time, case-control and cross-sectional studies give faster but weaker clues, and mechanism studies explain how.
  • Confounding, bad memory, reverse causation and funding can all mislead.
  • Relative risk can sound dramatic while the absolute risk is tiny; always ask for both.
  • Judge a claim by design, size, comparison, effect size, consistency and source.

Next: Reference intakes and food labels