Evidence-Based Health: How to Think About Health Information in a World Full of Noise

We live in an era of unprecedented access to health information and unprecedented confusion about what to do with it. Wellness influencers, supplement companies, news headlines, and well-meaning friends all confidently contradict each other, leaving most people uncertain about even basic questions.

The solution is not to find better authorities to trust blindly. It is to develop a working understanding of how health evidence is generated, evaluated, and misrepresented, so you can assess claims yourself. Simple, consistent health habits can be more effective than sophisticated research literacy for most people. However, knowing how to evaluate what you read helps you protect those habits from being disrupted by every new headline.

The Hierarchy of Evidence

Not all health research is equally reliable. The scientific community uses a hierarchy of evidence to assess the strength of different study types.

Anecdote and case reports occupy the bottom. One person’s experience, even a dramatic one, tells us very little about what will happen in other people. Human bodies are variable, placebo effects are real and powerful, and our memories of cause and effect are systematically unreliable.

Observational studies track patterns in populations without intervening. They generate valuable hypotheses but cannot establish causation. They are vulnerable to confounding, reverse causation, and selection bias in ways that are difficult to fully eliminate statistically.

Randomized controlled trials (RCTs) are the gold standard for establishing causation. Participants are randomly assigned to the intervention or a control condition, which controls for confounding variables that observational studies cannot. A well-conducted RCT is the most reliable design for answering whether a specific intervention causes a specific outcome.

Systematic reviews and meta-analyses pool data from multiple independent studies to produce more statistically powerful estimates with greater external validity. A well-conducted meta-analysis of multiple high-quality RCTs represents the strongest available evidence for any health claim.

When evaluating a health claim, the first question should always be: what type of study is this based on?

References:

• Ioannidis JPA. (2005). Why most published research findings are false. PLOS Medicine, 2(8), e124. PubMed

The Difference Between Association and Causation

This is the single most important distinction in health literacy, and the one most consistently obscured in media coverage.

“People who eat vegetables live longer” does not mean vegetables cause longevity. People who eat more vegetables also tend to exercise more, smoke less, sleep better, and have higher incomes. Controlling for all these differences is statistically complex and never fully complete. Any residual confounding can produce associations that do not reflect true causation.

Reverse causation is equally common. The association between low physical activity and poor health is partly explained by sick people being less able to exercise, not only by inactivity causing poor health.

Headlines routinely present associations as causation. “Coffee linked to longer life,” “Red meat linked to cancer,” and “Screen time linked to depression” are all associations. Sometimes causation is real and eventually confirmed by experimental evidence. Often it is not. Establishing causation requires either a randomized experiment or a theoretically plausible mechanism combined with multiple converging lines of evidence from independent research groups using different methods.

Publication Bias and the Reproducibility Problem

The scientific literature has a structural problem: studies with statistically significant positive results are far more likely to be published than studies with null results. This means the published record systematically overrepresents positive findings, creating a misleadingly optimistic picture of many interventions.

John Ioannidis, in what has become one of the most cited papers in medical literature, argued that because of this bias combined with small sample sizes and low pre-study probability for many tested hypotheses, the majority of published research findings are false. The argument generated significant debate, but the underlying problem it identifies is real and widely acknowledged.

The Reproducibility Project subsequently attempted to replicate 100 studies from top psychology journals and found that only approximately 36% produced results consistent with the original findings. In nutrition and supplement research, the picture is similar or worse. Many dietary interventions and supplements that generated excitement based on initial positive studies have failed to replicate when tested by independent groups with larger and better-designed trials.

This does not mean science is unreliable. It means that single studies, especially novel or surprising ones, should be treated as preliminary evidence until replicated across multiple independent research groups.

References:

• Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. PubMed

Industry Funding and Conflicts of Interest

Research funded by industries with a financial stake in the outcome consistently produces more favorable findings for those industries than independently funded research. This pattern has been documented across pharmaceuticals, food science, and supplement research.

A particularly striking historical example: in the 1960s, the Sugar Research Foundation secretly funded a literature review published in the New England Journal of Medicine that singled out dietary fat as the primary cause of cardiovascular disease while deliberately downplaying evidence implicating sugar. The foundation set the review’s objectives, contributed articles for inclusion, and received drafts before publication. This manufactured scientific consensus influenced public health recommendations for decades. The documents exposing this manipulation only became public in 2016.

This does not mean industry-funded research is automatically wrong. It means it warrants heightened scrutiny, particularly when it contradicts independent evidence or appears in isolation without replication.

When evaluating any health claim, always ask: who funded this study? Do the researchers have financial relationships with companies that benefit from the findings? Has this finding been replicated by independent groups without a stake in the outcome?

References:

• Kearns CE, Schmidt LA, Glantz SA. (2016). Sugar industry and coronary heart disease research: a historical analysis of internal industry documents. JAMA Internal Medicine, 176(11), 1680-1685. PubMed

Relative Risk vs. Absolute Risk

One of the most common ways health claims are made to appear more impressive than they are involves presenting relative risk reductions instead of absolute risk reductions.

A drug that reduces cardiovascular events from 2% to 1% over ten years has a 50% relative risk reduction. That sounds substantial. The absolute risk reduction is 1 percentage point: one fewer event per hundred people treated for a decade. Both statements are mathematically accurate. Only one is useful for weighing whether to take the drug.

When evaluating any health intervention, always ask for the absolute numbers: how many people need to take this, for how long, to prevent one adverse outcome? This is the number needed to treat, and it places the relative risk framing in its proper context.

References:

• Gigerenzer G, Gaissmaier W, Kurz-Milcke E, Schwartz LM, Woloshin S. (2007). Helping doctors and patients make sense of health statistics. Psychological Science in the Public Interest, 8(2), 53-96. PubMed

How to Evaluate Health Claims Practically

A few questions worth applying before acting on any health claim.

What type of study is this based on? An anecdote and a meta-analysis of RCTs are not equivalent. Neither are an in vitro cell culture study and a human clinical trial.

What was actually measured? Many studies measure surrogate endpoints, biomarkers assumed to correlate with outcomes, rather than outcomes themselves. A supplement that improves a biomarker does not necessarily improve the health outcome the biomarker represents. This distinction has caused significant confusion in cardiovascular and cancer research.

How large was the effect, in absolute terms? A statistically significant result can be biologically trivial. Ask for absolute numbers, not relative ones.

Has it been replicated independently? A single study, even a well-conducted one, is preliminary evidence. Convergence across multiple independent research groups is what builds genuine confidence.

Is there a plausible biological mechanism? Biologically implausible claims deserve extra scrutiny regardless of the study design.

Who benefits from you believing this? Not a reason to dismiss a claim automatically, but a reason to apply additional critical scrutiny to the funding and incentive structure behind it.

P-Values and What Statistical Significance Actually Means

The p-value is probably the most misunderstood concept in health research, and that misunderstanding is responsible for a significant portion of the false alarms and reversals that erode public trust in science.

A p-value of 0.05, the conventional threshold for claiming “statistical significance,” does not mean there is a 95% probability that the finding is real. It means that if the null hypothesis were true (that is, if there were no real effect), there would be a 5% probability of observing results at least as extreme as what was measured. That is a statement about data under an assumed condition, not a probability that the hypothesis is correct.

This distinction matters enormously in practice. A paper published in Nature in 2014 that became one of the most widely shared scientific articles of the decade made exactly this point: p-values are neither as reliable nor as objective as most scientists assume, and their systematic misinterpretation has contaminated the published literature across multiple fields, including medicine and nutrition.

The practical consequence: with the conventional p<0.05 threshold and no other constraints, if you run twenty tests on random data, one will cross the significance threshold by chance alone. In nutrition research, where researchers often test dozens of dietary variables against dozens of health outcomes and report only those that reach significance, the false positive rate climbs dramatically. This is called p-hacking, and it is common enough to have been demonstrated across large samples of published papers.

Statistical significance is also not the same as clinical significance. A study with ten thousand participants can detect effects so small that they have no meaningful impact on a real person’s health, yet those effects can be statistically significant. Effect size matters as much as significance, and rarely makes the headline.

References:

• Nuzzo R. (2014). Scientific method: statistical errors. Nature, 506(7487), 150-152. PubMed

The Special Problems of Nutritional Epidemiology

Nutrition research deserves its own section because it is where most people’s health decisions are influenced by media, and where the gap between what headlines claim and what the evidence actually supports is widest.

The fundamental problem with nutritional epidemiology is that people eat entire diets, not single nutrients, over entire lifetimes. Isolating the effect of one dietary component from all the others, while controlling for lifestyle, income, education, and dozens of other variables, is extraordinarily difficult. The tools available, primarily food frequency questionnaires in which people try to recall what they ate over the past year, are notoriously imprecise. Error in measurement of the exposure makes associations harder to detect and, paradoxically, can sometimes create the appearance of associations that do not exist.

John Ioannidis, in a subsequent analysis of nutritional epidemiology specifically, examined whether the associations observed between single foods and mortality outcomes were plausible given what was known about effect sizes from other health interventions. His conclusion was striking: many of the associations reported in the literature would, if taken at face value, imply that diet alone is responsible for a far larger proportion of deaths than all other causes combined, which is biologically implausible. The effect sizes observed are likely substantially inflated by confounding and measurement error.

This does not mean diet does not matter. It clearly does, and the overall dietary pattern matters most. The Mediterranean dietary pattern, whole foods over ultra-processed foods, adequate fiber, and sufficient protein have enough converging evidence from multiple methodologies, including intervention studies, mechanistic research, and population data, to justify confidence. What it does mean is that specific claims about single foods or nutrients based on observational studies alone should be treated with significant caution.

References:

• Ioannidis JPA. (2013). Implausible results in human nutrition research. BMJ, 347, f6698. PubMed
• Schoenfeld JD, Ioannidis JPA. (2013). Is everything we eat associated with cancer? A systematic cookbook review. American Journal of Clinical Nutrition, 97(1), 127-134. PubMed

The Limits of Individual Optimization

A final point that I think is important from a biomedical perspective: health is probabilistic, not deterministic. You can eat well, exercise regularly, sleep adequately, manage stress, and still experience serious illness. And you can live imperfectly and reach old age in good health. Individual biology, genetics, and chance all play roles that lifestyle cannot fully override.

The goal of evidence-based health behavior is to move probabilities in favorable directions, not to guarantee outcomes. This framing is both humbling and liberating. It means taking reasonable, evidence-backed actions without becoming paralyzed by every new finding or obsessive about perfection.

The interventions with the strongest evidence across the most independent research groups are also the least exciting: consistent exercise, adequate sleep, a diet built around whole foods, stress management, and correction of documented deficiencies. These appear repeatedly because the evidence behind them is genuinely convergent, not because they happen to be fashionable.

Have a question or a topic you would like covered? Leave a comment below or get in touch.