We live in an era of unprecedented access to health information and unprecedented confusion about what to do with it. Wellness influencers, supplement companies, news headlines, and well-meaning friends all confidently contradict each other, leaving most people uncertain about even basic questions.

Scientist in a white lab coat reviewing research papers next to a microscope in a laboratory setting
Producing reliable health evidence is a slow, painstaking process. Evaluating it critically requires understanding the specific limitations of each study design, something that rarely makes it into the headline.

The solution is not to find better authorities to trust blindly. It is to develop a working understanding of how health evidence is generated, evaluated, and misrepresented, so you can assess claims yourself. Simple, consistent health habits can be more effective than sophisticated research literacy for most people. However, knowing how to evaluate what you read helps you protect those habits from being disrupted by every new headline.

The Hierarchy of Evidence

Not all health research is equally reliable. The scientific community uses a hierarchy of evidence to assess the strength of different study types.

Pyramid diagram showing the hierarchy of evidence from anecdote at the bottom to meta-analyses of RCTs at the top
The hierarchy of evidence is not about dismissing lower-level research. Observational studies and case reports generate important hypotheses. The pyramid reflects how much confidence each design justifies for causal conclusions, not how useful each type is for generating ideas.

Anecdote and case reports occupy the bottom. One person’s experience, even a dramatic one, tells us very little about what will happen in other people. Human bodies are variable, placebo effects are real and powerful, and our memories of cause and effect are systematically unreliable.

Observational studies track patterns in populations without intervening. They generate valuable hypotheses but cannot establish causation. They are vulnerable to confounding, reverse causation, and selection bias in ways that are difficult to fully eliminate statistically.

Randomized controlled trials (RCTs) are the gold standard for establishing causation. Participants are randomly assigned to the intervention or a control condition, which controls for confounding variables that observational studies cannot. A well-conducted RCT is the most reliable design for answering whether a specific intervention causes a specific outcome.

Systematic reviews and meta-analyses pool data from multiple independent studies to produce more statistically powerful estimates with greater external validity. A well-conducted meta-analysis of multiple high-quality RCTs represents the strongest available evidence for any health claim.

When evaluating a health claim, the first question should always be: what type of study is this based on?

References:

  • Ioannidis JPA. (2005). Why most published research findings are false. PLOS Medicine, 2(8), e124. PubMed

The Difference Between Association and Causation

This is the single most important distinction in health literacy, and the one most consistently obscured in media coverage.

Diagram showing three reasons two variables can be correlated: real causation, confounding by a third variable, and reverse causation
Two things can be correlated for three fundamentally different reasons. Only RCTs reliably distinguish between them, because random assignment breaks the link between the treatment and all other variables that might explain the outcome.

“People who eat vegetables live longer” does not mean vegetables cause longevity. People who eat more vegetables also tend to exercise more, smoke less, sleep better, and have higher incomes. Controlling for all these differences is statistically complex and never fully complete. Any residual confounding can produce associations that do not reflect true causation.

Reverse causation is equally common. The association between low physical activity and poor health is partly explained by sick people being less able to exercise, not only by inactivity causing poor health.

Headlines routinely present associations as causation. “Coffee linked to longer life,” “Red meat linked to cancer,” and “Screen time linked to depression” are all associations. Sometimes causation is real and eventually confirmed by experimental evidence. Often it is not. Establishing causation requires either a randomized experiment or a theoretically plausible mechanism combined with multiple converging lines of evidence from independent research groups using different methods.

Publication Bias and the Reproducibility Problem

The scientific literature has a structural problem: studies with statistically significant positive results are far more likely to be published than studies with null results. This means the published record systematically overrepresents positive findings, creating a misleadingly optimistic picture of many interventions.

John Ioannidis, in what has become one of the most cited papers in medical literature, argued that because of this bias combined with small sample sizes and low pre-study probability for many tested hypotheses, the majority of published research findings are false. The argument generated significant debate, but the underlying problem it identifies is real and widely acknowledged.

The Reproducibility Project subsequently attempted to replicate 100 studies from top psychology journals and found that only approximately 36% produced results consistent with the original findings. In nutrition and supplement research, the picture is similar or worse. Many dietary interventions and supplements that generated excitement based on initial positive studies have failed to replicate when tested by independent groups with larger and better-designed trials.

This does not mean science is unreliable. It means that single studies, especially novel or surprising ones, should be treated as preliminary evidence until replicated across multiple independent research groups.

References:

  • Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. PubMed

Industry Funding and Conflicts of Interest

Research funded by industries with a financial stake in the outcome consistently produces more favorable findings for those industries than independently funded research. This pattern has been documented across pharmaceuticals, food science, and supplement research.

Multiple blister packs of pharmaceutical pills and capsules of different colors spread across a white surface
The pharmaceutical and supplement industries invest billions annually in research. Industry funding does not automatically invalidate findings, but it is consistently associated with results more favorable to the sponsor than independently funded studies on the same intervention.

A particularly striking historical example: in the 1960s, the Sugar Research Foundation secretly funded a literature review published in the New England Journal of Medicine that singled out dietary fat as the primary cause of cardiovascular disease while deliberately downplaying evidence implicating sugar. The foundation set the review’s objectives, contributed articles for inclusion, and received drafts before publication. This manufactured scientific consensus influenced public health recommendations for decades. The documents exposing this manipulation only became public in 2016.

This does not mean industry-funded research is automatically wrong. It means it warrants heightened scrutiny, particularly when it contradicts independent evidence or appears in isolation without replication.

When evaluating any health claim, always ask: who funded this study? Do the researchers have financial relationships with companies that benefit from the findings? Has this finding been replicated by independent groups without a stake in the outcome?

References:

  • Kearns CE, Schmidt LA, Glantz SA. (2016). Sugar industry and coronary heart disease research: a historical analysis of internal industry documents. JAMA Internal Medicine, 176(11), 1680-1685. PubMed

Relative Risk vs. Absolute Risk

One of the most common ways health claims are made to appear more impressive than they are involves presenting relative risk reductions instead of absolute risk reductions.

Diagram comparing relative risk framing versus absolute risk framing using the same data: a 50% relative risk reduction versus a 0.1 percentage point absolute risk reduction
The same data presented two ways. "50% risk reduction" and "risk reduced from 0.2% to 0.1%" are mathematically identical but create entirely different impressions. Headlines almost always use relative risk. The number that matters for individual decision-making is almost always absolute risk.

A drug that reduces cardiovascular events from 2% to 1% over ten years has a 50% relative risk reduction. That sounds substantial. The absolute risk reduction is 1 percentage point: one fewer event per hundred people treated for a decade. Both statements are mathematically accurate. Only one is useful for weighing whether to take the drug.

When evaluating any health intervention, always ask for the absolute numbers: how many people need to take this, for how long, to prevent one adverse outcome? This is the number needed to treat, and it places the relative risk framing in its proper context.

How to Evaluate Health Claims Practically

Healthcare professional in blue scrubs reading a medical anatomy textbook at a desk with a digital reader nearby
Reading primary sources rather than relying on secondary summaries changes how you understand health claims. Most of what gets lost between a published study and a news headline is precisely the information you need to evaluate it.

A few questions worth applying before acting on any health claim.

What type of study is this based on? An anecdote and a meta-analysis of RCTs are not equivalent. Neither are an in vitro cell culture study and a human clinical trial.

What was actually measured? Many studies measure surrogate endpoints, biomarkers assumed to correlate with outcomes, rather than outcomes themselves. A supplement that improves a biomarker does not necessarily improve the health outcome the biomarker represents. This distinction has caused significant confusion in cardiovascular and cancer research.

How large was the effect, in absolute terms? A statistically significant result can be biologically trivial. Ask for absolute numbers, not relative ones.

Has it been replicated independently? A single study, even a well-conducted one, is preliminary evidence. Convergence across multiple independent research groups is what builds genuine confidence.

Is there a plausible biological mechanism? Biologically implausible claims deserve extra scrutiny regardless of the study design.

Who benefits from you believing this? Not a reason to dismiss a claim automatically, but a reason to apply additional critical scrutiny to the funding and incentive structure behind it.

The Limits of Individual Optimization

A final point that I think is important from a biomedical perspective: health is probabilistic, not deterministic. You can eat well, exercise regularly, sleep adequately, manage stress, and still experience serious illness. And you can live imperfectly and reach old age in good health. Individual biology, genetics, and chance all play roles that lifestyle cannot fully override.

The goal of evidence-based health behavior is to move probabilities in favorable directions, not to guarantee outcomes. This framing is both humbling and liberating. It means taking reasonable, evidence-backed actions without becoming paralyzed by every new finding or obsessive about perfection.

The interventions with the strongest evidence across the most independent research groups are also the least exciting: consistent exercise, adequate sleep, a diet built around whole foods, stress management, and correction of documented deficiencies. These appear repeatedly because the evidence behind them is genuinely convergent, not because they happen to be fashionable.

🔬 Looking for supplements with real evidence behind them? The products we recommend on this site were selected because they meet the standards described in this post: replicated evidence from independent research groups, plausible biological mechanisms, and transparent funding. No proprietary blends with underdosed ingredients.

See Our Recommended Products →

Have a question or a topic you would like covered? Leave a comment below or get in touch.