What Can You Trust?

Key Points

  • The burden of proof of a therapy or approach can vary according to the relative risks and benefits.
  • Although some study designs are considered stronger than others, interpreting study results involves assessing the trade-offs between highly controlled situations and relevance to real life, then using the evidence that makes the most sense in the situation.
  • Therapies that are not especially dangerous and that have credible evidence that they may be helpful do not need as much proof of benefit as therapies that involve more risk or expense.
  • When assessing the reliability of a book, website or other resource, consider the authors' qualifications and financial ties,
  • BCCT recommends checking sources and validating information obtained from sources with any financial stake in therapies.

Search the Internet for “cancer therapy” and you will find tens of millions of webpages. How can you sort through to find the few pages that will be most valuable for you? How can you possibly know which websites or information to trust? How do you analyze and find clarity when "experts” give opposite recommendations? How can you tell the "snake oil” from truly valuable therapies?

First, please understand that BCCT does not claim to have all the answers. As we pull together our information and summaries, we struggle with these same issues: Is this source reliable? Is this research valid? Is this expert qualified to speak about this therapy?

However, even as we are always learning ourselves, we share with you some guidelines for determining whether a website or article rates higher or lower on a trust scale. You may also want to read about our approach to integrative cancer therapies.

Nancy Hepp

Type and Strength of Evidence

Highlighted Videos

BCCT project manager Nancy Hepp, MS, explains what evidence BCCT summaries are based on.

BCCT advisor Donald Abrams, MD, describes the difference between "evidence-based" and "evidence-informed" approaches in a 2014 presentation.

BCCT advisor Mark Renneker, MD, explains how to get accurate information about cancer.

Evidence Trade-offs

Evidence that a therapy “works” runs a whole range from completely unreliable to trustworthy. We present an overview of some of the issues in determining whether evidence is reliable and appropriate, first from a researcher's viewpoint, and then from a clinician's frame.

The Researcher's View: Hierarchy of Evidence

We present 12 levels in a research hierarchy of strength of evidence, starting with unreliable evidence and working down through increasingly credible sources of evidence (all examples are fictional). The evidence toward the bottom of this list (larger numbers) is generally regarded as stronger and more reliable than evidence toward the top of the list.

1. Nonspecific testimonials

Read more

2. Specific testimonials

Read more

3. Clinical or expert testimonial

Read more

Cells vs. People

Studies on human cells can be helpful in finding effects of drugs, radiation, natural compounds and other potential therapies on tumors. However, isolated cells or tissues in a highly controlled lab may behave very differently from tumors and other cells in real human beings.

Read more

4. Cell studies, usually with human tumor cells

Read more

5. Clinical observations, also called case studies

Read more

6. Retrospective observational studies

Read more

7. Prospective observational studies

Read more

8. Experimental studies with animals

Read more

9. Small prospective, experimental clinical studies, sometimes called “pilot studies”

Blinded Trials

The practice of “blinding” trials of treatments—so that patients are not aware of which treatment they receive—is held up as a hallmark of rigorous research. Blinding the physicians or practitioners to the treatment (a “double-blind study”) is considered even better.1

Read more

Read more

10. Large prospective, experimental clinical studies

Read more

11. Review of groups of studies

Read more

12. Meta-analysis of several studies

Read more

13. Clinical practice guidelines

Read more

Observational studies that come closer to real-life situations may provide more valuable information for clinical use than studies that are considered the “gold standard.”

The Clinician's View: Therapies in the Real World

Progression-free Survival

In assessing the effectiveness of cancer treatments, the outcome being measured can make a big difference in interpretation. For example, studies reporting only progression-free survival (PFS) may find effectiveness, yet many patients do not perceive much benefit.

Read more

The Gold Standard: Why Randomized Controlled Trials Don't Always Tell the Real-World Story

Many oncologists rely heavily on the outcomes of "gold standard" randomized clinical trials (RCTs) in recommending therapies to patients. Some researchers tell a more complex story. RCTs try to control all variables, but in the real world, outcomes may be different. That's why observational studies—what clinicians see in practice—actually matter.

GRADE (Grading of Recommendations Assessment, Development, and Evaluation) is a formal, scientific rating system. Research studies are rated according to the strength of the study design and the rigor of their conclusions. Similar to the hierarchy of evidence above, GRADE prioritizes randomized controlled trials but also considers information from observational studies when necessary.

Read more

The Evidence House: Valuing What the Physician Sees in Practice

BCCT advisor Wayne Jonas, MD, explains that the strongest study designs don’t always provide the best evidence:10

As most clinicians know, the reasons that patients recover from illness are complex and synergistic, and many cannot simply be isolated in controlled environments. The best evidence under these circumstances may be observational data from clinical practice that can estimate the likelihood of a patient's recovery in a realistic context.

Read more

Evidence House graphic
Evidence House illustration; click to open the full journal article

Jonas has proposed an “evidence house” with different “rooms” and different “wings” for different audiences and purposes:

  • Rooms in one wing of the house contain types of scientific information such as laboratory research. These rooms seek to find causes of disease, how therapies work, and proof of effectiveness—types of information that can be difficult to determine or that may take many decades.
  • Another wing has rooms with information about therapies’ relevance and usefulness in clinical practice rather than absolute proof of effectiveness.

Read more

"The Lower the Risk of Harm, the Lower the Burden of Proof"

Experimental therapies both in mainstream cancer medicine and in complementary medicine can be considered science-informed rather than fully science-based. Because the scientific process can be slow to accumulate enough evidence to be conclusive, and because cancer patients often don’t have decades to wait for rigorous research results, science-informed therapies are often the only option beyond standard therapies. These might include therapies supported by case studies and observational studies, or for which a strong theoretical rationale exists but empirical studies are unavailable, incomplete or inconclusive.

Therapies that are not especially dangerous and that have credible evidence that they may be helpful do not need as much proof of benefit as therapies that involve more risk or expense.

BCCT advisor Dr. Donald Abrams makes another critical point with respect to science-informed therapies: The lower the risk of harm, the lower the burden of proof. The burden of proof is lower if a therapy meets these criteria:

  1. It is unlikely to do harm.
  2. The patient considers it affordable.
  3. The patient is drawn to it or believes it may have value.

Therapies that are not especially dangerous and that have credible evidence that they may be helpful do not need as much proof of benefit as therapies that involve more risk or expense. BCCT views the use of science-informed, low-risk, affordable therapies as a reasonable option for patients. Stronger evidence of benefit is needed for therapies that are risky, expensive or otherwise burdensome.

Financial Ties

Websites or people who are trying to sell something have an incentive to make their product look the best that it can. Even when sales people have honorable intentions, benefits can be unconsciously promoted and potential harm downplayed. However, just because a website sells products doesn’t mean their information isn’t valid. BCCT recommends that you check the information on a site against other highly credible sources, which we describe below.

Websites or people who are trying to sell something have an incentive to make their product look the best that it can.

Read more

Author Qualifications

Red Flags

A few “red flags” cause us to question the value and validity of some sources:

  • Authors:
    • Authors and sponsors are not identified.
    • Conflicts of interest (such as an author’s or “expert’s” ties to manufacturers or organizations) are hidden.

Read more

The qualifications of authors should be described. Formal education and training, experience and independent investigation are all valid qualifications. These should be made available to the reader. Any conflicts of interest, whether financial or organizational, should be listed for authors.

BCCT’s Approach to Information

When we evaluate claims regarding therapies and treatments, BCCT strives to consider both experts’ financial interests and where they draw evidence from. We indicate in our footnotes where our information comes from and provide a link if possible so that you can check the source yourself. We look for the most credible sources available.

Clues to be skeptical and to verify any information through more reliable sources:

  • Is this person or website encouraging you to buy something?
  • Does this person or site make only vague statements about effectiveness without any evidence?
  • Do you have to pay for a product or therapy before you can receive specific information about it?

Sources We Trust

KNOW Ocology Resource

Evaluating Information in Social Media

Quite a lot of both outdated and inaccurate information is available online and is passed along through social media.

A 2018 study evaluated the accuracy of 150 videos on prostate cancer screening and treatment posted on YouTube. Study findings:12

  • Few videos provided summaries or references or even defined medical terms.
  • Videos with lower scientific quality were actually met with higher viewer engagement: more views and more "thumbs up" ratings and comments.
  • Comments often contained advertising and peer-to-peer medical advice.

We encourage our readers to check the dates on information, cross-check claims with reliable and authoritative science-based sources, and validate claims as far as possible before investing much time or money in miracle cures.

A study in 2021 concluded that "one-third of popular cancer articles on social media from 2018 and 2019 contain misinformation, and the majority of these contain the potential for harm."13

Misinformation included these aspects:

  • Misleading (title not supported by text)
  • Statistics/data don't support the conclusion 
  • Strength of the evidence is mischaracterized 
  • Unproven therapies (not studied, insufficient evidence)

Harmful information found:

  • Harmful inaction (could lead to delay or not seeking medical attention for a treatable/curable condition)
  • Economic harm (out-of-pocket financial costs associated with treatment/travel)
  • Harmful action (potentially toxic effects of the suggested test/treatment)
  • Harmful interactions (known/unknown medical interactions with curative therapies)

Our Resources collection includes many books, websites, videos and other resources that we have found to be trustworthy. However, we are open to critiques from our users of these resources. We are deeply grateful to our users for alerting us about resources that should be reconsidered or removed.

Written by Nancy Hepp, MS and reviewed by Laura Pole, RN, MSN, OCNS, and Michael Lerner, PhD; most recent update on August 23, 2021.


View All References

More Information

Enter your comments or questions below.

Comments (0)

Allowed tags: <b><i><br>Add a new comment: