Learning from Research: Strength of the Evidence

• Medical and scientific research helps us better understand the causes of different disorders and the ways they can best be diagnosed, evaluated, and treated.
• Public health research helps us understand how many people are impacted by a disorder and the affect it has on their lives.

Advances in knowledge happen through many different types of research studies. Generally speaking, evidence is collected in a variety of ways and looked at over time. This process improves our understanding of a disorder. It also guides more study and progress towards better health outcomes and quality of life.

Opinions and Anecdotes (Weakest Evidence)

Anecdotal evidence reflects the experience of one person or a small group of people. This type of reporting can not necessarily be applied to everyone. The experiences shared in anecdotal evidence, while real, is not applicable to larger groups of people.

• Often this type of science reporting can help bring ideas to light for further research that would be done in a more controlled testing environment.
• Opinions provided by experts based on their experiences are also valuable but not transferable to the general population.
• Opinion and anecdotal evidence are not objective.
• Results reported might be due to chance.
• Although anecdotes and opinions can help us learn, the evidence gathered from them is considered not strong enough to draw conclusions from.

Lab Studies

Laboratory studies done on cell samples (for example, blood, fluid or tissue samples) that are taken from a person(s) or testing carried out on animals can be useful in helping to predict what might occur in the human body. Effects in animals and people are not always the same. Cells that are tested in a lab can respond differently than cells tested in the human body. Due to these limitations, follow up research in humans still needs to be done to see if the effects seen in the laboratory are also seen in people.

Observational Studies

Observational research helps identify things that correlate (connect or are related). It is more difficult to prove a cause with observational studies. The knowledge gained from observational studies is often used to help develop more advanced research studies that test hypotheses (theories or reasons) for cause and effect. Observational studies include:

• Case reports/case series: reports on something identified in one person or a series (small number) of people that can provide information about a new disease or side effect of a treatment
• Case control studies: looks at something that has happened in the past (retrospective) in two groups of people: 1) one group with a specific symptom or condition and 2) one group without
• Cohort studies: looks at the use of a specific treatment(s) and the outcome(s) by looking retrospectively (in the past) or prospectively (currently or in future)

Clinical Trials

Clinical trials are designed to find answers to specific health questions. Questions might relate to the screening, diagnosis, treatment, or prevention of a disease. For example, many of the medications taken by people each day have been tested in people participating in a clinical trial to find out if the medicine is safe and effective.

• In a randomized controlled trial, a group of people are found to participate in research to test things like cause, effect, and safety. The group is divided into two smaller groups where people are randomly assigned to either a test group or a control group.
• Double blind, randomized, controlled trials are considered the “gold standard” of research testing. In blinded trials, a participant does not know which group they are assigned to. In a double blind trial, the researchers also do not know which group a participant has been assigned to. Blinding a trial helps remove participant and researcher bias (influence or prejudice).

A randomized controlled trial is not always possible.

• If too few people are impacted by a disorder (a rare disease) or it is too difficult to recruit participants, then other evidence has to be considered and weighed carefully to guide progress.
• For some invasive (surgical) treatments, it is not possible to have a control group of participants in the study.

The results of any clinical trial have to be considered in relation to the population studied. A very simple way to think of this is — if a randomized controlled trial showed the benefit of a specific treatment in children, we cannot assume the same treatment would have a similar benefit in adults.

Systematic Reviews and Meta-Analysis (Strongest Evidence)

What are the benefits of systematic reviews and meta-analysis?

• Bring together the results from multiple randomized control trials and look at the quality of the studies.
• Provide a summary of all the important research done in one area.
• Draw conclusions from multiple studies, which helps to remove researcher and individual study bias (prejudice or influence).
• Offer a “big picture” view of the existing evidence.

In theory, systematic reviews and meta-analysis can provide a strong level of evidence under these criteria:

• When they include studies that are comparable with respect to the type of population studied
• When the way studies are conducted (methods) are scientifically correct and comparable among studies
• When the methods used for data analyses are comparable among studies and scientifically correct

Meeting all these criteria is more often than not very difficult to achieve. Therefore the conclusions reached by most meta-analyses and systematic review articles have to be considered with great caution.