Introduction
To embark on research without systematically reviewing the evidence of what is already known, particularly when the research involves people or animals, is unethical, unscientific, and wasteful.
The EBR approach promotes:
- … the use of a systematic and transparent approach when justifying and designing a new study
- … the use of a systematic and transparent approach when placing new results in the context of existing evidence
When considering a new research question researchers use their scientific environment and context, personal interests and ambitions, and the knowledge base (underpinning epidemiological and basic science research). In an EBR approach a systematic and transparent approach to explicitly consider all earlier studies and to consider end user perspectives.
J Clin Epi. 2020
QUESTION 1: How often do scientific authors refer to the totality of earlier research?
Studies analysing how often scientific authors refer to the totality of earlier research found a general lack of a systematic approach. The main conclusion is, as one of the study authors said: “No matter how many randomized clinical trials have been done on a particular topic, about half the clinical trials cite none or only one of them. As cynical as I am about such things, I didn’t realize the situation was this bad” (Dr. Steve Goodman, New York Times, 17th January 2011).
Goodman was referring to a study he co-authored with Karen Robinson and published in 2011 . This study identified systematic reviews (SRs) of health care questions published in 2004 to collect trials that could potentially refer to 3 or more trials addressing the same question. Regardless of the number of trials that could be cited – a large number of trials could have referred to 10 or prior trials – the median number of prior trials cited was consistently 2!
ANSWER 1: A systematic and transparent approach is rarely used when citing earlier similar trials.
QUESTION 2: Are systematic reviews used to justify a new study?
In a descriptive cross-sectional analysis of 622 RCTs published between 2014 and 2016, only 20% explicitly mentioned a SR as justifications for the new study. 44% did not cite a single SR!
ANSWER 2: A systematic and transparent approach is rarely used to justify new studies.
QUESTION 3: Are systematic reviews used to inform the design of a new study?
A retrospective study used applications for funding to see if a SR is used in the planning and design of new RCTs. In the first cohort (2006-2008), 42 of 46 (89%) referred to a SR; in the second cohort (2013) 34 of 34 (100%) referred to a SR. However, very few studies used SRs to inform the design of their new trial beyond justifying the treatment comparison (>90% in both cohorts did not use SRs to inform study design).
ANSWER 3: A systematic and transparent approach is rarely used to design new studies.
QUESTION 4: Do authors put their results in the context of earlier similar research?
In a series of studies, Clarke and Chalmers [1998, 2002, 2007 & 2010] repeatedly showed that RCTs published in the month of May in the five highest ranking medical journals (JAMA; BMJ; NEJM; Lancet and Annals of Internal Medicine) almost never used a SR.
In 2013 Clarke and Hopewell updated this series and found there was still no improvement over time, with only 3% of RCTs containing an updated systematic review integrating their results and only 37% making any systematic attempt to place new results in context of what was already known.
ANSWER 4: A systematic and transparent approach is rarely used when placing new results in the context of existing results.
Introduction to Evidence-Based Research (pdf of slides)Download
