About Evidence-Based Research (EBR)

Introduction

A number of studies show that researchers, research funders, regulators, sponsors and publishers of research fail to use earlier research when preparing to start, fund, regulate, sponsor or publish the results of new studies. To embark on research without systematically reviewing the evidence of what is already known, particularly when the research involves people or animals, is unethical, unscientific, and wasteful.

Therefore, we have chosen to focus the concept of EBR on ensuring valuable research.

Our answers to the many challenges:

1. Use scientific methods to evaluate performance of research
2. Use scientific methods to improve the way research is conducted
3. Use scientific methods to monitor research practice over time

We promote the following to achieve these aims:

• … the use of a systematic and transparent approach when justifying and designing a new study
• … the use of a systematic and transparent approach when placing new results in the context of existing evidence
• … more efficient production, updating and dissemination of systematic reviews

THUS:

Traditionally, when formulating a new research question researchers use their scientific environment and context, personal interests and ambitions, and the knowledge base (underpinning epidemiological and basic science research). The EBR approach suggests that, in addition, a systematic and transparent approach should be followed to explicitly use all earlier studies and to consider end user perspectives.

QUESTION 1: How often do scientific authors refer to the totality of earlier research?
Studies analysing how often scientific authors refer to the totality of earlier research found a general lack of a systematic approach. The main conclusion is, as one of the study authors said:

“No matter how many randomized clinical trials have been done on a particular topic, about half the clinical trials cite none or only one of them. As cynical as I am about such things, I didn’t realize the situation was this bad”.Dr. Steve Goodman, New York Times, 17th January 2011

Goodman was referring to a study he co-authored with Karen Robinson and published in 2011 . They examined all systematic reviews (SRs) of health care questions, published in 2004 that included a meta-analysis combining 4 or more randomised controlled trials (RCTs), so identifying studies that could potentially refer to 3 or more studies within the same area. Even though a great number of the included studies could have referred to 10 or more previous pieces of research, the median number of references for these studies was consistently 2!

ANSWER 1: A systematic and transparent approach is rarely used when citing earlier similar trials.

QUESTION 2: Are systematic reviews used to justify a new study?
In a descriptive cross-sectional analysis of 622 RCTs published between 2014 and 2016, only 20% explicitly mentioned a SR as justifications for the new study. 44% did not cite a single SR!

ANSWER 2: A systematic and transparent approach is rarely used to justify new studies.

QUESTION 3: Are systematic reviews used to inform the design of a new study?
A retrospective study used applications for funding to see if a SR is used in the planning and design of new RCTs. In the first cohort (2006-2008), 42 of 46 (89%) referred to a SR; in the second cohort (2013) 34 of 34 (100%) referred to a SR. However, very few studies used SRs to inform the design of their new trial beyond justifying the treatment comparison (>90% in both cohorts).

ANSWER 3: A systematic and transparent approach is rarely used to design new studies.

QUESTION 4: Do authors put their results in the context of earlier similar research?
In a series of studies, Clarke and Chalmers [1998, 2002, 2007 & 2010] repeatedly showed that RCTs published in the month of May in the five highest ranking medical journals (JAMA; BMJ; NEJM; Lancet and Annals of Internal Medicine) almost never used a SR.

In 2013 Clarke and Hopewell updated this series and found there was still no improvement over time, with only 3% of RCTs containing an updated systematic review integrating their results and only 37% making any systematic attempt to place new results in context.

ANSWER 4: A systematic and transparent approach is rarely used when placing new results in the context of existing results.

Thus, while many people assume that all research is evidence-based, the evidence clearly indicates that this is not the case!

THEREFORE:

• We encourage researchers to make greater demands on themselves and thus achieve more credible and valuable results.
• We agree research must be free, but not at the expense of increased waste!
• We state that all phases of research must be systematic and transparent, including the planning of new research and the interpretation of new results

The Evidence-Based Research Network

To address the problem outlined above a group of Norwegian and Danish researchers initiated an international network, the ‘Evidence-Based Research Network’. The EBRNetwork was established in Bergen, Norway in December 2014 with initial partners from Australia, Canada, Denmark, the Netherlands, Norway, the UK, and USA.

The aim of the EBRNetwork is to reduce waste in research by promoting:

• No new studies without prior systematic review of existing evidence
• Efficient production, updating and dissemination of systematic reviews

The EBRNetwork has suggested a new working definition of a systematic review:

“a systematic review is a structured and preplanned synthesis of original studies that consists of predefined research questions, inclusion criteria, search methods, selection procedures, quality assessment, data extraction, and data analysis. No original research study should be deliberately excluded without explanation, and the results from each study should justify the conclusion.” 

In 2016 members of the EBRNetwork published an analysis article in the BMJ “Towards evidence-based research” discussing EBR and its role in preventing research waste. The article contained the EBR Statement, detailing the different stakeholders’ responsibilities in meeting the aims of EBR, and a flow chart for EBR.

Using scientific methods to evaluate performance of research

A 2022 Scoping Review identified 69 relevant meta-research studies: 34 studies evaluating whether redundancy existed among similar studies and 42 studies exploring whether authors of clinical studies had used a systematic and transparent approach to avoid redundancy in health research, with seven studies addressing both aspects. Twenty-three studies had concluded that redundancy was present among similar clinical studies, and three studies had identified redundancy among similar systematic reviews (SRs). The studies evaluating the use of SRs to inform research and its reporting paint a dire picture:

• Fifteen studies reported no or poor use of SRs to inform justification of a new study.
• Six studies showed no or poor use of SRs to inform design.
• Seven studies demonstrated no or poor use of SRs when placing new results in the context of existing research.

In conclusion, current best available meta-research shows that researchers make no or poor use of SRs when justifying and/or designing new studies, or when placing new study results in the context of existing similar research.

In October 2020 a series of three article was published in the Journal of Clinical Epidemiology further developing the concept of EBR and its use in justifying new studies and putting the results in context:
#1-What Evidence-Based Research is and why is it important?
#2-Using an Evidence-Based Research approach before a new study is conducted to ensure value
#3-Using an Evidence-Based Research approach to place your results into context after the study is performed to ensure usefulness of the conclusion

EBR Multimedia Resources

Introduction to evidence based research from Caroline Blaine

Introduction to Evidence-Based Research (pdf of slides)Download