Evidence base and critical appraisal

Man in library with iPad

Accurate, evidence based information is a cornerstone of high quality information for patients and the public. There are a number of types of ‘evidence’, for example, scientific and clinical evidence such as research studies, systematic reviews and clinical guidelines; evidence from patients and carers about their experiences; and evidence from practice, particularly in areas of health where they may be little research. All have their place in the production of health information.

This page explores clinical and scientific evidence and is intended to introduce you to the concept of evidence-based medicine in relation to clinical evidence, and look at how you can use it to produce accurate, reliable and trustworthy information.

Further support and information can also be found on the PIF Toolkit, which contains best practice and useful links to support the creation of accurate and reliable health information.

Information that is not evidence-based relies on other sources:

  • Opinion (‘the doctor said it, so it must be true’)
  • Experience (‘it worked for me’)

The history of medicine is full of examples of things doctors thought were true, before they were scientifically tested and shown to be wrong. And while experience is powerful, it can be misleading. If I get better from a cold five days after drinking herbal tea, does it mean the tea cured me? No, because I would have got better any way. It’s a co-incidence.

Information based on evidence relies on the results of tests, done as fairly as possible to give a reliable answer. Fair tests can be repeated, to check that the results weren’t just down to chance.

Fair tests

Asking a question

You need to be clear about the question you want to answer, before you start looking for the evidence. The type of evidence you need, will depend on the question you ask.

The best fair test for most treatments is the randomised controlled trial. In a randomised controlled trial, people are allocated different treatments randomly (to avoid doctors choosing patients they think will do well on certain treatments) and the treatment group results are compared. If only one treatment is being tested, it will be compared to a dummy (placebo) treatment. This is the control group.

Trials are often ‘blinded’, which means the patient doesn’t know which treatment they’re getting, or ‘double blinded’ which means neither the patient nor doctor knows until the end of the trial. This helps reduce bias from people expecting a certain treatment to work well.

But other studies, such as observational trials, are important too, especially in areas where it’s difficult to carry out a randomised controlled trial. Observational studies record what people are doing (for example taking medicines, eating a particular diet) and look to see what happens to them.

Keeping up to date with evidence can be time consuming, there’s a vast amount of evidence published every day, in medical journals, at conferences, or press released by companies and universities.

You need to be clear about the question you want to answer, before you start looking for the evidence. The type of evidence you need, will depend on the question you ask. Click here to review the Types of evidence.

When searching for evidence, you need to set rules beforehand on where you’ll look and what type of studies you’ll include, to guard against ‘cherry-picking’ the information. This is called having a search strategy.  The best way to look for evidence is to carry out a systematic review.

You need rules about which studies are good enough to include in your review, or you end up with a summary of lots of unreliable studies.  The process of checking studies against these inclusion/exclusion criteria, and of assessing the quality of the evidence, is called critical appraisal.

PIF has published an introduction to the different types of evidence for patient information, and advise on which source to use when.

Top Tips

Thumbs-up

  1. Library services can support you in finding evidence. Often they will do literature searches for you and / or provide training on how to do a literature search. They will also source evidence for you and may even provide an updating service.
  2. You do not have to do a systematic review before writing a patient leaflet. There are lots of groups carrying out systematic reviews, and lots of groups using them to set guidelines.
  3. It is good practice to include references and footnotes in patient information and be prepared to share this if asked. You might also have a link on your leaflet to further sources of information on your website, for example.
  4. If a doctor wrote a leaflet, it does not automatically mean that it is evidence-based.  How does the doctor know his information is up-to-date? It’s important to ask doctors to give sources for their information, too.
  5. Leaflets should be dated and regularly reviewed. At review, you should check that the information in the leaflet hasn’t changed, by looking at a reliable source of evidence.
  6. Evidence –based information does not have too difficult for patients to understand. You don’t have to use complicated language to explain about studies. The important thing is that the information you provide is based on good quality sources of evidence.

 

Acknowledgement: Sarah Smith, PiF and Anna Sayburn, Head of Consumer Health, BMJ Evidence Centre
Page last updated: 2/1/2018