Systematic Reviews

Introduction

We live in an era of evidence based medicine, where it is expected that all clinicians should ensure their decision making is informed by the best available scientific evidence. The development of systematic reviews of randomized controlled trials of health care has been promoted by The Cochrane Collaboration.  Key features of such reviews, as defined by the Cochrane collaboration, are that they address a clearly formulated question, using systematic and explicit methods to identify, select, and critically appraise relevant research, and that they collect and analyze data from the studies that are included in the review (1).  Statistical methods such as meta-analysis are an optional additional feature that can be used to analyze and summarize the results of the included studies (1).

Systematic reviews have become increasingly important in health care as they are seen as a key method for determining the value of different diagnostic or management strategies and thus are able to provide essential information to underpin the evidence based approach.  For example, systematic reviews are often heavily cited to support recommendations contained within the ever increasing number of clinical practice guidelines.   Systematic reviews also provide a straightforward method for clinicians to keep up to date with their field (2,3).  Although highly valued by clinicians and policymakers, it is noteworthy that the standard of systematic reviews can vary and not all are high quality, for example, many do not describe clear search strategies or methodologies as recommended by PRISMA (4) or QUORUM guidelines (5).

High quality systematic reviews have proven valuable to clinicians and policymakers alike, but still have a range of important limitations. The most important limitation is that such reviews have systematically excluded qualitative studies from the evidence base thus making it difficult to understand the processes at play when implementing complex interventions or the key issues that will be important to their future integration into routine practice.  Such a crucial omission may be a factor contributing to the difficulties so often seen in translating "evidence" into practice.   Thus even if a given complex intervention is demonstrated to be "effective" in a particular context there remains uncertainty about why and how it was successful in that setting and the broad range of factors that would need to exist in order to ensure successful implementation and integration in other circumstances or contexts.  Another challenge, is that the exclusion of qualitative studies from the systematic review process means that there are no clear mechanisms for collating and interpreting qualitative data which is being compiled in the same subject area, for example, patient experiences of living with a specified chronic condition, in order to develop conceptual models or gain a clearer picture of the problem under investigation. 

Consequently,  in recent years there has been growing interest in developing robust methods for synthesising qualitative data and a growing range of approaches described  (6) including: thematic analysis;   framework synthesis; meta-ethnography (7);  critical interpretive synthesis (realist synthesis) (8) ; and  meta-narrative approaches (9). In this section we suggest that NPT can provide a valuable method to aid the conduct and interpretation of systematic reviews of a range of different types of qualitative study and that there are three main ways in which it could be used.

  • To support the development of research questions and overall design of a systematic review.
  • To serve as a framework for data analysis within a systematic review
  • To support the interpretation of a systematic review's results

Each of these is discussed in its own section; click on the links above

Things to consider

  • What are the questions I am trying to answer by undertaking a systematic review and how might NPT be of value?
  • What kinds of research will or will not be amenable to this type of systematic review?

References

  1. Green S, Higgins JPT editors. Cochrane Handbook For Systematic Reviews of Interventions 4.2.6. (Updated September 2006) In: The Cochrane  Library, Issue 4, 2006. Chichester, UK: John Wiley & Sons Ltd.
  2. Oxman AD, Cook DJ, Guyatt GH. Users' guides to the medical literature. VI. How to use an overview. Evidence-Based Medicine Working Group. JAMA 1994;272:1367-71. 
  3. Swingler GH, Volmink J, Ioannidis JP. Number of published systematic reviews and global burden of disease: database analysis. BMJ 2003;327:1083-4. 
  4. Moher D, Liberati A, Tetzlaff J, Altman DG, for the PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 2009;339:b2535. 
  5. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF, for the QUOROM group. Improving the quality of reporting of meta-analysis of randomized controlled trials: The QUOROM statement. Lancet1999;354:1896-1900. 
  6. Elaine Barnett-Page and James Thomas. Methods for the synthesis of qualitative research: a critical review.  Methods for Research Synthesis Node, Evidence for Policy and Practice Information and Co-ordinating (EPPI-) Centre, Social Science Research Unit, Institute of Education, 18 Wobrun Square, London WC1H ONS. http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=188. ESRC National Centre for Research Methods NCRM Working Paper Series Number (01/09).
  7. Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton S.  Synthesising qualitative and quantitative evidence: a review of possible methods.  J Health Serv Res Policy 2005; 10 (1): 45-53. 
  8. Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review - a new method of systematic review designed for complex policy interventions.  Journal of Health Services Research Policy 2005;10 (Suppl 1):21-34. 
  9. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O, Peacock R.  Storylines of research in diffusion of innovation: a meta-narrative approach to systematic review.  Social Science and Medicine 2005; 61:417-430.