Interpretation

Introduction

While a randomised controlled trial is the best design for evaluating the effect of an intervention, this methodology tells you little or nothing about why the intervention does or does not work. Process evaluations that run in parallel to, or after, trials have become a common approach to learning more about why the trial generated the result it did, particularly for complex interventions that aimed to change behaviour. Such evaluations may contribute to:

  • Interpreting the study results
  • Understanding the mechanisms of actions of the interventions
  • Developing or refining models that explain intervention effects
  • Designing future interventions
  • Providing, for the purpose of a systematic review of interventions, additional data on the operation, delivery and receipt of the intervention

How to structure these evaluations is not always obvious. NPT provides an interpretive framework that can be used to focus discussion with those delivering and those receiving the intervention. Moreover, the consistent use of a single theoretical framework would enable a body of knowledge to grow about classes of intervention (eg. decision support systems for GPs; educational interventions that aim to improve lifestyle decisions made by patients), which will make it easier to both interpret results and design new trials. Use of NPT for interpretation will also allow trialists to make propositions about mechanisms, which can then be tested by other trialists in future trials. These latter trialists can, of course, also make direct use of NPT when designing these trials (see Intervention Design).  NPT can be used within other process evaluation frameworks, such as the one developed by Grant and colleagues (1, 2).

Things to consider

  • If you are planning a process evaluation, NPT may help you to structure this evaluation.
  • NPT will encourage you to focus on the range of people, situations, times and places that are involved in all aspects of enacting that process of providing the intervention (or comparator).
  • NPT is not about individuals intentions and perceptions, it is focused on helping you to making sense of collective, distributed, patterns of work.

Illustrative example - process evaluation in primary care

Applying the Normalization Process Model to problem solving therapies for psychosocial distress and nurse-led clinics for heart failure treatment in primary care

Carl May and colleagues (3) applied the Normalization Process Model (NPM, which is part of NPT) to two different trials to understand the implementation of two complex interventions: (i) the delivery of problem solving therapies for psychosocial distress, and (ii) the delivery of nurse-led clinics for heart failure treatment in primary care. Use of NPM highlighted a number of issues, for example:

Relational integration: accountability, confidence and trust

When patients entered the heart failure clinics trial, the management protocols and techniques that nurses employed were based on clinical guidelines. But nurses required additional training and ongoing professional support to help them acquire and feel comfortable with their level of knowledge in this sphere. This improved their individual expertise and accountability by providing a strong theoretical background to heart failure - in other words the training was focused on the knowledge under- pinning the intervention (the guideline) rather than its application in clinical interactions. This explains why nurses felt inadequately prepared for the practicalities of seeing patients with complex problems in the clinic. Thus, in the initial stages of the trial remedial work needed to be done, because professionals delivering the intervention expressed concerns about their confidence in delivering the intervention. The patients on the other hand, expressed confidence in the health professionals and felt that the intervention gave them greater confidence in managing their health.

Contextual integration: not just a problem of funding

In our two case studies, we can only speculate about their potential for contextual integration. In the case of PST answers to this question will depend critically on what resource allocation models are employed. From the perspective of healthcare commissioners, it would appear most cost-effective to place the delivery of PST in the hands of new breeds of healthcare providers, such as depression care managers or graduate mental health workers, since these tend to be less qualified and hence less expensive. In a situation of expanding resource allocation, such a shift would be unlikely to be seen as prejudicial by general practitioners or existing healthcare professionals, but may rather be welcomed as an additional resource. However in the more common situation where resource for healthcare is finite or even decreasing, any consequent shift in allocation would be likely to meet resistance from existing healthcare staff.

References

  1. Grant A, Treweek S, Dreischulte T, Foy R, Guthrie B. Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials 2013, 14:15.
  2. Grant A, Dreischulte T, Treweek S, Guthrie B. Study protocol of a mixed-methods evaluation of a cluster randomized trial to improve the safety of NSAID and antiplatelet prescribing: data-driven quality improvement in primary care.  Trials 2012; 13:154.
  3. May CR, Mair FS, Dowrick CF, Finch TL. Process evaluation for complex interventions in primary care: understanding trials using the normalization process model. BMC Fam Pract. 2007 Jul 24;8:42. PubMed PMID: 17650326; PubMed Central PMCID: PMC1950872.

 

Further Reading

Scott Wilkes, Nicola Hall, Ann Crosland, Alison Murdoch, Greg Rubin. General practitioners' perceptions and attitudes to infertility management in primary care: focus group study. Journal of Evaluation in Clinical Practice. 2007. online ISSN 1356-1294.