Randomised Control Trials – are they still the gold standard?
Our Future Health (OFH) is an RCP programme of public debates, podcasts, and expert roundtables that will raise awareness on some of the most pressing issues facing our NHS today. Over the course of 2018, OFH is aiming to raise, and to find ways to tackle, some of the most pressing issues facing our NHS today. From investment and resources to research and innovation we will start to ask the complex questions needed to shape healthcare policy for the future. More information about OFH is available at https://ourfuturehealth.rcplondon.ac.uk
In March, as part of OFH’s theme of innovation, the RCP hosted an event to discuss randomised control trials (RCTs).
RCTs have long been considered the gold standard for drug development as their use establishes causation in a precise, controlled manner, typically measuring success of a new drug or intervention against a group of randomised control participants receiving a placebo. However as research advances, there have been criticisms levelled at the use of RCTs, particularly when responding to epidemics or use in conditions with high mortality. RCTs can also be problematic due to the selection of participants, which often excludes parts of the population.
For this event, the RCP brought together physicians, pharmaceutical companies, researchers and regulators to assess whether the RCT is still the gold standard for medical research. Participants at the event heard presentations from Professor Ashley Woodcock and Dr Elaine Murphy (critically appraising RCTs) and Professor David Webb and Professor Alan Boyd (outlining why RCTs should still be considered the gold standard).
Over the coming weeks and months, the OFH team plan to bring many of the viewpoints that were heard on the day to their website, in the form of a series of blogs on the subject. Here, we share the views of two attendees from the roundtable: Dr Daniel Martin (critical care consultant and anaesthetist at the Royal Free and reader at University College London) and Professor Anne Lingford-Hughes (chair of the Academic Faculty of the Royal College of Psychiatrists; professor of Addiction Biology at Imperial College London).
RCTs in critical care medicine: a square peg for a round hole – Dr Daniel Martin
In evidenced-based medicine, our acceptance of the RCT as the universal ‘gold standard’ has led to the belief that this is the only form of study capable of providing acceptable evidence upon which to base our practice. While it may be the ideal scientific tool in some scenarios, it is a very blunt instrument in others. In the perfect experiment, one single factor is altered in order to evaluate its impact on a defined outcome. But how should one approach research in the presence of a highly heterogenous, rapidly changing and chaotic setting, as we often find on an intensive care unit (ICU)?
The first fundamental issue to consider when investigating this group of patients is the lack of a clear definition of what constitutes ‘critical illness’. It is loosely defined as a life-threatening condition in which organ failure is a feature. The patient may have multiple underlying diagnoses, ranging from meningitis to malignancy, with artificially defined constructs such as ‘sepsis’ and ‘acute respiratory distress syndrome’ superimposed. It is little wonder that very few major advances in critical care medicine have occurred as a result of any RCT over the last two or three decades.
With such heterogeneity of pathology, disease severity and co-morbidity within this patient group, the effect of any evaluated intervention will vary according to the individual patient, ranging from beneficial to harmful. The average effect of the intervention across the entire population may therefore appear to be neutral. Thus, this bastion of the evidence-based medicine, well-suited to clearly defined homogeneous patient groups may not be the ideal tool in critical care medicine.
Another factor that has hampered the use of RCTs in this context is the fact that the considerable majority of severely unwell patients lack the mental capacity to consent to enrolment into a study, through a combination of reduced conscious level, sedative medications or mechanical ventilation. Agreement to enroll patients must therefore be sought from their next of kin or a professional representative. Engaging in a conversation about research, in the context of attending a critically ill loved one, is a difficult prospect for relatives on the ICU, who are already faced with extremely distressing decisions, amid uncertainty and grief. Making any decision on behalf of another presents an additional barrier and makes opting out the easiest option. These present particular challenges in studies with time-critical entry points and have undoubtedly contributed to the lack of very large-scale trials in ICUs and the protracted time required for their completion.
It is vital that specialties do not abandon evidence-based medicine in examples such as this, but rather they should seek to adapt the way in which they approach research in order to find the correct tool for the job. Stratified medicine, the use of sophisticated techniques to identify subgroups likely to respond to specific treatments, may represent a superior strategy for research in critically ill patients. The success of the next phase of evidence- based medicine in this field will depend on whether we can design studies with suitable rigour to test these targeted therapies.
Harnessing big data:
not as simple as it sounds – Professor Anne Lingford-Hughes
As the roundtable event progressed, and we discussed the relative strengths and weaknesses of RCTs, conversation turned to ‘big data’ and what this might mean for the future of research. Big data is receiving much attention about its potential to improve healthcare. As patient records have become increasingly digitalised, such big data driven approaches are now more feasible.
However, for such approaches to benefit the patient and improve health, the records must therefore contain good quality information about any clinical or demographic variables that may impact on improving outcomes.
The challenge and concern is that this may not be the case, as many relevant datasets in the UK are not well coordinated. Firstly, there is no single medical or care computerised record that is used across all settings – they are set up to meet clinical needs rather than research. If we consider substance use and misuse, which is known to have a substantial impact on health, recording about alcohol consumption or drug use in a general medical or psychiatric hospital or in primary care is often at best limited and commonly woeful.
The records from ‘non-specialist’ settings such as Hospital Episode Statistics which has data on the acute hospital system (A+E/ ICU/inpatient/maternity), or the Clinical Practice Research Datalink, which has data on primary care, are currently not linked up with records from specialist addiction services or their reports to the National Drug Treatment Monitoring System (NDTMS). Therefore if we are to understand how substance use and abuse may be impacting on health and healthcare, information needs to recorded accurately and consistently across these datasets, which also need to be linked and available to the research community. While there is widespread acknowledgement of these needs, overcoming them for a range of databases still appears an aspiration and mired in bureaucracy. If we just focus on data from hospital populations then we will perpetuate the lack of development in care for those patient populations whose records are outside this population, and we also risk missing key modifiable variables that may deliver highly cost-effective improvements to care.
As health services develop, a further challenge is that some clinical services have now moved outside the NHS to other providers. In addition where services are commissioned by local authority, research and training may be forfeited to deliver a cost-effective service, particularly when there is a competitive bidding process between several different providers. To use substance misuse services as an example, lead providers are now commonly from the third sector, with generally different computer systems to their NHS partners though all have to use NDTMS.
Substance misuse services is not the only example of providers moving outside the NHS, but rarely do I hear about ensuring appropriate links with the third sector (or other providers) regarding the sharing of data. The risk for any clinical service being outside the ‘traditional NHS’ is that their patients are further marginalised and improving clinical outcomes is therefore unlikely.
More broadly, the common separation in secondary care between psychiatry and other medical services means that mental health outcomes, such as depression or anxiety, rarely appear among primary or secondary measures for clinical trials of what are seen as ‘medical’ or ‘surgical’ interventions. This is despite an acknowledgement of how important the relationships are between mental and physical health, as well as recognition that such comorbidity is the norm. This separation perpetuates the ‘organic’ vs ‘functional’ divide which should have been relegated to the past.
The accompanying headlines to recent stories that psychosis and depression were ‘physical not mental’ illnesses (since they are associated with inflammation) underlines how much attitudes still need to change.
Stigma is still a pervasive issue in psychiatry – not only for patients but also for clinicians – where the brightest and best may be discouraged from working in this area with immense potential, where they could develop and deliver transformative improvements.
As mental health receives greater attention in society with greater understanding of those with mental health disorders, stigma within the medical profession must also surely diminish.