I heard an interview on today’s Today programme on Radio 4 with Sir Michael Rawlins, the chairman of NICE. In the interview he highlighted the need for post-research evaluation to ensure the research promise of an intervention is realised.
This notion of experience as a form of evidence has been of interest to me for many years, ever since I was approached by a GP who was using bupropion in smoking cessation. His perspective was that bupropion was pretty rubbish for his patients and wanted to understand why the discrepancy with the evidence (which stated it was effective and evidence-based).
My own perspective is that the experience of ‘coal face’ clinicians is crucial in addressing some of the biases seen in clinical research. In a very nicely funded and implemented randomised controlled trial (RCT) you may very well exclude patients with co-morbidities – alas, in the ‘real world’ clinicians don’t have that luxury. Take an intervention such as cognitive behaviour therapy (CBT) for depression, those carrying out the trial will use an experienced CBTer as opposed to a ‘coal face’ clinician who may have basic training in the matter. The bottom line is that conditions in a RCT cannot be the same as those of the ‘coal face’ – that should surely be a worry.
Currently, the ‘evidence’ used in EBM is overwhelmingly devoted to the hard evidence obtained from trials such as a RCT. I’m not suggesting that RCTs are not useful, far from it. However, I’m increasingly of the opinion that harnessing the experience of clinicians is vital to supplement the evidence found in clinical trials. Experience can be a form of evidence.
Interestingly, in my previous post highlighting 16 years of EBM (that linked to this JAMA article) the JAMA article had the following passage (from their 1992 article):
“A new paradigm for medical practice is emerging. Evidence-based medicine de-emphasizes intuition, unsystematic clinical experience, and pathophysiologic rationale as sufficient grounds for clinical decision making and stresses the examination of evidence from clinical research. Evidence- based medicine requires new skills of the physician, including efficient literature searching and the application of formal rules of evidence evaluating the clinical literature.”
In the passage they highlight that EBM de-emphasises, amongst other factors, the ‘unsystematic clinical experience’. While I completely agree with that sentiment, it opens the door for the emphasis of SYSTEMATIC clinical experience.
So, who’s for the systematic collection of clinical experience?
Trip Database Blog 
October 17, 2008 at 11:42 am
What is “systematic clincal experience” (SCE) – and how can bias be prevented?Can SCE only downgrade evidence from RCT/SR – or can it ugrade it as well?Lot’s of questions pop into my mind… Martin
LikeLike
October 17, 2008 at 12:00 pm
Hi Martin,I’m not suggesting I know all the answers. However, there are a number of issues that relate to the current methods of generating ‘truth’.I think it’d be a real shame if we assume that the RCTs can’t be improved on and stop looking.The first step is – surely – to acknowledge there are problems with the current methods and then build from there.
LikeLike
October 17, 2008 at 7:13 pm
Now I totally agree with Jon here – we have the Hawthorne effect and also we have the “Hawthorne investigator” effect.You simply try harder when you believe in something.I was involved in the first ACE inhibitor trials for heart failure – we used to give very small people who happened to be old 50mg of captopril 3X a day – unfortunately this caused a lot of them to fall over – hence the 1st dose hypotension warning which really does not resonate in practice PS look at gabapentine – its supposed to work!!!
LikeLike
October 20, 2008 at 6:30 am
Jon your comments resonate with me. While EBM (or EBP)initially was set up to systematise clinical experience with time even EBP gurus have started to sit EBP alongside clinical experience. Lack of coal-face validity bedevills a lot of published guidelines. One of the reasons is that the very evidence base we use for evidence is heavily biased. You list some namely cop-morbidities and expert interventions for highly operator dependent interventions. The very database is skewed by ignoring the “heart sink” patient in whom we really don’t know what is going on, the many clinical questions which have not been studied and therefore published on and the narrative complications to patients which render them unique and therefore unable to be subjected to the sort of studies methodological fundamentalists regard and valid. Do you think the shift from Sackett pooh-poohing clinical experience to now having experience standing alongside EBP occurred because he got some clinical experience himself? What then of a way forward – is narrative analysis casuistry and hermeneutics a possible way forward or another blind alley?
LikeLike
October 20, 2008 at 7:27 am
@Alan: I still cannot see why “narrative analysis casuistry and hermeneutics” would do better for patients. The individual physician is much more prone to bias than well-done and well-documented systemic research. If you dithc trial evidence from EBP, it is no longer EBP!
LikeLike
October 20, 2008 at 7:58 am
I wouldn’t claim to know the best way to take it forward. However, I think there needs to be a wider scepticism of what EBM/EBP can offer. It has strengths but it also has weaknesses. I think these weaknesses need to be examined and made explicit. One of the many strengths of EBM/EBP is to be continually sceptical. This scepticism should be extended to EBM/EBP itself.I have long been of the opinion that no single construct/methodology can represent the ‘truth’. In other words it’s not one methodology or the other. When I did my PhD on the diffusion of innovations I used two methodologies, the ‘scientific’ Social Network Analysis and the ‘sociological’ Actor Network Theory. These different methodologies were both useful partly because they highlighted different aspects of the phenomena of diffusion.I think what I’m saying is that ‘truth’ depends on perspective. Currently, no single methodology will accurately portray ‘truth’. So, give clinicians multiple perspectives and let them incorporate that into their practice. I think this is an inherent feature of clinical practice. But, the potential problem being the current perspectives are from EBM/EBP and/or their – often – isolated clinical experience.
LikeLike
October 22, 2008 at 1:14 am
I am not suggesting ditching EBP, but I am arguing that EBP itself is biased. EBP deals with populations whereas a clinician deals with the individual. Yes the clinician can be biased in the way the clinician extracts information from the patient, but in the end the clinical process deal with an entity, with all the particularity of that individual, not an average. Further EBP often (especially in the addiction field in which I work) has not yet addressed the questions with the complexity needed to provide meaningful answers to the clinical issues we daily confront. The really key question is how do we distill, study and pass on clinical experience. I am not convinced that the current EBP methodology cover all that – some but not all.
LikeLike