Search

Trip Database Blog

Liberating the literature

Open Search
Category

rapid reviews

Oral tapentadol for cancer pain

I was looking at Twitter when I saw this tweet:

So, why not see what the highly experimental Trip Rapid Review system makes of oral tapentadol for cancer pain.  So, I spent five minutes and came up with this result:

We found three trials and the actual Cochrane Systematic Review found four, the trial difference was an unpublished one (but well done Cochrane for finding that one).  Frustratingly the actual review had no forest plot – so our pseudo-plot (above) will have to do.

Our system gave a score of 0.42 which suggests reasonable, but unspectacular, results for oral tapentadol.  The actual Cochrane conclusion is:

“Information from RCTs on the effectiveness and tolerability of tapentadol was limited. The available studies were of moderate or small size and used different designs, which prevented pooling of data. Pain relief and adverse events were comparable between the tapentadol and morphine and oxycodone groups“

It’s difficult to compare end-points – Cochrane says it’s as good as morphine and oxycodone (which may be good, bad or indifferent – I don’t know) while our system suggests it’s ok/not bad.

Given the highly experimental nature of our system I think we give consistently good results.  The important next steps are:

  • Improve the system – which we’re about to start on, via our Horizon 2020 funded work.
  • Validate the approach so we can understand when it works well and when it doesn’t.

We’ve used this approach before (see the example on SSRIs for the management of hot flashes) with good results and our earliest internal tests found around 85% agreement.

I’m not suggesting this is a replacement system and I’m under no illusion of the potential for harm – if mis-used – but it’s a novel approach which should see further developments.  While much of the developments in machine learning, text mining etc in systematic reviews are about replacing humans in the standard systematic approach I see this approach as being wholly more revolutionary.

0

Evidence, hourglasses and uncertainty

Long-term readers of this blog will know I struggle with many aspects of the systematic review process.  At the time of writing, my ‘A critique of the Cochrane Collaboration‘ has been viewed over 18,300 times and ‘Ultra-rapid reviews, first test results‘ nearly 10,000 times.

I believe the main justification given for conducting systematic reviews is to obtain a really accurate assessment of the effectiveness (or ‘worth’) of an intervention.  So, the thinking goes that spending 12-24 months is worth the cost (financial, opportunity, etc) due to the accuracy of the prediction it then gives.

My immediate response is that is demonstrably false. In my article ‘Some additional thoughts on systematic reviews‘ (just under 5,000 views) the evidence is clear that if you rely on published journal articles to ‘inform’ your systematic reviews (which is the case in the vast majority of systematic reviews) there is approximately a 50% chance that the effect size is likely to be out by over 10%.

But, even if we suspend being evidence-based and believe that systematic reviews can be relied upon to give us an accurate estimate of an effect size, is everything fine? I don’t think so and the image below illustrates my thinking.

It’s an hourglass!  At the top are all the unsynthesised trials, all floating around and the uncertainty is moderate.  Someone then spends 12-24 months pulling these together in a systematic review (likely of published trials and therefore ‘a bit dodgy’) and the certainty is reduced at the aperture of the hourglass.  But then, when you apply it to the real world of patient care, the uncertainty flares out again.  In the above example the intervention has a NNT of 6, so the intervention needs to be given to 6 people to obtain the desired outcome in 1 person.  Which is the 1 person?  Where’s the certainty?

If we were to spend significantly less time doing a review it might indicate a wider hourglass aperture (perhaps suggesting an NNT of 5-7).  In what situations does that matter?  I don’t think we’ve even started to explore these issues. In other words, when is it appropriate to spend 12-24 months on an systematic review and when is a significantly less resource intensive approach ‘ok’?

Is it irony that the reality is the type of review (systematic versus ‘rapid’) doesn’t alter the effectiveness of an intervention?  After all the compound remains the same, untroubled by the efforts of trialists.  Sorry, getting sociological there – must be time to sign off for now.

0

Search safety net

As we move forward after the introduction of our Premium product we can start to plan future developments and one is a search safety net!

Using our click stream data we can see what articles you’re looking at suggest other documents that you should consider.  Take this network map (click to enlarge):

This is based on searches on Trip for urinary tract infections.  Each blue square (node) represents an article and the lines linking them are created when a user clicks on two (or more) articles in the same search session.

If we have this information we can build a really useful system.  A user comes and does a search of Trip for UTI and finds, from articles in the bottom right of the above image, a number of articles (marked in red in the image below):

It is clear that they may have overlooked four articles (marked as blue nodes) so we alert the user.  It gives them a chance to double check the results.  They may have deliberately excluded them or they may have simply made a mistake.  If it’s the latter then the system will have served its function as a safety net.

I’ve started using the phrase ‘Trip makes finding evidence easy’ but with this technique we could also claim that ‘Trip easily helps you not miss key evidence’.  Not quite as succinct, but you get the picture!

This is a value added service so I envisage it only being available to Premium users of Trip.  Hopefully another reason to upgrade.

0

The light at the end of the tunnel…

…is, I hope, not the light of an oncoming train. I’ve nabbed that line from my favourite band – Half Man Half Biscuit (HMHB) who wrote The Light At The End Of The Tunnel (Is The Light Of An Oncoming Train) a good few years ago! My love for HMHB aside, I keep reflecting on how things seem to be going really well for Trip and I’m desperately hoping we’ve turned a corner.  So, why the optimism:

  • 2014 was pretty good.
  • We’re working on the new Freemium version of Trip.  What’s going to come out is going to be impressively good and some of the premium upgrades will be great.
  • We’re involved in the really interesting EU funded project which will be doing some really innovative things.  I’ll blog about that more when the final specifications are agreed, but we’ll be looking at making Trip more multi-lingual, we’re going to be improving the Trip Rapid Review system and loads of work around similarity which is useful for the next point.
  • Relatedness/similarity is looking very useful for what we want to do with regard developing our financial viability.  The measures we’re developing will allow us to do all sorts of interesting things, for instance we can highlight a new book that’s useful to a particular clinician, we can highlight a new trial that’s pertinent to an existing systematic review.  Many more uses on top of that, but I’ve got to keep some secrets.
  • I’m starting to realise the value in our clickstream data (helped by two separate teams and soon to be joined by a PhD student as part of the EU project).  You only have to look at most of this year’s blog posts to see I’m working hard on this.  This can help with the relatedness work but it can do other useful things, such as improving the search results and better predicting new articles that are of use to a Trip user.  If our mission is to ensure health professionals get the right evidence to support their care – using clickstream data will make it so much more effective.  The advantage of the clickstream data is that it’s Trip’s data to utilise, it’s our IP.  It’s at the heart of our future.  I actually think it’s this point that’s making me so happy/optimistic.
  • Lots of other nice bits and bobs e.g. I’ve just been invited to lecture in the USA in Autumn/Fall; I’m part of a large consortium bidding to be a support team for complex reviews; I’m presenting at the wonderful Evidence Live; I’m making headway in my new NHS job (I am lead for Knowledge Mobilisation for Public Health Wales); I’m waiting to hear about a large MRC grant (not optimistic but something to look forward to).

Long may this continue!

0

Blog at WordPress.com.

Up ↑