I was looking at Twitter when I saw this tweet:

So, why not see what the highly experimental Trip Rapid Review system makes of oral tapentadol for cancer pain.  So, I spent five minutes and came up with this result:

We found three trials and the actual Cochrane Systematic Review found four, the trial difference was an unpublished one (but well done Cochrane for finding that one).  Frustratingly the actual review had no forest plot – so our pseudo-plot (above) will have to do.

Our system gave a score of 0.42 which suggests reasonable, but unspectacular, results for oral tapentadol.  The actual Cochrane conclusion is:

“Information from RCTs on the effectiveness and tolerability of tapentadol was limited. The available studies were of moderate or small size and used different designs, which prevented pooling of data. Pain relief and adverse events were comparable between the tapentadol and morphine and oxycodone groups“

It’s difficult to compare end-points – Cochrane says it’s as good as morphine and oxycodone (which may be good, bad or indifferent – I don’t know) while our system suggests it’s ok/not bad.

Given the highly experimental nature of our system I think we give consistently good results.  The important next steps are:

  • Improve the system – which we’re about to start on, via our Horizon 2020 funded work.
  • Validate the approach so we can understand when it works well and when it doesn’t.

We’ve used this approach before (see the example on SSRIs for the management of hot flashes) with good results and our earliest internal tests found around 85% agreement.

I’m not suggesting this is a replacement system and I’m under no illusion of the potential for harm – if mis-used – but it’s a novel approach which should see further developments.  While much of the developments in machine learning, text mining etc in systematic reviews are about replacing humans in the standard systematic approach I see this approach as being wholly more revolutionary.