I love it when we roll out new features and few have been as significant and innovative as this one.  Over the last few months I’ve been working with the wonderful team at RobotReviewer to introduce two major improvements to Trip.

Identification of RCTs.

Trip has featured a search results category called ‘Controlled trials’ for years.  To identify trials we used a filter to highlight trials in PubMed and imported them in to Trip.  This used a series of keywords and was good at identifying trials but was also prone to identifying a number of other articles that were not trials.  In other words there were a number of false positives (ie noise) and we invariably missed a few trials as well.

RobotReviewer used machine learning to identify trials from Trip and it works brilliantly.  In internal tests our controlled trials is about 97% accurate, which is amazing.  The total ‘count’ of trials has dropped by over 200,000 which means they were incorrectly identified by the filter.  So, when using the controlled trials filter you’re significantly more likely to just find trials and avoid the noise of incorrectly identified trials!

Automatic assessment of bias.

Last year the RobotReviewer team published RobotReviewer: evaluation of a system for automatically assessing bias in clinical trials.  The paper concluded:

“Risk of bias assessment may be automated with reasonable accuracy. Automatically identified text supporting bias assessment is of equal quality to the manually identified text in the CDSR. This technology could substantially reduce reviewer workload and expedite evidence syntheses.”

In short their techniques pretty much matched human ability in assessing bias.  Now, in conjunction with Trip, they have extended their techniques to work on the controlled trials that Trip has: abstracts.  With very little loss of accuracy we have just released this feature (see their blog for more technical details).  In this first image it shows what to expect:

RR1

The ‘Estimate of bias…’ is clickable to reveal:

RR2

This is a significant moment for Trip and I’m delighted that we have this feature.  Assessment of bias is not most people’s idea of fun and if we can help reduce the barriers to using evidence – which we have with this feature – then everyone should be delighted.