Trip Database Blog

We’ve been working very hard on the next version of TRIP and we’re getting there!  There are two main components of work, functionality and design.  These are performed by separate people (a techie and a designer) and eventually brought together by the techie.  We’re hoping to go live in the next 4-6 weeks.

We had a first round of designs about 2-3 weeks ago and had the latest version yesterday.  So, it was a hectic day going over them and giving feedback to the designer.  But the design is close to being signed-off.  It’s actually a complete overhaul of the site, including a new logo.  It will look and behave radically differently, it’s a significant step forward for TRIP.

See below a small sneak preview of a component of the results page (click on image to increase the size).  Some hidden for tease value and some hidden as they need altering. I hope you enjoy it 🙂

I’m doing some work (well, thinking really) around clinician similarity and information needs.

Basically, if a UK general practitioner does a search for diabetes the intention/information needs are likely to be different than, say, a Brazilian endocrinologist. Yet, TRIP returns the same results.

If we created a similarity score (based on profession, interests and geography) we could we show the results as per normal but also have an area ‘Clinicians like you who searched for diabetes looked at these articles…’. 

We could also introduce something similar at an article level – ‘Clinicians like you who looked at this article also looked at these…’.

In a way, it’s using the experience of previous similar users of TRIP to filter and hopefully improve search results.

What do people think?

As part of the new upgrade to the site we’re experimenting with a number of new features. An ‘Important Papers’ feature is a side-effect of one of our efforts, but what’s that?

When you search TRIP our algorithm is designed to show the best, most research, which is great.  However, much of the latest research is built on significant ‘historical’ papers in that field, for the time being we’re calling them ‘Important Papers’ (happy to take suggestions for other names).  These are important papers associated with the main search results.

To illustrate what I’m talking about, take an example search for ‘warfarin anticoagulation’, on the main TRIP the top results can be seen here (the top result being: Comparative Effectiveness of Warfarin and Newer Oral Anticoagulants for the Long-term Prevention and Treatment of Arterial and Venous Thromboembolism (Veterans Affairs Evidence-based Synthesis Program Reports 2012)).  Using our ‘Important Paper’ feature, the top 3 results are:

• Placebo-controlled, randomised trial of warfarin and aspirin for prevention of thromboembolic complications in chronic atrial fibrillation (Lancet, 1989)
• Oral Anticoagulants vs Aspirin in Nonvalvular Atrial Fibrillation An Individual Patient Meta-analysis (JAMA, 2002)
• A risk score for predicting stroke or death in individuals with new-onset atrial fibrillation in the community: the Framingham Heart Study (JAMA, 2003)

We’re using a number of measures to infer importance including citations and social media.

Also, to be clear these results will be presented away from the main results (as a side-feature).

Will this feature be a hit?  I’ve no idea.  The results are certainly interesting and user feedback will decide if it stays.

For a number of years I’ve been pondering the numerous relationships contained within TRIP.  These are numerous and a few examples include:

• Relationships between articles e.g. via citations, by being in the same publication
• Relationships between users e.g. linked by clinical interests, by geographic location
• Relationships between articles and users e.g. looking at the same articles

I can’t help feeling there is value in these links and I do not mean in the financial sense.

Take a really simple example (click on image to enlarge it)

This is an imaginary search undertaken by 5 users and a line signifies which papers they looked at.  We can deduce some things:

• Papers six and seven weren’t liked.
• Paper two looks the most popular
• Users 2 and 3 appear similar/close (both looking at two of the same papers)

Now, if we add an extra level of data:

In this image the rounder reddish boxes signify doctors while the green boxes signify nurses.  Do these inferences seem reasonable?

• Paper five is really suited to nurses while papers one (to a point), two and three are more ‘doctor’ focused
• Paper four has mixed interest.

Imagine if you can add extra detail (different types of doctors, different geographic location) and lots of data (something we have lots of in TRIP) you might be able to generate a really powerful system.  Could it inform search results?

What do people think? I’ve really simplified things to make a point and I doubt the data will ever be as clear cut. 

Are you in for a treat?  I’d like to think so.

I had a long but rewarding meeting with Phil (our genius techie) and Reuben (our new and wonderful design guy) to thrash out the final details of the next upgrade to TRIP.  I am so excited by the proposed developments which are:

• 3 major innovations – for me really really important developments
• A handful of significant improvements
• A load of minor improvements
• A design overhaul

I’m being deliberately vague with the details for now.  But as things develop and designs get drawn, features get available I may well share them here and on Facebook (if you didn’t know we had a Facebook page, click here).

No firm timetable but I’d like to think it’ll be ready in 3 months.

Also, I’ve started planning the next upgrade, the one after this one, but much depends on how the innovations from this one take off.  If you have any suggestions then feel free to let me know.

It’s less than a month ago when I heard the gutting news that a potential purchaser of TRIP had pulled out (with no reason) from purchasing TRIP.

But every cloud has a silver lining.  While waiting for the acquisition issue I’d not given a huge amount of thought to the next updates of TRIP, not since the survey of last year (click here for details).  But not now – WOW – it’s been a great two weeks of reflection.

I’ve met with Phil (our genius techie) to discuss the updates from last year’s survey and they seem all straightforward(ish) to implement.  We had our request for donations (click here, it’s not too late) which has generated a good amount.

I really like being open about what I’ve been thinking recently, but it’s so special (at least I think so) that I need to keep it under wraps.  It’s built on our social learning tool called TILT but goes way beyond it.  One possible offshoot of this idea is to have organisational accounts of TRIP.  This would allow organisations to upload their own documents to TRIP and then these would be searchable via TRIP.  So, the University Hospital of Bristol might create an account and upload documents (local guidelines, protocols, antibiotic resistance data, clinic opening times – whatever they wish).  A local doctor or nurse could link their profile to the University Hospital of Bristol’s profile and when they search they’d see local documents.  In addition to local documents the organisational account might add their link-resolver details – making linking to full-text documents so much easier.

The big issue for me is needing to make it as painless as possible for organisations to upload their documents.  Also, it needs to be easy for individuals to find their institution.  Both shouldn’t be too problematic

So, feel free to comment or add any feature you think would make it even more powerful.

As mentioned this is a relatively small offshoot of a bigger idea which I hope to reveal gradually over the next month or so,

Following on from my recent post about TILT (click here) I’ve decided to get the wisdom of the crowd to try and improve things – I really don’t want to give up on the idea.

So, if you can spare five minutes then please take the survey – click here.

Thanks!

A search was undertaken to identify articles that compared rapid reviews to systematic reviews, further articles were identified following feedback on a list promoted via the evidence-based health mail list and various forms of social media. The list of identified articles can be found here.

Without a clear appreciation of the best way to summarise the documents, I’ve gone with a number of lessons I’ve observed from the literature combined with some personal observations.  Your feedback and suggestions for improvements would be appreciated.

Lesson 1: The notion of a rapid-review is ill-defined. However, introducing one methodology isn’t necessarily appropriate. What is important is transparency behind the process.
Observation 1: The methodology behind systematic reviews varies a great deal as well. Also, what constitutes rapid? In the literature it was typically less than 5 weeks. A lot of my work is undertaken in less than 5 hours. So, I’m very supportive of the notion of transparency.

Lesson 2: The tension between speed and accuracy is a common theme.
Observation 2: While it may appear obvious it’s important that it’s made explicit.

Lesson 3: Rapid reviews tend to look at a focused question while systematic reviews will typically look at broader topics. Also, they tend to focus on efficacy or effectiveness while not be used to examine safety, economics or ethics.
Observation 3: I’m not sure how accurate this statement is. However, I do know the broader the question the less likely it is be answerable quickly.

Lesson 4: Meta-analyses are often not undertaken in rapid reviews, so no effect sizes given – typically just a sign of an interventions effect. Any results are less generalisable and less certain.
Observation 4: A rapid review might be able to say if a treatment is likely to be better than another, it’s less able to say how much better it is. This may or may not be be important.

Lesson 5: Trial quality assessment is important, poor quality studies are likely to overestimate the benefits of a therapy or the value of a test.
Observation 5: Again, this is linked to the time factor. If you only have two days to return a response what should you do? For our ultra-rapid reviews it seems sensible to be transparent and make explicit the short-cuts and possible effects. In our ultra-rapid reviews we aim to use secondary studies but we will use abstracts of primary research as well. One paper suggested that a moderately robust summary of the evidence is better than no evidence.

Lesson 6: The conclusions between a rapid review and a systematic review do not – typically – differ. The extra effort undertaken by carrying out a systematic review may not greatly impact the final conclusions.
Observation 6: Unsurprising, but needs to be taken in the context of the points raised above. Also, an understanding of why they don’t agree is needed.

Lesson 7: Rapid reviews, when compared with systematic reviews occasionally differ. In the papers that compared the rates of difference between rapid reviews and systematic reviews were 4/39, 1/14, and 1/6.

The study that reported 4 differences in conclusion out of 39 reviews compared NICE and BUPA judgements around funding. This may well have reflected genuine differences, semantic differences (ie BUPA used a different classification system than NICE), difference in the year the review was taken (BUPA typically published their reviews earlier than NICE) and genuine judgement differences e.g. BUPA reported that percutaneous vertebroblasty for osteoporosis said it should be used in ‘trial only’ while NICE said ‘evidence adequate’ (but added caveats).
The same paper reported another study showing 1/14 differences but I was unable to ascertain the reason for the difference due to poor referencing.
In the 1/6 case the rapid review reported that the intervention was experimental while the large cost-effectiveness study indicated that the intervention was safe and efficacious. No reason was supplied for the discrepancy.
Observation 7: Clearly more research is needed to understand differences and I’d be very keen to see how ultra-rapid (less than 1 day) reviews compare with rapid and systematic reviews.

Conclusion: This is a fascinating topic that needs more research to make robust conclusions.  I looked into this topic due to my work in ultra-rapid reviews and wanting to know how they might stack up against more robust methods.  There appears to be no evidence on the matter.  I have two forms of comfort:

• In my time me and my various teams have published over 10,000 questions and many of our answers have been viewed over five thousand times.  In that time I am only aware of one serious problem with an answer.
• I have always said that what we do is not a systematic review but we invariably do better than most rushed clinicians when searching the evidence for an answer.  If our service is ‘wrong’ then it suggests providing evidence resources to clinicians (knowing they’ll do a worse job) is also wrong. 

Transparency is the key message for me.  Being clear in communicating the methods used and also in communicating the likely effect of the methodological short-cuts.