Trip Database Blog

Liberating the literature

Evidence Live: Community Rapid Review

The Community Rapid Review idea has been discussed for a while now and the final stage, before we move to production, is coming very soon.  Next week I will be running a workshop at Evidence Live on the idea.  It’ll be an interactive exploration of the thinking behind the idea and will hopefully see some final constructive criticism to guide the final product.

If you’re going to Evidence Live you can reserve a place via this link.


Medicines information coming to the Answer Engine

The Answer Engine started less than 6 months ago and has firmly established itself as a well-loved feature on Trip.  Currently, the answers are mainly linked to intervention efficacy style questions.  But, in around 4-6 weeks, we’ll be rolling out medicines information.  So, for a given drug we’ll allow users to easily see answers to questions about contraindications, warnings, interactions etc.  For instance:


Cochrane records on Trip – opinions please

Cochrane is one of the most popular resources in Trip.

Historically we have received automated updates from Wiley with the intention of ensuring we were always up to date with the records. It has come to our attention that there had been some problems with the process and that we were missing a number of key Cochrane reviews.

We have reached out to Wiley to help us resolve these issues.  Unfortunately, this is taking far too long for my liking and so we’re faced with a dilemma:

  • Continue – indefinitely – waiting for Wiley to support Cochrane’s inclusion in Trip = not up to date records of Cochrane.
  • Move to Cochrane’s records on PubMed = up to date records but not as seamless access to full-text as if it were Wiley’s.

I’d welcome peoples views as to which is the better option, please let us know in the vote below:



The evidence pyramid

A great explanation of the evidence pyramid (taken from this Walden University site):

We use this concept in Trip to help users navigate the results (helping indicate the likely reliability of the evidence):

The main place to see it is slightly to the right of the result.  Here you see the pyramid, a colour-banded representation of where the evidence lies in the pyramid and a phrase describing the content (e.g. systematic review, guideline etc). You’ll note the colour coding which can be found to the left of each result and it is carried through to the ‘Refine by’ area.

We’ve just had a large usability study of the site and this is what the report states:

  • Users from research or information management backgrounds understood the pyramid icon, but users who were new to it wondered what it was.
  • When they figured out what it was, or I talked them through it, they appreciated it and it appeared to add to their experience.
  • Recommend adding a mouse-over explanation or some kind of ‘Introducing the pyramid of evidence’ box somewhere, as part of onboarding or in help.

Bottom line: it’s useful but only when you know what it means!  With no understanding of the concept it’s just confusing.

I need to go and sit on the ‘naughty step’ and contemplate why I fell in to the trap of assuming users know all this sort of stuff (oh yes, and to fix it).



Top oncology articles

From the last month, the top ten most viewed oncology articles on Trip:

  1. Screening for breast cancer with mammography. Cochrane
  2. Interventions for the treatment of oral and oropharyngeal cancers: targeted therapy and immunotherapy. Cochrane
  3. A systematic review and economic evaluation of intraoperative tests [RD-100i one-step nucleic acid amplification (OSNA) system and Metasin test] for detecting sentinel lymph node metastases in breast cancer. NIHR HTA
  4. Aspirin use for the primary prevention of cardiovascular disease and colorectal cancer: U.S. Preventive Services Task Force recommendation statement
  5. Multiple myeloma. BMJ Best Practice
  6. Guideline Summary: Oncology evidence-based nutrition practice guideline. Academy of Nutrition and Dietetics
  7. Parental decision making about the HPV vaccine. Cancer Epidemiology & Biomarkers and Prevention
  8. Immunotherapy for advanced renal cell cancer. Cochrane
  9. Cancer and Rehabilitation (Treatment). eMedicine
  10. Cervical cancer and HPV. NICE CKS

Two stand out articles in the list for me – 7 and 9.  All the other articles are ‘secondary evidence’.

In the case of 7 it’s a primary research article and it’s from 2010. So, as well as being primary research, it’s also relatively old. This is significant as it potentially links in with our efforts to gather uncertainties (to improve research procurement – both primary and secondary research).  We’re always looking to analyse click patterns to see if we can unearth uncertainties but finding the rights signals is problematic.  The fact that a significant number of users have an information need met by an oldish article from down the evidence pyramid suggests it’s a ‘hot topic’. Perhaps one for a systematic review or further research?

Number 9 is an eTextbook – the lowest level of evidence (in the Trip hierarchy).  Again, lots of interest and users are not finding their information need met by higher quality research.

I’m excited by this ‘discovery’ one to discuss with my research funding colleagues!

Highlighting new research: the results

Last week I asked users from primary care to rate a series of new articles added to Trip.  We asked them to let us know which of 12 articles they thought looked interesting/important.  With 342 votes the results are in:

  1. Systematic literature review on effectiveness of self-management support interventions in patients with chronic conditions and low socio-economic status. EvidenceUpdate
  2. Shared decision making in primary care can reduce antibiotic prescribing. NIHR Dissemination Centre
  3. Multimorbidity and polypharmacy. NICE, advice
  4. Extended hours in primary care linked to reductions in minor A&E attendances. NIHR Dissemination Centre
  5. Antimicrobial stewardship: changing risk-related behaviours in the general population. NICE, Clinical Guideline
  6. Capsaicin 8% Patch in Painful Diabetic Peripheral Neuropathy: A Randomized, Double-Blind, Placebo-Controlled Study. EvidenceUpdates
  7. Support at Home: Interventions to Enhance Life in Dementia (SHIELD) evidence, development and evaluation of complex interventions. NIHR HTA
  8. Randomised controlled trial and economic analysis of an internet-based weight management programme: POWeR+ (Positive Online Weight Reduction). NIHR HTA
  9. Cutaneous melanoma. SIGN Guideline
  10. Nitric oxide breath analysis for the management of asthma. HTA Database
  11. Long-Term Continuous Ambulatory ECG Monitors and External Cardiac Loop Recorders for Cardiac Arrhythmia. Health Quality Ontario
  12. Canadian Youth Perceptions on Cannabis. Canadian Centre on Substance Abuse

Presented graphically:

Systematic literature review on effectiveness of self-management support interventions in patients with chronic conditions and low socio-economic status won by a single vote.

What was most interesting to me was that it still only obtained 13% of the votes (while those towards the bottom received only 4%).  So, only around 1 in 8 people voted for it.  I was expecting higher scores.  Perhaps that’s a reflection on the diversity of primary care and the diversity of our users.  Perhaps one to test in a different speciality!

Helping highlight the most important new documents on Trip: Primary care

This is an experiment to see if Trip users find it helpful and interesting to try and highlight which article, aimed at primary care, is of most use.  Below are a number of articles that we have identified as being high quality and suitable for primary care.  But the question is are they of use and/or interesting?  Can you look through the list and tick those that you think meet the criteria of being interesting or useful.

We’ll post the results in the very near future.  This will show if people enjoy rating articles and also if there are clear favourites!  And, if it works for primary care, we’ll explore other areas as well.

What we’re working on

The new upgrades have been well received and appear stable (phew).  But we’re not resting and we’re working on a  number of projects; the main ones being:

  • Learning to rank: The search algorithm decides what order the results appear in Trip.  Ours has been unchanged for many years and this shows how good it is.  However, technology has moved on over the years and one of the most exciting developments has been the technique known as learning to rank (LTR or L2R).  This uses machine learning to help understand what attributes of a result make them most likely to be clicked.  It then uses this to rank articles on the likelihood of being clicked.  This will be properly tested using a technique known as interleaving.  Timescale: We’re hoping to start testing in the next 2-3 months.
  • Community rapid review: Early days in the plans but there has been considerably enthusiasm from Trip users wanting to get involved.  An outline can be seen here while an important aspect, an educational environment, is also an important consideration (click here). Timescale: Finalised outline by July with system released by the end of the year.
  • Automated rapid reviews: Is it possible to create automatic reviews?  We think so and have started out exploring this as an option.  While not as detailed as the semi-manual community rapid review system it will still be really useful and in ways that many will not expect.  In addition, the methods used lend themselves to visualisations which will further enhance their impact. Timescale: By the end of the year, hopefully sooner.
  • Strategy: I have started reviewing the next steps of Trip with an external strategy expert.  It’s not been the most comfortable experiences (strategy is not my favoured environment), but I feel we’re making real progress. Timescale: Within 3-4 months.

At the same time we’ll continue to focus on the quality of the content and our new manual broken link system’s really useful in that respect.  We’re also working through our major providers of content to completely refresh their links. Last month was the National Guideline Clearinghouse and this month was Cochrane and CKS.

The above seems lovely an ambitious and we’re loving it!


What’s new for early 2017

We’re really excited by the release of the latest upgrade to the Trip Database.  There are a number of major upgrades to the site:

The Answer Engine: delivers answers to your clinical questions, quickly, with a single click!  For more information see this post for more details.



SmartSearch: an amazingly useful system to help speed up your search and to help users avoid missing important documents.  More information can be seen here.



Search suggestions: a pre-existing system that has been integrated much more intimately with your search experience.  Now appearing mid-way down the page; designed to prompt users to use it if the initial results aren’t focused enough.



Broken links: broken links are problematic for any search system.  We have an automated system which works reasonably well, but not perfectly.  Our new manual system should allow users to easily let us know if one of our links to a dead end!  Take our your frustration by clicking the link!



Other improvements: there are a number of other, minor changes:

  • We have improved the display and integration of the automatic assessment of bias via the RobotReviewer system.  For details of this automated system see this article.
  • We have tidied up the top of the results making it all much cleaner.

Blog at

Up ↑