Search

Trip Database Blog

Liberating the literature

Trip Evidence Service

We started Trip due to our work on clinical question answering.  We set up a number of responsive services whereby a health professional/manager could ask us a question and we would answer it using the best available evidence. We worked to clinical time frames (days, sometimes hours) and as such these were not systematic reviews. However, these were likely better than a busy health professional, with limited evidence skills, could achieve given their busy schedules.

Trip, and the main staff involved, have answered over 10,000 clinical questions.  These have been mainly for health professionals but we also have experience of answering questions for other sectors such as commissioners and policy makers.

All staff involved love this work and as the quotation goes “Find something you love to do and you’ll never have to work a day in your life“. It is therefore natural, and long overdue, that we are getting back in to this work.

The actual Trip Database continues to be driven by a desire to support clinical question answering but recognise that sometimes this is not enough. Search is fine but sometimes people require something more formal.  We’re not proposing to undertaken systematic reviews, there are other out there for that. Our offering is aimed at ad hoc evidence needs, covering areas such as:

  • Background work needed at the start of a project or before committing to it – has similar work been done before? If so, is this work needed or can you learn from the previous work?
  • A second opinion – have you seen a review and are not sure if you can trust the findings?
  • A general overview of a particular topic – have you been asked to look at a clinical area and need an overview to help you orientate you as to the state of play regarding the latest evidence and guidance?
  • A clinical Q&A – do you have a clinical question and need a rapid answer?

In fact, if you have any evidence need, let us know and we’ll see what we can do.

We have huge experience in this area, cutting edge evidence systems and a pragmatism that can see trusted answers delivered rapidly and cost-effectively. If you’re interested then get in touch via this link, even for an informal discussion.

 

Q&A – going back to our roots

The work on our automated system has taken around two years and much of our work has focused on that. But now it’s time to look back at the main purpose of Trip, to help health professionals find answers to their clinical questions!

This lovely equation is taken from Shaughnessy and Slawson’s work on Information Mastery:

So, each component is important, assuming you’re interested in useful clinical information:

  • Relevance – how relevant is the information for the clinical question?
  • Validity – how robust is the information? Is it based on good or weak evidence?
  • Work to access – do you get the answer quickly or does it take a long time?

So, for a given question Trip needs to maximise ‘relevance’ and ‘validity’ and reduce ‘work’.

Relevance: For a given set of search terms (which is how the searcher represents their clinical Q) we need to ensure the more useful documents are returned. We’re working on a new algorithm which should be boosted by learning about previous searches. This should improve relevance of results.

Validity: Our algorithm (current and future) will always favour higher-quality content. But we need to be aware that high-quality content is the tip of the evidence pyramid – it’s more robust but there’s lot less of it, so it answers fewer questions.

Work: Our search is quick and if we boost relevance then there are fewer articles for health professionals to look through. However, search returns articles likely to answer the Q, it doesn’t directly answer the Q. We’ve explored this with the answer engine but that is hampered by limited scalability.  So, how can we reduce the work?  If we can predict the question can we highlight the likely passage from the top articles so the user can immediately see that answer and/or judge the relevance of the document?  This seems like a rich seam to mine.

So much to think about…

MeSH on Trip

In April I posted that we had started to investigate adding MeSH to Trip. Just a quick note to report that our investigations have successfully been completed, thanks to great help from the NLM.  This has allowed us to auto-tag documents (PDFs or web-pages) with MeSH** with a reasonable accuracy.

So, all documents on Trip will have associated MeSH terms:

  • PubMed documents – autograb the MeSH terms assigned by the NLM
  • Non-PubMed documents (e.g. guidelines, synopses) – auto-tag documents using our new system

After that it’s a question of getting it to work on Trip!  Hopefully within three months…

** Note: only assigns MeSH, not major/minor or subheadings

Next upgrade to Trip

We’re currently planning our next round of updates to the site and below is our current list of potential upgrades. I say ‘current list’ as things tend to change and you – as a reader of the blog and probable user of Trip – have the ability to comment and change things yourself. Either leave a comment below or email me directly: jon.brassey@tripdatabase.com.

PICO search

A much loved aspect of Trip and one that has not been touched since its launch around ten years ago.  We plan to incorporate the work we’ve done around the PICO annotation of article titles. This should make the results even more focused on the PICO elements users have added.

MeSH and advanced search

This is a probably but not definite! We’re working with the NLM on some tools to help us annotate documents with MeSH terms. If they deliver on this, then we’ll introduce MeSH. We plan to use MeSH via an overhauled ‘Advanced search’ and possibly to link it in via the main search, we’ll have to see how that pans out.

Extra full text

We’re should be boosting the amount of full text articles by around 20-25%. Currently we link to PubMed Central, but we will now have the ability to link out to much more.

Search itself

Our algorithm is, like the PICO search, quite old and as such is well overdue an overhaul.  This will improve all sorts of things, such as:

  • Better search results generally
  • Removal/management of low-relevancy articles
  • Better handling of guidelines
  • Various annoying, minor, irritating issues

Content quality control

A few content issues have cropped up, so we’re doing a ‘deep clean’ of those!

Community features

This is a bit vague as we’re doing some experimenting as we go along on this!  But we’ve got two features in mind, both allowing users to interact with the site and share their experience/learning.

 

PICO search – time for improvement(s)? If so, how?

Our PICO search is really popular but it’s not changed since it was introduced years ago.  For those unfamiliar with it, this is what it looks like:

Our search works on a contingency search basis.  So, assuming the user enters 4 search terms (a P, I, C and O) we do an initial search looking for the terms in the title only. If there aren’t many results we repeat with P, I and C as a title search and O as title and text.  If there are still too few, we do P and I as title and C and O as title and text etc.

It seems to work well, but I’m sure it could be improved especially given our work with the KConnect project – where we specifically annotate articles with the PICO elements (albeit only the RCTs and systematic reviews).

But as we look to overhaul it we need your help….

Do you use the PICO search?  Is it perfect?  If not, how might it be improved?

 

 

Understanding (bad) search results

Given the recent interest in guidelines and Trip we’ve had a number of new conversations with users from the USA.  I’ll reproduce (well, slightly edited) one below as I think it’s broadly useful for others to understand a fairly common search problem. In this case the person highlighted how frustrating searching for guidelines can be, so here’s my response:

 

Firstly, as previously mentioned, guidelines are particularly problematic as they typically long and cover a broad range of topics. This creates two different problems – missing them and noise!

Missing them

So, you might have a guideline ‘The diagnosis and management of hypertension‘ which covers loads of areas. It might mention gestational hypertension as a chapter, but nothing more. If you searched for gestational hypertension this guideline would not feature highly as gestational is not mentioned in the title (our system favours matches in the title). So, it appears low down in the search results (even though that chapter might be the best result there is for the user). This is frustrating and we have a few things we’re going to try in the near future to improve this! But, your problem is more the noise…

Noise

Take one of your example search safety mechanical ventilation, when you search on the main Trip (so no refinement) you get the following results:

I’ve highlighted the scores – which relate to how relevant the document is to the search terms (BTW only I get to see those scores). But when you refine to USA Guidelines it gets noisy:

You’ll see, from result 3 the relevancy goes right down. In fact, only 2 (of 85 USA guidelines we return) look reasonable.

This comes back to the size of guidelines. They’re long and invariably the search terms appear within them – but they might mention safety, mechanical and ventilation all in different contexts. But as they include the terms they are counted as ‘hits’ and returned. Search can be stupid!!

The thing that we think we’ll introduce is a relevancy cut-off so we could say only return documents with a relevancy score above 0.1, 0.2 (we’d need to test) but allow users to ‘see ALL documents’ – if they want the noise!

Latest and greatest

This is an often overlooked feature of Trip, so it’s about time I highlighted it.

Latest and greatest takes a topic and looks at the latest evidence for the topic and also the ‘greatest’ – by that, the articles that have been clicked on most for the last 12 months.  A list of topics can be found here but you can access the latest and greatest for any topic via the link at the top of any particular search, for example:

We particularly like the ‘greatest’ side of the feature as it allows uses to easily see the articles deemed most useful/interesting for a given topic.  A bit like a topic-based clinical zeitgeist!

Free and easy to use.

Some examples below:

Autism

For the full list, click here.

Multiple sclerosis

For the full list, click here.

Schizophrenia

For the full list, click here.

The post-NGC landscape, a sample of US guidelines added to Trip this month

I’ve written about our attempts to mitigate the loss of the National Guideline Clearinghouse (even producing a ‘conversion’ how to use Trip guide).  But below are a sample of the new guidelines, added this month, to Trip:

As mentioned, the above is a sample, a modest sample 🙂

Searching for guidelines post-NGC

This post is to help National Guideline Clearinghouse (NGC) users navigate Trip to find the guidelines they need.

Firstly, Trip links to over 3,500 guidelines from the USA (and over 10,000 guidelines in total).  The NGC used to provide summaries to less than half this amount (for a variety of reasons). But Trip is much more than guidelines, we includes a broad range of resources arranged around the evidence hierarchy – as you use Trip you’ll come to appreciate this.

Another thing to consider is Trip’s a very small organisation with a budget a fraction of the NGC and therefore we are not able to mimic all of the sophisticated search refinements of NGC’s.  We are funded via a freemium business model (to understand the differences see the chart here).  Note the guidelines are provided for free but please consider subscribing to help support our efforts (individual and organisational subscriptions are available).

Anyway enough preamble, to search for guidelines navigate to Trip and you’ll see this screen:

I suspect it’s superfluous but I’ve added a big arrow showing where you add the search terms.  Once you’ve searched you go to the results page:

This is the results for all our content so you may want to refine the search for guidelines or USA guidelines, this is easy:

The refine feature is on the right-hand side of the results page. This allows you to refine results by any evidence type, but the two highlighted are for all guidelines and USA guidelines.  If you click on the USA guidelines the results are restricted to just those:

 

Simple!

You can further refine by year (see towards the bottom of the refine area on the results page).  Trip Pro also allows advanced search and refine by clinical area.

For further information on using Trip we have produced a selection of ‘how to’ videos and you may find the ‘Tour’ interesting:

Any further questions, just send them my way: jon.brassey@tripdatabase.com

 

Blog at WordPress.com.

Up ↑