Trip Database Blog

The TRIP Database was designed to help me answer questions for the ATTRACT service, a service I’m still involved in.  The ATTRACT methodology involves receiving a clinical question from a clinician and answering it, using the best available evidence, within 4-6 hours.  It’s clear to see that this is not a systematic review!  However, it’s unclear what the effects are so I’ve been doing some searching for evaluations of rapid reviews versus systematic reviews.  It’s early days in the review but a couple of interesting papers are:

• Expediting systematic reviews: methods and implications of rapid reviews
• Evidence summaries: the evolution of a rapid review approach

If you know of others please let me know.

Frequently, when reading around a subject I get side-tracked in thoughts and started thinking about carrying out rapid reviews from within TRIP.  It might work something like this:

1. The user starts by using the TRIP PICO search (click here to see it).
2. A user selects those articles that s/he wants to view.
3. These are then all opened up in new windows framed by a special TRIP frame (so we can do some clever stuff highlighted below).
4. The user can read the articles, highlight passages they want including in the review and ‘send’ them to a review builder on the TRIP site (by simply pressing a button)
5. After this process is carried out the user goes to the review builder, add a narrative to link the passages and TRIP publishes the reviews (after also adding references, etc)

We could even monitor new content added to TRIP and alert the original creator when new articles on the topic are published (arguably we could add them to the actual review as well in a ‘new evidence’ section!

It’d work for ATTRACT and I’m guessing it might work elsewhere…!

I read about an interface for PubMed that had created a proximity search.  I wasn’t sure what ‘proximity search’ was and I imagined it was a geographic proximity.  My thoughts were going down the geographic route due to my interest in geolocation and search in TRIP.   Currently, we take no account of geography in our search results so the UK, Canadian, USA etc guidelines all have similar weights.  However, I’ve thought for years (and probably blogged about it) that it makes sense if you’re searching from Australia, Australian guidelines appear higher up the results than others.

However, after some digging, it appears that it relates to proximity of search terms.  In other words only return documents where two search words are close.  I wasn’t convinced about this so asked our Facebook followers (click here for our Facebook page) and one comment said that it can be useful and gave the example of searching for drug abuse.  So, you could return documents where drug and abuse appear close by.  It made perfect sense to me, I hope it does to you.

But, my thoughts returned to the geographic proximity issue.  I think the following scenario sounds great:

TRIP works with librarians within organisations (e.g. hospitals, universities) to allow organisational accounts which allows 3 things to happen:

• The organisation can load their own documents (e.g. guidelines) for searching.
• The organisation uploads their link resolver (allowing easy resolution to full text documents)
• The organisation promotes their organisational TRIP account to those working/studying in the organisation so they can search the above local documents and/or easily link-out to full text documents.

So, two questions:

• How does the above scenario sound?
• Do people like the idea of proximity searching where people can find words that appear close to each other?

Really interesting questions for me and hopefully you!

I’m still going over the future direction of TRIP and it’s exciting.  We have one potential buyer that we’re talking with and that’s potentially a great move for TRIP – making it secure and allowing investment in the product.However, we cannot allow the potential purchase to divert us from planning for the future. 

As well as all the design and functionality issues mentioned before (see here) I’ve been thinking about developing a premium version of TRIP.  TRIP, as is, would remain free but we’d add functionality to a ‘paid for’ version.  I can see a number of possible enhancements that might make subscribing worthwhile, for instance:

• Development of the answer engine, possibly including content from other subscription services.
• Linking out to full-text.  If an organisation subscribed the library could add the link resolver details to the site and a user would then be able to seamlessly link-out to full-text.
• Allow the organisation to add their own content to the site (e.g. local guidelines, pathways etc).

If we don’t get bought I’m not sure where the money will come from – we’ll see!

Relatively early days of this year and it’s already proving interesting.

I’m still working on the answer engine idea (see here for last post on the topic) and have had the first meeting about that.  There are lots of issues to be resolved, but these appear fairly clear and none particularly worry me!

One topic that has concerned me is the relationship between TRIP and any answer engine we create.  Fortunately, that has clarified over the months and I can easily see a fit that allows a users to search TRIP and get a combined set of results – TRIP results and associated answers.  I’ve done some mock-ups and it fits seamlessly.  So, very excited. 

As ever money is the limiting factor but we may have some good news on that front in the near future.  But, due to this and commercial sensitivities around the answer engine I’m having to stop being quite so open and transparent – which is a real shame.  I think that helps explain why there have been few blog posts this year – even though it has been a really exciting start.

Those of you who know me or have got a feel for TRIP (via this blog or using the site) will know that our biggest motivation is allowing clinicians to get rapid answers to their clinical questions. The TRIP Database has been running for nearly 15 years and we’ve helped shape clinical search, spawning many similar search tools (perhaps the most recent being the eye-wateringly expensive (and soon to be rebranded) NHS Evidence). 
However, over the last few years my reservations about clinical search has grown.  To define that further, I mean clinical search for busy clinicians (as opposed to librarians, information specialists, academics etc).  Those that need a rapid answer to their clinical questions.

The current search paradigm is that users add 1-3 search terms, press ‘search’ and sift through the top 10-20 results to find the answer they’re looking for.  Below are a few problems with this (there are many more):

• We know that clinicians are typically poor searchers, so there’s a handicap from the start.
• The way clinician’s select which papers/documents to look at is problematic based on a number of factors.
• From our experience of Q&A (answering over 10,000 clinical questions) we know the average number of references to answer a question is over two.  So, that suggests that clinicians will need to open 2+ documents and read them to find the information they need – highly inefficient, especially if the document is long.

Using the example of NHS Evidence, they have a vast resource (£700,000+ for marketing alone.  Their marketing budget is nearly 30 times higher than the entire TRIP Database budget!), a good brand, a competent implementation, part of the NHS ‘family’ yet have embarrassingly low search stats.  I believe one of the main reasons is that search is not something that works for most clinicians.  It’s a paradigm, defined by Google and people seem happy to settle for it.  Is it any wonder that clinician’s main source of answers to their questions is to ask a colleague?  One obvious reason that clinicians ask other clinicians  (but there are clearly others) is that they get an answer – not 10-20 links that may answer their question.

So, to me, any solution is to go back to first principles – in this case a clinician with a clinical question.  What do they want? An answer.  I’m hoping that’s not controversial – it seems obvious to me. 

To reiterate, imagine you’re a busy clinician and have a clinical question, what would you prefer:

• A robust answer.
• A list of 10-20 results, any number of which may contain all or part of your answer.

Is it only me that sees this as a ‘no brainer’?

If you’re one of the people that thinks the latter – please contact me (jon.brassey@tripdatabase.com) as I’d love to understand your perspective better

Anyway, moving on to the notion of delivering an answer – this is where it gets complicated but also interesting.  I firmly believe that we shouldn’t shy away from a challenge, not one that’s so important as this.  With the experience gained in TRIP (with search and Q&A) I actually think that the issue is manageable.

Basically, I’m planning on building a system that will take a clinical question and deliver an answer. 

However, one thing for sure, it’s too big for TRIP to do it on it’s own.  So, we’ve started by getting together a small group of people representing organisations who have a vested interest in getting this right.  Those who share in the vision.  To me, the biggest challenge will be managing the disparate bodies – from the small to the very large.

If I can pull this off, we’ve got every chance of making an industry-wide change for the best.

Do I need to apologise for there being such a gap (one month) between my last post and this?  I’ve no idea!
Basically, we’ve been working very hard on two main areas of work:

• The next upgrade of TRIP.  This should appear April/May 2012 and has taken lots of work with the surveys, interviews and reading around the topic.
• An answer engine.  For commercial reasons I have to be vague on this for now (something I’m not comfortable with) but it’ll take shape over 2012.  I’ve identified a number of partners to get this collaboration going and our first meeting should be in February next year.  While my enthusiasm is driving this (initially at least) it won’t be a TRIP product, we will be one of the partners.  I’ve come to realise that TRIP can only achieve so much on our own.  But the initial partners are very strong and should help me realise my dreams.

All the above aside, I’m starting to wind down for Christmas.  It’s been a long year and I need it.

If you celebrate Christmas – enjoy.

Best wishes

jon

TRIP typically gets an overhaul once a year and ahead of changes for early 2012 we’ve started a series of surveys asking people what they like and don’t like about TRIP.  I’ve already posted three posts on the first round of results (see here, here and here).  This allowed me to draw up a list of proposed changes which I’ve discussed with the TRIP techie – Phil.  As a result of this discussion I started a second round of questions, to probe some of the issues raised in the first round of questions. 

One area of real interest is the ‘Associated results’ on the right-hand side of the TRIP results page (image below).

These are resources that could be useful to a user, but are not a core part of the TRIP index.  These are problematic for a few reasons:

• They increase the time it takes for the page to load.
• It’s not scalable, in that we could put many more resources there – but it wouldn’t work.
• It looks messy.

So, one question I asked in the 2nd survey was “One area we’re looking at are the results on the right-hand side that link to clinical trials, PubMed, BNF etc. We’re thinking they may be distracting, after all you’re coming to TRIP for the great set of main results. So, which of these sounds best?”

I think I was hoping that most would indicate that they don’t use them and we could, perhaps, relegate them to the bottom of the results page.  But, and reinforcing why it’s useful to ask your users, here are the results so far:

• Leave them alone, they’re great and I use them all the time – 15.8%
• Leave them alone, they’re useful and I use them from time to time – 52.6%
• Remove them completely I rarely/never use them – 2.6%
• Make them less prominent e.g. at the bottom of the results so they can be used if the answer can’t be found in TRIP – 28.9%

So, nearly 70% of users want them left alone!  This creates a dilemma as to how to increase the number of associated results yet still use no more space.  I have some ideas (e.g. use a table or buttons that shows results only when clicked) but one to be worked through with our designers.

Irrespective of the outcome – I’m really glad I asked!

TRIP aggregates thousands of new articles per month.  The majority are from primary research but we also include 500-750 new articles from secondary sources e.g. systematic reviews, guidelines.

We appreciate that this is important information and have two mechanisms for helping users identify new content.

The first is the ability to carry out a search and only select content added in the last month.  The image below highlights where this feature is.

The second has been our efforts to categorise each new piece of content by various clinical categories.  This has been acheived in three main ways:

• By publication type.  For instance we assume each article in the journal Cancer is appropriate for our oncology updates.
• By keyword matching.  This is slightly trickier!  We have over 25 clinical categories in TRIP (e.g. cardiology, oncology) and we started by creating a long list of keywords/terms associated with that clinical area.  For instance, the following words are some example terms associated with cardiology: atrial fibrillation, cholesterol, hypertension.  In total we have identified 334 cardiology terms (we’re not claiming it’s exhaustive).  What we have done is cross check each document title in TRIP (NOTE: title words only) with all the category words to assign clinical categories to each document.  A document can have multiple categories.  For instance a document called ‘Prostate cancer screening in the elderly’ would be assigned to urology, oncology and geriatrics.  This is not foolproof and only today I have spotted a problem.  We have the word sinus associated with Otolaryngology-ENT which unfortunately returns documents with sinus rhythm in the title!  However, the errors are relatively few and far between and we hope the benefits far outweigh the negatives! 
• Manual identification.  In the case of primary care we cannot assign keywords, so this content is manually identified and typically restricted to the secondary evidence base.

You can view the new content via the links below.  Note: these are updated monthly, after we carry out the manual upload of new content.

• Allergies and Immunology
• Anesthesiology
• Cardiology
• Critical Care
• Dentistry
• Dermatology
• Emergency Medicine
• Endocrinology
• Gastroenterology
• Geriatrics
• Hematology
• Infectious Disease
• Neurology
• OB-Gyn
• Oncology
• Ophthalmology
• Orthopedic
• Otolaryngology-ENT
• Pediatrics
• Physical Medicine
• Primary Care
• Psychiatry
• Pulmonology
• Radiology
• Rheumatology
• Surgery
• Urology

As mentioned above our methods aren’t foolproof and we welcome feedback.  Let me know via jon.brassey@tripdatabase.com

I’m really pleased to announce a new feature to TRIP, one we’ve been working on for a while now.  It also supports one of the major requests from the recent survey – help us refine our search (which was the 3rd most requested improvement).  This new feature is not our only answer to search refinement but it’s a great start.

The new feature allows users to refine search by clinical area.  So, you can do a search for – say – measles and then restrict these results to neurology documents.  The results will then be restricted to those articles that are related to both measles and the broad area of neurology.

How does this affect the results?  The first set of 3 results (below) are when you do a search for measles while the second set of results are the top 3 results when searching for measles and restricting to neurology.

Measles (top 3 results)

• Antibiotics for preventing complications in children with measles (Cochrane)
• Vitamin A for treating measles in children (Cochrane)
• Routine vitamin A supplementation for the prevention of blindness due to measles infection in children (Cochrane)

Measles restricted to Neurology (top 3 results)

• Measles-mumps-rubella-varicella combination vaccine and the risk of febrile seizures (Pediatrics)
• Measles virus-specific plasma cells are prominent in subacute sclerosing panencephalitis CSF (Neurology)
• MMR vaccination and febrile seizures: evaluation of susceptible subgroups and long-term prognosis (JAMA)

But the best way to see the power is to simply give it a go and try it out.  I really am pleased with it.

To help you orientate yourself, the picture below shows you were to find it!

How does it work?

Another significant piece of feedback from the survey was one of transparency – how does TRIP work.  So, this seems a good start at being more transparent.

We have over 25 clinical categories in TRIP (e.g. cardiology, oncology) and we started by creating a long list of keywords/terms associated with that clinical area.  For instance, the following words are some example terms associated with cardiology: atrial fibrillation, cholesterol, hypertension.  In total we have identified 334 cardiology terms (we’re not claiming it’s exhaustive).  What we have done is cross check each document title in TRIP (NOTE: title words only) with all the category words to assign clinical categories to each document.  A document can have multiple categories.  For instance a document called ‘Prostate cancer screening in the elderly‘ would be assigned to urology, oncology and geriatrics.

When someone does a search on TRIP we examine all the returned documents and build a category filter based on the assigned clinical categories (most popular at the top).  By clicking on a single category the documents are restricted to that category. 

NOTE: the category ‘Primary care‘ is an exception as we cannot find suitable keywords to adequately identify suitable documents.  These are therefore assigned manually each month.  This started relatively recently so the cohort of suitable documents is small at present.

Finally, given the nature of the methods used we do not claim that it’s foolproof.  The system will invariably miss some documents and include some documents that it shouldn’t.  Those minor irritants aside it’s still really powerful – enjoy.

I hope this all makes sense, if not get in touch via jon.brassey@tripdatabase.com