Locating new content on TRIP

TRIP aggregates thousands of new articles per month.  The majority are from primary research but we also include 500-750 new articles from secondary sources e.g. systematic reviews, guidelines.

We appreciate that this is important information and have two mechanisms for helping users identify new content.

The first is the ability to carry out a search and only select content added in the last month.  The image below highlights where this feature is.

The second has been our efforts to categorise each new piece of content by various clinical categories.  This has been acheived in three main ways:

• By publication type.  For instance we assume each article in the journal Cancer is appropriate for our oncology updates.
• By keyword matching.  This is slightly trickier!  We have over 25 clinical categories in TRIP (e.g. cardiology, oncology) and we started by creating a long list of keywords/terms associated with that clinical area.  For instance, the following words are some example terms associated with cardiology: atrial fibrillation, cholesterol, hypertension.  In total we have identified 334 cardiology terms (we're not claiming it's exhaustive).  What we have done is cross check each document title in TRIP (NOTE: title words only) with all the category words to assign clinical categories to each document.  A document can have multiple categories.  For instance a document called 'Prostate cancer screening in the elderly' would be assigned to urology, oncology and geriatrics.  This is not foolproof and only today I have spotted a problem.  We have the word sinus associated with Otolaryngology-ENT which unfortunately returns documents with sinus rhythm in the title!  However, the errors are relatively few and far between and we hope the benefits far outweigh the negatives! 
• Manual identification.  In the case of primary care we cannot assign keywords, so this content is manually identified and typically restricted to the secondary evidence base.

You can view the new content via the links below.  Note: these are updated monthly, after we carry out the manual upload of new content.

• Allergies and Immunology
• Anesthesiology
• Cardiology
• Critical Care
• Dentistry
• Dermatology
• Emergency Medicine
• Endocrinology
• Gastroenterology
• Geriatrics
• Hematology
• Infectious Disease
• Neurology
• OB-Gyn
• Oncology
• Ophthalmology
• Orthopedic
• Otolaryngology-ENT
• Pediatrics
• Physical Medicine
• Primary Care
• Psychiatry
• Pulmonology
• Radiology
• Rheumatology
• Surgery
• Urology

As mentioned above our methods aren't foolproof and we welcome feedback.  Let me know via jon.brassey@tripdatabase.com

Making search slightly smarter

I'm really pleased to announce a new feature to TRIP, one we've been working on for a while now.  It also supports one of the major requests from the recent survey - help us refine our search (which was the 3rd most requested improvement).  This new feature is not our only answer to search refinement but it's a great start.

The new feature allows users to refine search by clinical area.  So, you can do a search for - say - measles and then restrict these results to neurology documents.  The results will then be restricted to those articles that are related to both measles and the broad area of neurology.

How does this affect the results?  The first set of 3 results (below) are when you do a search for measles while the second set of results are the top 3 results when searching for measles and restricting to neurology.

Measles (top 3 results)

• Antibiotics for preventing complications in children with measles (Cochrane)
• Vitamin A for treating measles in children (Cochrane)
• Routine vitamin A supplementation for the prevention of blindness due to measles infection in children (Cochrane)

Measles restricted to Neurology (top 3 results)

• Measles-mumps-rubella-varicella combination vaccine and the risk of febrile seizures (Pediatrics)
• Measles virus-specific plasma cells are prominent in subacute sclerosing panencephalitis CSF (Neurology)
• MMR vaccination and febrile seizures: evaluation of susceptible subgroups and long-term prognosis (JAMA)

But the best way to see the power is to simply give it a go and try it out.  I really am pleased with it.

To help you orientate yourself, the picture below shows you were to find it!

How does it work?

Another significant piece of feedback from the survey was one of transparency - how does TRIP work.  So, this seems a good start at being more transparent.

We have over 25 clinical categories in TRIP (e.g. cardiology, oncology) and we started by creating a long list of keywords/terms associated with that clinical area.  For instance, the following words are some example terms associated with cardiology: atrial fibrillation, cholesterol, hypertension.  In total we have identified 334 cardiology terms (we're not claiming it's exhaustive).  What we have done is cross check each document title in TRIP (NOTE: title words only) with all the category words to assign clinical categories to each document.  A document can have multiple categories.  For instance a document called 'Prostate cancer screening in the elderly' would be assigned to urology, oncology and geriatrics.

When someone does a search on TRIP we examine all the returned documents and build a category filter based on the assigned clinical categories (most popular at the top).  By clicking on a single category the documents are restricted to that category. 

NOTE: the category 'Primary care' is an exception as we cannot find suitable keywords to adequately identify suitable documents.  These are therefore assigned manually each month.  This started relatively recently so the cohort of suitable documents is small at present.

Finally, given the nature of the methods used we do not claim that it's foolproof.  The system will invariably miss some documents and include some documents that it shouldn't.  Those minor irritants aside it's still really powerful - enjoy.

I hope this all makes sense, if not get in touch via jon.brassey@tripdatabase.com

An answer engine

I gave a talk at the recent Public Health Wales staff conference (I work part-time for them) about my experience of answering clinical questions, the usefulness of evidence and the problem with search.  I highlighted that clinicians want answers to their questions, not links to ten articles that might contain their answer (or part of it) - which is what search does.

As you may well know I'm passionate about answering clinician's questions and I view search as - at best - a partial solution.  As TechCrunch reported in 2008:

"Despite attempts to evolve search into something more human friendly, there’s still a big hole there. As useful as Google is, it doesn’t answer questions very well"

I also heard an interview by Steve Wozniak (who co-founded Apple with Steve Jobs) and he too was talking about the limitations of search and how we need to create an answer engine. 

What would be needed to create an answer engine for clinicians?  TRIP Answers has a collection of over 6,000 Q&As but what would it take to make that 60,000 large? 

I am very tempted to try and create an open repository of clinical Q&As and allow users (probably after a vetting process) to upload their own Q&As.  Why not cut up guidelines into their constituent recommendations?  Why not cut-up individual chapters in - say the Green Book - into separate Q&s?

To do this requires volunteers, co-explorers, people believing that clinician's want answers to their questions.  Anyone else fancy transforming the clinical information space?

Survey: final section of analysis

The final results from the analysis are below.

Content

• Perfect - 59%
• Not enough content - 16%
• Too much content - 25%

How the site looks

• I think it looks awful - 4%
• Ok - 49%
• Not bothered - 26%
• Great - 21%

If you were required to login to TRIP to use it, how would you feel?

• I'd stop using the site - 6%
• I'm happy to do it, but make it easy - 53%
• I'm happy to do so, as long as I get a better service - 25%
• I'm not too keen on this idea - 16%

It's interesting to note how similar the results are compared with our early analysis (based on 345 respondents), the above is based on 518 respondents.
So, the results are all out.  It now just requires a fair amount of reflection, which I'll post on the blog at a later date.  I'll then plan round two of the survey - which will be trying to really focus on the issues and how we can best meet them.

Survey: more results

The survey closed last week and with 518 respondents it's taking time to analyse them.  I'm now ready to post the results from the two questions 'What's poor/weak about TRIP?' and 'What would make you use TRIP more?'.  In our early analysis of 345 responses we combined these as there was significant overlap, and we're doing so again.  As with the earlier analysis I'll show results for 3 categories of users (doctors, information specialists, others).

Doctors

• Full text - making it easier to link to full text.
• Weird results - sometimes finding strange articles in the results.
• Design - main concern was clutter!
• Emails - improve the design and increase the frequency.
• Refine search - make it easier to refine the search results.

Information Specialists

• Advanced search - improve the advanced search options
• Transparency - how do we define the category filters, what sources are searched by TRIP, what synonyms do we use.
• Full text - as above.
• Refine search - as above.
• Design - as above.

Other

• Emails - as above.
• Refine search - as above.
• Full text - as above.
• Transparency - as above.
• Other database - link to a greater number of external databases.

Global

The following are results taken from all respondents, irrespective of background.  This time I'm displaying the top ten.

• Full text
• Transparency
• Refine search
• Advanced search
• Emails
• Weird results
• Design
• Save search - allow users to save their searches and send updates for new hits. NOTE: this already exists but it's clearly not obvious.
• Mobile friendly - possible create an app or a more dedicated mobile friendly site
• Evidence slider - reinstate the evidence slider. 

I'm really pleased with the results.  There are loads of things for us to consider but none of them are particularly daunting.  I'll reflect on these over the next few months as well as publishing the remaing results from the survey.

Thank you for taking part.

Refining search

I'm about 40% of the way through the analysis of the TRIP survey and one thing is clear, a big issue for our users is the ability to easily refine the results.  Users, after an initial search, sometimes find too many results and want to focus the results to a more manageable set of records to examine.

One idea (which I really like) was for the ability to further restrict results based on clinical cateogry.  In other words a person selects 'Systematic Reviews', allow them to restrict on a publication basis e.g. only show results from Cochrane, DARE etc.

Seems reasonable to me!

Survey ended, start of analysis

In total we had 518 results, which was a brilliant result.  Early results below (it'll take me a while to analyse all the results.

Professional background

• Doctor – 45.4%
• Information specialist – 19.9%
• Other - 9.5%
• Other health professional – 8.9%
• Nurse – 8.3%
• Pharmacist – 7.5%
• Patient/carer – 0.6%

What do you use TRIP for?

• Answering clinical questions – 67.0%
• Research – 57.7%
• Keeping up to date – 49.8%
• Other – 9.8%

Teaching, training and education were included in the 'Other' section

How do you use TRIP?

• Pull (search) – 58.9%
• Push (e.g. rss, email) – 7.9%
• Both – 33.2%

What aspect of TRIP do you find works best for you?

• The ease of differentiating between evidence types
• Ease of use
• Wide evidence coverage
• Guideline coverage
• Monthly emails
• Colour coding
• Linking to other databases e.g. Medline, clinical trials
• PICO/Search wizard
• Free
• Speed

Further results will be posted when I've analysed them.