Trip Database Blog

The survey closed last week and with 518 respondents it’s taking time to analyse them.  I’m now ready to post the results from the two questions ‘What’s poor/weak about TRIP?’ and ‘What would make you use TRIP more?’.  In our early analysis of 345 responses we combined these as there was significant overlap, and we’re doing so again.  As with the earlier analysis I’ll show results for 3 categories of users (doctors, information specialists, others).

Doctors

• Full text – making it easier to link to full text.
• Weird results – sometimes finding strange articles in the results.
• Design – main concern was clutter!
• Emails – improve the design and increase the frequency.
• Refine search – make it easier to refine the search results.

Information Specialists

• Advanced search – improve the advanced search options
• Transparency – how do we define the category filters, what sources are searched by TRIP, what synonyms do we use.
• Full text – as above.
• Refine search – as above.
• Design – as above.

Other

• Emails – as above.
• Refine search – as above.
• Full text – as above.
• Transparency – as above.
• Other database – link to a greater number of external databases.

Global

The following are results taken from all respondents, irrespective of background.  This time I’m displaying the top ten.

• Full text
• Transparency
• Refine search
• Advanced search
• Emails
• Weird results
• Design
• Save search – allow users to save their searches and send updates for new hits. NOTE: this already exists but it’s clearly not obvious.
• Mobile friendly – possible create an app or a more dedicated mobile friendly site
• Evidence slider – reinstate the evidence slider. 

I’m really pleased with the results.  There are loads of things for us to consider but none of them are particularly daunting.  I’ll reflect on these over the next few months as well as publishing the remaing results from the survey.

Thank you for taking part.

I’m about 40% of the way through the analysis of the TRIP survey and one thing is clear, a big issue for our users is the ability to easily refine the results.  Users, after an initial search, sometimes find too many results and want to focus the results to a more manageable set of records to examine.

One idea (which I really like) was for the ability to further restrict results based on clinical cateogry.  In other words a person selects ‘Systematic Reviews’, allow them to restrict on a publication basis e.g. only show results from Cochrane, DARE etc.

Seems reasonable to me!

In total we had 518 results, which was a brilliant result.  Early results below (it’ll take me a while to analyse all the results.

Professional background

• Doctor – 45.4%
• Information specialist – 19.9%
• Other – 9.5%
• Other health professional – 8.9%
• Nurse – 8.3%
• Pharmacist – 7.5%
• Patient/carer – 0.6%

• Answering clinical questions – 67.0%
• Research – 57.7%
• Keeping up to date – 49.8%
• Other – 9.8%

Teaching, training and education were included in the ‘Other’ section

How do you use TRIP?

• Pull (search) – 58.9%
• Push (e.g. rss, email) – 7.9%
• Both – 33.2%

What aspect of TRIP do you find works best for you?

• The ease of differentiating between evidence types
• Ease of use
• Wide evidence coverage
• Guideline coverage
• Monthly emails
• Colour coding
• Linking to other databases e.g. Medline, clinical trials
• PICO/Search wizard
• Free
• Speed

Further results will be posted when I’ve analysed them.

One thing I’m wrestling with is the different types of information needs of clinicians.  I’m interested – at the moment – in two broad distinctions:

• Broad – user typically wants background information e.g. eTextbook
• Specific – user wants to answer a specific question e.g. what is the best antidepressant in pregnancy?

TRIP is set-up to answer specific questions.  While we have eTextbooks they appear lower down in the search results (as they are typically of lower quality) so, for a user to consistently see them, needs to click the eTextbook filter.

But can TRIP do more for Broad information needs? Are clinicians conscious of the differences; do they approach TRIP thinking ‘I need background information?’.

Perhaps we need a prominent ‘show background information’ button?  Seems slightly clumsy to be, but plenty of time to ponder!

Earlier today we posted some preliminary results of our survey (click here).  I’ve now had a chance to look at the main textual results and – while viewing them – I spotted the overlap to responses to the questions ‘What is bad about TRIP?’ and ‘What features would make you use TRIP more?’.

The response has been great and has been really useful in helping to focus our minds.  Below is a brief overview of the main results.  I’ve given a global result plus broken down into 3 categories based on profession (apologies for lumping in ‘Other health professionals’):

Doctors (top 3 improvements)

• Access to full-text articles
• Improve the advanced search
• Smart phone app

Information specialists

• Improve the advanced search
• Clarity of the results (how the results are arrived at)
• Poor, unrelated articles appearing in the results

Other health professionals

• Refine search – make it easier
• Improve the monthly email
• Improve the Boolean search

Global results for all respondants (top 7)

• Refine search – make it easier
• Improve the advanced search
• Access to full-text articles
• Poor, unrelated articles appearing in the results
• Improve the monthly email
• Improve the Boolean search
• Clarity of the results (how the results are arrived at)

Overall, lots of good ideas.  The one worry for us is the frequent reporting of ‘wrong’ articles appearing in the search results.  Something for us to deal with well before the next upgrade.

As ever, one big thing that was apparent from the users – the love for TRIP 🙂

We posted our latest survey yesterday afternoon and so far we’ve had an amazing 345 responses (if you’ve not done it, it’s not too late https://www.surveymonkey.com/s/WMS38PY).

I’m really keen to get our users input into the next upgrade to TRIP (scheduled for early 2012) and this is the first stage survey.  We asked people to leave their emails if they want to take part in further surveys and we’ve already got 200+.  The first survey asks mainly high-levels questions (e.g. what’s good about TRIP, what’s bad etc).  The second survey will ask more specific questions about functionality, design etc.  I really can’t wait.  But before the second survey I have to finish reading a few books and papers on search and to fully analyses the first round of results. 

A lot of the questions will require some form of textual analyses but other lend themselves to a simple numerical analysis, so some early results below:

Background (top 3)

• Doctors – 44.1%
• Information specialists – 18.8%
• Other health professionals – 10.7%

What do you use TRIP for?

• Answering clinical questions – 69.6%
• Keeping up to date – 50.7%
• Research – 56.5%
• Other – 10.1%

How do you use TRIP?

• A pull way (searching the site) – 55.9%
• A push way (monthly emails, RSS etc) – 8.4%
• Both – 35.7%

One issue we’re looking at is the content offering of TRIP, what best describes it for you?

• Perfect – 59.1%
• Way too much – I get too many results – 24.3%
• Way too few – I often get few and/or poor results – 16.5%

Another area of interest is the design of TRIP (how it looks). We have recently rolled out a new design and we’re not convinced it’s great. What do you think?

• The site looks great – 22.9%
• I’m not bothered how the site looks – 26.1%
• I think it looks ok – 47.5%
• I think it looks awful – 3.5%

If you were required to login to TRIP to use it, how would you feel?

• I’d stop using the site – 5.5%
• I’m happy to do it, but make it easy – 53.0%
• I’m happy to do so, as long as I get a better service – 24.9%
• I’m not too keen on this idea – 16.5%

So, lots of interesting results.  But the biggest ‘steer’ will be the textual analysis of the 3 questions:

• What’s good about TRIP?
• What’s bad about TRIP?
• What feature would make you use TRIP more?

Bing, the world’s 2nd best used general search engine (after Google) has just announced the release of adaptive search.  A brief overview can be seen via this searchengineland blog article.

Adaptive search excites me enormously and it’s something we’re working on at the moment.  I say working on, we’re working on the theory and how it might work.  We need to save up our pennies and test a few assumptions before building it!

The principle is that adaptive search learns the type of person you are and alters the results based on what it has learnt about you.  This means different people would see different results.  For instance, a general practitioner in the UK might search for hypertension and s/he would see different results to a cardiologist based in Canada.  It makes sense that they see different results as their contexts are different.

We’re actually pretty confident with the theory but the biggest assumption is, for this to work, people need to login.  So, we’re exploring that and allowing people to login with their Facebook and Twitter accounts should make this easier. 

This may seem a strange title as many of you will feel that TRIP is an evidence-based tool already.  Well, TRIP helps users find the best available evidence, I’m comfortable with that.  However, there is more to this than that….

Since the new interface/functionality went live I’ve been tidying up loose ends, fixing a few broken bits of functionality and looking forward to the next changes to TRIP.  This has coincided with a wonderful opportunity to get involved with a research grant that will explore creating a search interface based on academic theories around information gathering.  This will involve creating a series of interfaces which will then be tested with a group of clinicians.  So, we’ll be doing research to explore various aspects of search interface design and results display.  We’ll be creating an evidence-base for clinical search.

I’ve no idea if we’ll get funding but the person in charge is hugely influential so I think we’ve got a reasonable chance.  Even if it doesn’t happen I’ve already learnt an awful lot from just reading the background papers. 

Exciting times!

The latest upgrade to TRIP had lots of new features easily visible to users.  However, for all the external changes we have made a number of internal ones.  One of the unhidden wonders has been a broken link reporter!  Basically, if someone clicks on a link and it doesn’t result in a valid webpage being open we now get an email telling us the link isn’t working.  Previously, we had no system for this and it was down to the goodwill of our users to let us know.

When the new feature went live it was truly depressing as I was personally receiving about an email every minute.  However, it soon showed up that a few particular publications were causing the bulk of the issues. Within 4 days these links were fixed – causing a 90% reduction in the error messages.  Now, 3 weeks later and lots of hard work, we’ve fixed approximately 99% of all the broken links.

For most people they will not notice a difference – but it’s great to know our system’s working well and long may it continue.