Trip Database Blog

We’ve just had our 16,000 registered user of TRIP, that’s 1,000 new registrants in 3 weeks.

Our 16,000 user was a student from Fiji!

Here are a few recent Facebook posts (to see our full Facebook page click here).  Remember, the Facebook posts are typically small posts that are perhaps not ‘worthy’ of a blog post (but feel free to contradict me!)

• The BNF is now live on the TRIP Database (February 12th)
• Just about to add 300+ clinical guidelines from the Australian guidelines portal (February 15th)
• Just added records from two sites aimed at undergraduate medical students: Almostadoctor & MediVids (February 17th)
• Highlighted how good TILT was at highlighting significant new research (February 25th)
• Work on updating the systematic review filter that auto-searches PubMed (February 28th)
• Reported on the positive response to a training session I ran for a NHS organisation and offering to run more courses – just ask (March 2nd)
• I’m looking to introduce OpenURL into TRIP and gradually understanding the issues/complexity (March 3rd)
• Exploring hooking TRIP up to Google Translate (e.g. Spanish & Welsh) (March 7th)

We’ve been busy!

When users register on TRIP they can record keywords of interest and/or clinical areas of interest.  By doing this we send them a monthly email with new content that matches their interests. This information is displayed in the following format (click on image to enlarge):

I’m not a huge fan of this look as I think it’s too complicated. I raised this with Phil (the TRIP techie) and he wasn’t convinced by some of my ideas – so he suggested I ask the users what they thought!  The results are below…

When the monthly email arrives from TRIP, do you

• Ignore it – 2%
• Skim read it – 35%
• Follow some of the links – 62%
• I never receive them – 1%

The TRIP emails currently contain links to all new content. We are considering including the “top” 2 or 3 articles in each subject/interest area – highlighting them (title, publication and URL) in the email. What do you think?

• Sounds good, yes please – 77%
• Sounds good, but I’m worried it would make the email too long – 15%
• No thanks, there’s already too much information – 8%

Overall, how would you rate the emails

• Great – 22%
• Good – 56%
• Ok – 17%
• Poor – 5%
• Awful – 0%

What do you most like about the current emails?

• short, infrequent
• They are useful for concise
• They are well-organized and easy to use
• All wanted information is very clear arranged and I can grasp on more details with only one klick.
• Helps me with current awareness derived from a credible source! I often suggest my clients sign up for these emails directly

What do you most dislike?

• tough to see the significance without ANY detail
• Some features didn’t appear to work, specifically, I’ve clicked and chosen some of the links and have tried both the email and the send to rtf file but neither of those features have worked for me.
• Not enough information to help me decide whether to click on links
• Too wordy. Bullet points with links if the item looks interesting
• There is a lot of unrelated information in the links. Articles come up that have nothing to do with the topic.
• No information about the top articles in the emails compels readers to click on the multiple links. If article titles are listed, like Table of Content alerts from Journals, would be more user friendly.

We also asked for additional comments and got a number – including some significant love and fondness for TRIP.  A few raised issues of accuracy of the results (an issue we’re aware of an looking into).  The issue of primary care content raises it head yet again. The content for the emails is automated and I can think of no way of automatically deciding if an article is suitable for primary care or not.

Overall, it appears that most people are happy with the emails (I’m surprised) but a number would like additional information displayed to give a flavour of the content.  In the above example, with 12 interests and over 3,000 links – how do you select 4-5 that give a flavour?

These are all challenges that we need to deal with.  Fortunately, as people are broadly happy with the emails there is no huge pressure – so we’ll include it in our next upgrade (in 2-3 months). 

But if anyone cares to make a suggest we’d love to hear from you…

A while ago I read this article on TechCrunch Marissa Mayer’s Next Big Thing: “Contextual Discovery” — Google Results Without Search and it inspired me.  It helped me discover a recurring theme in my thoughts (and on this blog) that search is useful for known unknowns. The flipside of that is unknown unknowns, something you didn’t know you didn’t know.  In the above article Mayer talks about pushing information to people.  She talks alot about location but it doesn’t need to be restricted to that.

Currently on TRIP we have the ability for people to record their clinical area of interest and/or specific keywords of interest.  The specific keywords works reasonably well.  You say you’re interested in stable angina, and we email you every article that is about stable angina – simple and the feedback is positive.  Move to the wider clinical areas of interest (e.g. cardiology) and things get more problematic as it’s so broad.

But what if you’re like one of the people who doesn’t register or if they do they don’t record any interests?  Fine you use TRIP, it works well and that’s arguably enough.

But I want TRIP to do more.

If we encourage people to login we can start to note the articles they click on and start to create clinical hot spots of interest.  When they next login to TRIP (or we email them) we show them new articles that match there previous interests.  They don’t need to search.  There are a few issues (e.g. does previous searching predict future interests?) but I feel this is an avenue worth pursing.  I’m actually confident we can create these clinical hotspots and use that to the benefit of our users.

But what do you think?  Please comment to let me know.

In the middle of last year I attempted to draw conclusions about the difference in costs between TRIP and NHS Evidence (click here for the full blog post).  Since then I’ve been trying to understand the costs of NHS Evidence and have managed to arrive at the following figures (NOTE: I’ve had to use Freedom of Information requests which are tedious see here & here.  I welcome NHS Evidence correcting any figures below):

• Total budget – £24,438,000
• Content costs (BNF, Cochrane) – £10,675,971
• Remaining costs (This includes the budget for the specialist collections which I understand is approximately £2,000,000 for all 34) – £13,762,029

Of those remaining costs, here’s a flavour of the spending:

• Consultants and other temporary staff – £2,813,258
• Staff costs (the 41 employees) – £2,863,742
• Marketing – £730,000

Using the same search figures as before = 15,811,716 searches at a cost of £13,762,029 (excluding content costs from the total budget).  BTW since I quoted that figure a few people have suggested that actual figure for number of searches is too high as it includes search figures for other databases managed by NHS Evidence.  However, a specific question, via Freedom of Information, refutes this – so I will use this higher figure.

Each search on NHS Evidence costs 87.04 pence

TRIP on the other hand has now reduced its costs (on the TRIP Database) and will this year run on a maximum of £35,000.  Using the same search figures as before (8,058,648) this equates to a cost per search of 0.43 pence per search.

Therefore, each search on NHS Evidence costs 202 times more than on TRIP.

Why does this bother me?  A number of reasons:

• The NHS is facing massive challenges to the budget yet NHS Evidence appears immune.
• NHS Evidence distorts the market and TRIP suffers. 
• NHS Evidence does not appears to be engaging with librarians (that’s certainly the feel I get from conversations and emails). The recent decision to give the specialist collections to non-librarian consortia reinforces this feeling.

But the main one is that I don’t feel they offer a superior service to TRIP.  I would go as far as to say that I feel TRIP is significantly better at supporting real, frontline clinical staff.  However, I still stand by my main conclusion of last year that search isn’t the answer to properly supporting clinicians to practice evidence-based healthcare.

With regard to these latter points I’d love to have the funds to test this.  NHS Evidence got any spare change?

UPDATE: We’ve had an email from someone who used to work on the predecessor of the NHS Evidence – the National Library for Health (NLH).  They point out that the old budget for the NLH was approximately £9,000,000.  The only significant difference between the two (in relation to funding content (is the £5,000,000 paid to the BNF.  So, comparing the old NLH and the new NHS Evidence there is a difference in funding level of around £10,000,000. They would like to know what does that extra money get you?

I’ve no idea, perhaps NHS Evidence can tell us?

UPDATE 2: I tweeted about this post and subsequently saw this tweet from Ben Goldacre (@bengoldacre) he of Bad Science fame:

A search on NHS Ev costs 200X one on TRIP: I find TRIP better

Below is a copy of an email I sent to one of the design teams we use, I’m hoping it’s self-explanatory.  Feel free to comment on the idea and possible approach.

In TRIP we add around 4-5,000 new articles per month and currently this gets added to the index. In a way this hides it. I feel we could make it much more visible and accessible.



I’m increasingly aware that to search you need to know what you’re looking for ie a known unknown. I want to create a space where people can browse for content without really knowing what they’re looking for, so more like unknown unknowns.

A few points:

• I’d like users to be able to select the time period (from 1 month to 12 months) perhaps some sort of visual system e.g. slider.
• I’d like them to be able to select the grade of evidence (we have the slider in the results filter, which could be used/aopted).
• I’d like to create some specialist areas e.g. view all the latest cardiology articles.
• But I’d also like them to be able to search e.g. show latest articles on cholesterol
• But I’d also like them to start with ALL latest evidence and burrow.

I’d hate for it to be a simple list – so boring. I want it to be immersive, I want them to enjoy exploring the data. I’d like it to be visual. I quite like the idea of using a word cloud to help navigate. But is that good? Old fashioned?

I’m still at very early stages of my thoughts and am quite keen to have some input from a design perspective.


Do you fancy doing a little bit of work on this? I’m not expecting proper designs, perhaps some outlines? I’m not saying this should be design led but I think it could help guide my thoughts on how/if it develops.

I hope everyone had a good new year.

In 2010 we added a number of blogs to TRIP, for instance:

• Clinical correlations
• Trust the Evidence
• National Prescribing Centre blog

We’re fairly selective about which blogs to include and have identified a few more to add in the near future.  However, we’re hoping to hear from users of TRIP with suggestions of any blogs to include.  A few – loose – criteria:

• High quality
• Content is clinically focused and clinically usable
• Aimed at health professionals

Let me know if you can think of any.

Also, we’re always looking out for high-quality content.  So, any non-blog sites you find useful, that fit the criteria above, then feel free to let me know.

It’s a bit of a tradition to do some sort of review of the year.  This year I’m combining this with my love of the data visualisation site Wordle.  Wordle takes blocks of text and displays them in a manner that allows you to instantly see which words are used more frequently that others. The more often a word is mentioned the bigger the word is displayed.  I’m hoping this makes sense!

For the review of 2010, I’ve taken the titles of all the documents manually uploaded to TRIP during the year and given it to Wordle to display, the results are below.  The graph should expand when it’s clicked, but if not the full word cloud can be viewed here.

RIS is the file format for reference management software and we’ve been asked on a number of occasions if TRIP can support this format.  In other words, allow users to select search results in TRIP and export them – via RIS – into their reference management software package.  We’re delighted to say this has now been released and users can select it via the drop-down menu at the top of the search results (see image below – click to make bigger).  NOTE: TRIP does not have access to all meta-data so the export will be for partial results only.

Another bit of work we’ve been doing is looking to 2011 (see previous post).  I’m very keen to redesign the results page to make the results more useful.  One feature that we’ve spent a lot of time creating has been the background knowledge boxes (these appear at the top of many of the search results that link to eTextbooks linked to the search terms – see image above).  However, we’ve started to look at how popular this feature is and yesterday it was used 30 times.  This seems relatively little, so perhaps we need to consider if such a modestly useful feature deserves such prominence. 

It’s always the battle, balancing the demands on precious screen space between results and ‘added value’ features. 

Feel free to comment on any of the above topics…