Trip Database Blog

I’ve not mentioned NHS Evidence much and it’s a coincidence that I mention them twice in 6 weeks. I last blogged about them then (click here) to highlight that their searches are massively more expensive than TRIP’s.

My interest in NHS Evidence goes back longer than 6 weeks! I have contacts (and had contracts) with the predecessor to NHS Evidence (National Library for Health) and was keen to see how NHS Evidence would change things. Unfortunately, I was disappointed by their lack of ambition and innovation. Although, I’m still hoping to be proved wrong and they’ll do something special – clinicians (and patients) deserve it (especially with their budget of nearly £25 million).

But, the reason for this post, relates to some advertising NHS Evidence produced earlier this year. I saw it in the middle of March in the Health Service Journal. In that advert they claimed two things which I felt were inappropriate:

• The most dependable clinical and non-clinical information online
• answer all your clinical and non-clinical questions

It makes me shudder that this got past any internal processes. As a gateway for evidence surely they should be able to substantiate any claims they make. Both claims, to me, were absurd. I could go on, at length, as to why the claims are so foolish and if people are unclear then please contact me separately (jon.brassey@tripdatabase.com) and I’ll explain. 

After seeing these adverts I felt quite strongly that NHS Evidence shouldn’t make such statements and therefore complained to the Advertising Standards Authority. I recently received a letter telling me that NHS Evidence has confirmed that the advert has been withdrawn and that they will not repeat the claims.

It’s a small victory.

But here’s a challenge to NHS Evidence – fund some comparative research and compare yourself to other products. Not just TRIP, but DynaMed, NHS Scotland’s rather nice Knowledge Network and others. Help everyone learn from the strengths and weaknesses of others. If it helps we’ll part-fund it (we’ll give 0.5% of our budget if you match that with 0.5% of your budget).

As it happens in another freedom of information request NHS Evidence (where I asked about any comparisons they make with other search engines) they reported:

“we don’t compare our performance against other services”

This last quote is quite simply scandalous.

I sometimes feel there is a pressure to blog, to write something, just to show people we’re still here and thinking of them!  While things may appear quiet (we’ve not blogged, twittered or Facebooked much) we’re still very busy.  However, much of the work is behind the scenes as we gear up the development of our TILT project.  I’ve been busy finalising the specifications and have as good as signed off on these.  TILT is pretty much in stealth mode so what can I say….

• It’s a way to record and share learning
• While search focuses on known unknowns (you need to know you don’t know something before searching for it) TILT will focus as much on unknown unknowns as known unknowns!
• When I talk about search, in TILT, I’m not referring to document search – something far more useful and concise.
• At it’s core is clinical usefulness, it’s a bottom-up approach.

Well, I hope that’s whetted people’s appetites.

It’ll launch, probably as a beta towards the end of October.

Changing tack, if you’re on Facebook please visit our page (TRIP on Facebook) and ‘Like’ us 🙂

Another month and another manual upload of 600 plus records.  TRIP actually imports more records than that, the majority come via automatic methods.

In addition to the monthly upload we’ve completely revamped (due to URL changes) the following sites:

• Diabetes UK
• British Thoracic Society
• British Association of Dermatology
• British Society for Rheumatology
• European Association of Urology
• Washington University Division of Emergency Medicine Journal Club

A busy month, but well worth the effort.

I’m very pleased to announce that TRIP now has over 10,000 registered users. 

The vast majority of these users are health professionals enrol and there’s pretty much an even split between doctors, information specialists and other health professionals.  The users are from all around the globe, highlighting TRIP’s global appeal and reach.

To celebrate we’ll be rolling out a significantly enhanced e-mail alert system.  Previously, if a user had recorded, say, ‘prostate cancer’ as an area of interest they would simply get an e-mail reporting how many new articles matched this keyword.  Following feedback we have improved this so the results are broken down into EB Synopses, Systematic Reviews, Guidelines (pertinent to your country), Core Primary Research and Extended Primary Research (see below) 

The number of articles suitable for the developing world (click here for further information) continues to grow and appears to be speeding up.  As of today we have identified 288 articles.  It’s not a massive number, but a review of these reveals a high level of accuracy – so I’m happy that things are moving in the right direction.

We never envisaged this initiative being ‘the’ answer, but part of something bigger.  In relation to that we have a number of ongoing discussions and if/when these come to fruition we’ll announce them here.

NHS Evidence was created to replace the NHS’s National Library for Health.  I was disappointed when it was released as it appeared little more than a clone of TRIP.  I was hoping for something more innovative to help support clinicians better and also to push the search agenda along – there is nothing like robust competition to stimulate innovation.

I have undertaken a number of Freedom of Information requests and have found out some interesting facts (click here to see the requests and responses).

What I found is as follows:

• The budget for NHS Evidence in 2010/11 is £24,438,000
• The number of searches this year is 15,811,716 (based on the average monthly figures for February to April 2010 multiplied by 12)
• This means a cost per search of 154p

By comparison, this is how TRIP stacks up:

• The budget for 2010/11 (for our search engine) – £45,000
• The number of searches this year is 8,058,648 (worked out using the same figures and method as above.
• This means a cost per search of 0.56p

Therefore, each search on NHS Evidence is 276 times more expensive than TRIP!

Why I raise this is not to rubbish NHS Evidence, although the figures are unflattering.  It’s more a feeling that it highlights the limitations of search.  Is this the law of diminishing returns?  Is search really the answer to clinical uncertainty?  I’m increasingly convinced that search is not the solution, irrespective of how much money you throw at the problem.

I’d love to have any comment on the above, are my conclusions sound?  Do the figures add up?

Answering clinical questions is at the heart of the TRIP Database.  We’ve been answering clinical questions for over 10 years and answered well over 10,000.  Most of these questions have been answered using a standard methodology:

• Receive and clarify the question
• Conduct a search
• Read and appraise the articles
• Write and answer
• Post answer on the website

This can frequently be time-consuming and therefore relatively expensive.  Search systems, even the TRIP Database, doesn’t answer questions – it returns articles that may help the clinician answer their question.

There are alternate models of answering questions and from the research evidence the most consistent source of answers for clinicians is their own colleagues.  But, there are half-way houses and my current favourite is facilitated by a service called StackOverflow which is a Q&A service for those involved in IT.  The company behind StackOverflow have recently been increasing the availability of the underlying software to power other Q&A systems and this was featured, today, on the ReadWriteWeb blog. 

One of the first non-IT uses for StackOverflow that I was aware of was GasExchange, a Q&A service for anaesthesia (check it out, it’s great).  It’s been running for many months and has an active community.  I’d love to see this sort of thing happen for other specialities – including primary care. 

I spend a lot of time pondering how best to serve up the results on TRIP.  Currently all results are merged with the ability to filter the results by category (systematic reviews, guidelines etc).

But, there are clearly different types of content:

• Research evidence – systematic reviews, synopses, guidelines, primary research etc.
• Background knowledge – eTextbooks.
• News
• Patient information

The mock-up below (click to enlarge) shows a way of making this distinction clearer.  Let me know what you think!

TRIP has become increasingly interested in improving access to the best available evidence for the developing world (see our crowdsourcing initiative).  In launching the initiative I’ve managed to meet a number of interesting people. 

Yesterday I had the pleasure of meeting the editor of the Africa-Health journal who’s knowledge and experience of the issues facing health professionals in the developing world was impressive.  We talked about a number of potential new projects but as ever funding is an issue.

I recommend readers of this blog, especially those in the so-called ‘developed world’ to spend some time reading the journal – it highlights the issues facing health professionals in these areas and gives a sobering perspective.