NHS Evidence was created to replace the NHS’s National Library for Health. I was disappointed when it was released as it appeared little more than a clone of TRIP. I was hoping for something more innovative to help support clinicians better and also to push the search agenda along – there is nothing like robust competition to stimulate innovation.
I have undertaken a number of Freedom of Information requests and have found out some interesting facts (click here to see the requests and responses).
What I found is as follows:
- The budget for NHS Evidence in 2010/11 is £24,438,000
- The number of searches this year is 15,811,716 (based on the average monthly figures for February to April 2010 multiplied by 12)
- This means a cost per search of 154p
By comparison, this is how TRIP stacks up:
- The budget for 2010/11 (for our search engine) – £45,000
- The number of searches this year is 8,058,648 (worked out using the same figures and method as above.
- This means a cost per search of 0.56p
Therefore, each search on NHS Evidence is 276 times more expensive than TRIP!
Why I raise this is not to rubbish NHS Evidence, although the figures are unflattering. It’s more a feeling that it highlights the limitations of search. Is this the law of diminishing returns? Is search really the answer to clinical uncertainty? I’m increasingly convinced that search is not the solution, irrespective of how much money you throw at the problem.
I’d love to have any comment on the above, are my conclusions sound? Do the figures add up?
Trip Database Blog 
August 4, 2010 at 8:16 pm
nice analysis!
LikeLike
August 5, 2010 at 9:13 am
Fascinating stats and intersting question 'Is search really the answer to clinical uncertainty?'
Reinhard
LikeLike
August 6, 2010 at 1:50 pm
How many of those hits are actually just Evidence based related – some of us still think they include hits against HDAS – the bibliographic databases – in their figures… and that could account for a lot of those hits a month.
LikeLike
August 16, 2010 at 8:39 am
But are you just using searches (i.e. from the single search box)as your denominator? Total ACCESSES to use the other material provided by NHS Evidence , e.g. specialist collections etc would reduce the cost per access?.
Rod
LikeLike
August 16, 2010 at 1:53 pm
Hi Rod,
Thanks for the comment. It's true that the analysis is crude. For instance it excludes the additional resources you mention as it also excludes the signifcant number of searches carried out on TRIP via our web-services.
Another point to consider is that the NHS Evidence funding represents a massive increase over the NLH and I'm not aware of any significant increase in information provision (e.g. specialist collections, extra content etc).
One could also mention the marketing budget (£730,000) which alone is 16 times larger than the total TRIP budget.
But to me the real issue is that TRIP, NHS Evidence etc still rely on serving up 10-20 search results for a given search. Most clinicians search using 1-2 terms which means the search results are fairly general. Who's to say who does it better TRIP or NHS Evidence. I think, due to our better coverage of content type, we answer more clinical questions for grassroots clinicians. I'm sure NHS Evidence would have a different perspective.
But this focus on trying to get the best 10-20 results for a crude search is the problem. No matter how much money you throw at the issue you'll still only be able to show 10-20 top results. With little knowledge of the searchers intention (other than the 1-2 search terms) you serve general results which MAY answer the searchers question.
To me, that's the main problem and I'm happy to expand significantly if you wish. email: jon.brassey@tripdatabase.com
LikeLike