Trip Database Blog

Interesting timing (given my last post)

Just-in-Time Information Improved Decision-Making in Primary Care: A Randomized Controlled Trial

Background
The “Just-in-time Information” (JIT) librarian consultation service was designed to provide rapid information to answer primary care clinical questions during patient hours. This study evaluated whether information provided by librarians to answer clinical questions positively impacted time, decision-making, cost savings and satisfaction.

Methods and Finding
A randomized controlled trial (RCT) was conducted between October 2005 and April 2006. A total of 1,889 questions were sent to the service by 88 participants. The object of the randomization was a clinical question. Each participant had clinical questions randomly allocated to both intervention (librarian information) and control (no librarian information) groups. Participants were trained to send clinical questions via a hand-held device. The impact of the information provided by the service (or not provided by the service), additional resources and time required for both groups was assessed using a survey sent 24 hours after a question was submitted. The average time for JIT librarians to respond to all questions was 13.68 minutes/question (95% CI, 13.38 to 13.98). The average time for participants to respond their control questions was 20.29 minutes/question (95% CI, 18.72 to 21.86). Using an impact assessment scale rating cognitive impact, participants rated 62.9% of information provided to intervention group questions as having a highly positive cognitive impact. They rated 14.8% of their own answers to control question as having a highly positive cognitive impact, 44.9% has having a negative cognitive impact, and 24.8% with no cognitive impact at all. In an exit survey measuring satisfaction, 86% (62/72 responses) of participants scored the service as having a positive impact on care and 72% (52/72) indicated that they would use the service frequently if it were continued.

Conclusions
In this study, providing timely information to clinical questions had a highly positive impact on decision-making and a high approval rating from participants. Using a librarian to respond to clinical questions may allow primary care professionals to have more time in their day, thus potentially increasing patient access to care. Such services may reduce costs through decreasing the need for referrals, further tests, and other courses of action.

NHS Evidence have published a briefing document. It’s pretty predictable and certainly doesn’t look radically different from what the NLH aims to achieve (click here to see the NLH’s aims/objectives).

Although I knew about it already, it was still difficult to see them explicitly stating that a dedicated Q&A service was out (see page 7).

I forwarded the following quote to Dr Gillian Leng (who I met early on in NHS Evidence’s existence to discuss a Q&A service):

“Despite attempts to evolve search into something more human friendly, there’s still a big hole there. As useful as Google is, it doesn’t answer questions very well…”

The quotes comes from the industrial strength TechCrunch (click here for the full article). My reason for sending the quote is my brand of logic, which goes like this:

• No-one doubts that clinicians generate a large number of clinical questions.
• Technology alone does not answer clinical questions.
• Therefore, you cannot rely on technology alone to answer clinical questions.

So, I’ll be fascinated to see what a busy clinician with a burning clinical question gets via NHS Evidence. A few ‘easy’ facts:

• NHS Evidence will concentrate on information from accredited sources with a focus on synthesised content (e.g. systematic reviews, meta-analyses).
• Our analysis of 327 dermatology questions showed that such ‘synthesised’ content answered 2% of the questions.
• Therefore, even if a clinician uses a half-way decent search they will still get to content that is unlikely to answer their question.

For the sake of the patients and the clinicians I really hope NHS Evidence has got something up it’s sleeve.

The BMJ PG have given a lot of thought into the re-branding of their ‘evidence’ output – the BMJ Evidence Centre. There is also news of a new ‘point of care’ tool, Best Practice, which will be available in the UK from early 2009. It appears that this is already available in North America via the name BMJ Point of Care. I know little about either, but I should be visiting the BMJ Evidence Centre in the near future and hope to learn more then.

The Evidence Centre was highlighted in an editorial in this week’s BMJ (click here; alas you need a subscription to read). Part of the same editorial continues with the ‘reaction’ to the overuse of the RCT, reporting:

“Michael Rawlins is right. In his Harveian Oration at the Royal College of Physicians last month he clearly signalled the medical profession’s overuse of randomised controlled trials to guide clinical decision making…
…Perhaps Michael Rawlins will be similarly influential, but in the opposite direction—by emphasising the dangers of relying too much on randomised controlled trials almost to the exclusion of other types of useful information. “

In my recent blog post – Experience as evidence – I highlighted some concerns around the prevailing view of ‘evidence’ and the above editorial shows that this questioning is gaining traction.

Another ‘angle’ is that the BMJ Evidence Centre is cleverly positioning itself to take advantage of the current situation to show itself to be radically different from a certain competitor who relies almost exclusively on the RCT – Cochrane. In which case should the ‘editorial’ be an advertorial?

… and it’s EvidenceUpdates. They have a new website (click here) but the service seems the same, which is a great thing as EvidenceUpdates is great! As their website states:

This service is unique: all citations (from over 150 premier clinical journals) are pre-rated for quality by research staff, then rated for clinical relevance and interest by at least 3 members of a worldwide panel of practicing physicians.

Click here for the free-access TOC. Lots of really interesting articles, honest

An interesting use of search data from Google, this time the search terms associated with flu recorded over time (click here for full post). The hypothesis is that people search for flu when they think they have it.

It would be interesting to overlay the search term graph with recorded incidence; I imagine that will follow.

The search results count for each category is now back.

The search results count was disabled due to excessive load on our servers. Now we’ve sorted out the servers, we have reinstated the counts.

A brilliant bit of news for Monday morning!

For interest: So many filters, so little time: the development of a search filter appraisal checklist

Historically, TRIP has used number of searches as a sign of usage/success. However, we have begun to question the merit of this for a number of reasons:

• The search statistics also included third-party sites that use TRIP via web-services, so users never actually come to the site (the 3rd party site sends a search string to TRIP and we return the results which the site integrates into their pages).
• The number of searches doesn’t take into account how people interact with the site. For instance a user searching on TRIP might look at 2 categories (e.g. EB synopses and guidelines) and that counts as 3 page views (first results page, EB synopses and guidelines). We feel this is much more meaningful.
• On a more pragmatic basis, our advertising income is based on page views not searches!

In reality this means that in September we had 740,697 page views while we were searched 1,138,699 times. The discrepancy can be explained in part due to 3rd-party access via our web-services, but also because our page views are based on Google Analytics, which can give a conservative view of actual page views (as it requires things like java to be activated to work!). Irrespective of potential discrepancies we still feel the page views are more meaningful.

So, goodbye to search numbers and hello to page views….