NEJM Clinical Decisions - Results

I blogged about this a while ago in relation to our Socrates project. Well, the results are now out (click here) and make fascinating reading.

There were 3 treatment options (see article for more details) and the results were split fairly evenly (37.5%, 37.4% and 25.2%). I wonder what that indicates about the roll of evidence? How much does it influence practice?

TRIP in press

A recent review of TRIP in Comprehensive Ophthalmology Update (click here for PubMed entry) was very flattering. Some of the snippets:

"..it is clear that the number of sources included in the database is impressive in both quantity and quality"

"...this is really an exhaustive resource for virtually any clinical scenario you can imagine."

"The TRIP Database is a remarkable combination of simplicity and expansive depth. It is hard to imagine that a physician looking for evidence-based information on any entity likely to be encountered in clinical practice would ever need another resource other than the TRIP page."

Another recent article (Click here for PubMed entry) looked at TRIP's performance compared to a number of other high profile resources. It used an interesting methodology (perhaps strange would be a more accurate description). Basically, it used two main resources to answer clinical questions and then, if the answer was not found in these, the other resources (including TRIP) got used by the clinicians.

In short, TRIP answered more questions than any other single resource. This is made all the more impressive in that TRIP was only used when 'easier' questions had already been answered by the two 'main' resources.

Facebook

I'm not sure why but I've started using Facebook. I think my profile is available via this link http://www.facebook.com/profile.php?id=759565005. However, you need to login to Facebook to see some of the stuff and need to be 'my friend' to see all the stuff.

NHS Choices

The NHS Choices website has launched, it's a patient-focused site "that aims to put you at the centre of your healthcare." It looks nice and has some interesting areas that help users to find suitable health services. It also gives you an online health check. Apparently I have no risks for cardiovascular diseases. There is also as extensive list of patient information leaflets, which I may have to add to TRIP (after checking they are not copies of those on NHS Direct).

My main interest is in search, so that's what I've spent most time on. A search for prostate cancer was interesting. The 'hottest' part of the results page was a set of options showing me what my GP was reading (and similar options). Lower down you actually get to the results, which seem appropriate. It seems strange that, as a user, the main results are not displayed in the most obvious part of the page. However, given the prominence of the link, I imagine an awful lot of people will be interested in what their GP was reading. The 2 results are below

• Clinical Knowledge Summaries: Clinical topics full list Breast cancer — suspected … GI (lower) cancer — suspected … Prostate — benign hyperplasia
• Clinical Knowledge Summaries: Clinical knowledge DH referral advice for suspect cancer

As a patient I would find that list slightly alarming! So, he's looking at breast cancer, GI cancer, BPH - surely he should be looking at information relevant to my condition!

Apparently, my GP is reading the following in relation to my search of hypertension:

• Diabetes — hypertension
• Diabetes — hypertension … Hypertension … Hypertension in pregnancy
• Alcohol — problem drinking
• Alcohol — problem drinking

This site has obviously had large sums of money spent on it (I hear £3.5 million), it just seems a shame that it is let down by the poor implementation of search....

Update on the way forward

So far the overwhelming support (5 out of 6 responders) has been for option one – using a quality slider.

One respondent (Ben) came up with a wonderful idea that ties in very well with a historical feature of TRIP – colour coding. Around 5-7 years ago, when TRIP introduced results categorisation (‘evidence based’, ‘primary research’ ‘Q&A’ etc) we gave each category a colour. However, over time things got complicated and we decided to drop the colour coding.

Ben suggested we try giving the ‘quality slider’ a visual colour change. I’m guessing this could be multiple colours or different shades of the same colour. In the latter case it could be something like the darker the colour the higher the quality. How this might manifest itself in the results page I’m not sure, some immediate thoughts (so not necessarily well though out):

• Each result has a coloured box or bar representing the likely quality.
• The results themselves are written in a particular shade.
• Each result has the underlined hyperlink given the particular colour.

Something for the design guys to get their heads round – assuming we go with the quality slider!

The next steps for TRIP

We are currently planning our next upgrade to TRIP and we need your help! Let us know via the comments facility below, via the TRIP Database contact form (click here) or e-mailing me. The planned upgrade will go 'hand in hand' with a significant improvement in the search algorithm.


BACKGROUND: TRIP has a reputation for offering access to high-quality clinical literature. We wish to retain that reputation.

PROBLEM: Many clinical questions cannot be answered using high-quality material. Therefore, currently, users would need to go to other websites to find their answer. We are exploring the possibility of adding a broader range of material to support enhanced questions answering. However, simply adding lesser-quality material into the main search would ‘dilute’ the quality.

PROPOSAL: To add extra, lower-quality, material into TRIP in such a way as to ensure we retain our high-quality reputation while still enhancing our ability to facilitate clinical question answering. To that end I can see three main possibilities:

1) Auxiliary search: I'd envisaged an initial search to cover the higher quality material. If the user feels they need additional information they move a 'slider' (click on image below) to include a wider variety of content. This would sit at the top of the search results.



Pros: Allows users to easily include/exclude content.

Cons: Sliders are not a widely used ‘technology’ and users might not fully understand them.

2) Exclude from core search. This would be to include all the material in the search but only include the core 'quality' material in the main search (where all the results are 'mixed' together and returned on the first results page after a search). To select the lower quality material a user would need to click on the particular category section.

Pros: Cheap.

Cons: Users might not use the category search therefore never seeing the results.

3) Contingency. We could get the system to carry out a search as outlined in point 2) above. However, if there are less than, say, ten results which are highly textually relevant, the system adds the additional content to help boost the number of relevant results.

Pros: Should work very well, with little user intervention.

Cons: The inclusion of lesser quality material will not be explicit which might impact on our ‘high quality’ reputation.

Alternatively, if you can think of any other solution we'd be delighted to hear from you!

Algorithm working again!

Hurrah!

Things are pretty much back to normal with the search algorithm, so apologies for the poor service. For some reason PubMed has been having problems as well. A poor time for searchers.

This 'scare' with the algorithm has further emphasised the need to ensure the best search results possible. TRIP still has work to do on this and it is our priority. We're hoping to get started on a major overhaul of this by the end of the month.

Top ten search tips for TRIP

I was asked to produce a top ten search tips for a librarian (Beryl) in the Wirral (my birthplace) and was happy to help out. She suggested I share the list, so here goes:

1. TRIP is a free resource aimed at delivering the highest quality literature to support clinical practice. As such we focus on quality of content not quantity.
2. When searching using more than one search term tends to get the best results. A search on 'asthma' will yield a large number of results while a search on, say, 'asthma and steroids' brings back a more focussed set of results.
3. On the results page you can filter the results based on the type of material you're after e.g. guidelines, systematic reviews etc.
4. TRIP also carries out an automatic search of PubMed using the clinical queries interface. TRIP allows these results to be 'specialised' to only return results from core journals from numerous specialities e.g. cardiology, oncology, immunology.
5. The advanced search allows greater flexibility in searching. For instance, you can look for keywords that only appear in the title of a document. You can also combine multiple searches.
6. My-TRIP allows you to record keywords of interest. Then, when TRIP updates the content (monthly), any new material is e-mailed to the user. Registering with My-TRIP also allows users to comment on documents included in TRIP.
7. TRIP supports various different ways of enhancing searches. These include the ability to use brackets, phrase searching (by placing phrase in quotation marks), stemming (e.g. 'cancer*' searches for any term beginning with 'cancer' e.g. cancer, cancerous).
8. Where documents have conclusions TRIP has attempted to capture them. Where there are conclusions simply click on the blue 'i' button. When there is no conclusion this button is 'grayed out'.
9. The latest version of TRIP includes snippets (small auto-generated segments of the article on interest) these can significantly improve recall of appropriate documents. These can be turned off if they are too intrusive.
10. TRIP also searches two other collections of date medical images and patient information. These can be accessed via the links above the search box.

If anyone can think of any other then let me know! For additional search tips you can visit our more comprehensive list by clicking here.

Quality not quantity

Around 5 years ago TRIP only searched secondary review articles (systematic reviews, guidelines, CATs etc). This was fine, but our analysis found that this meant only around 20%. We decided then that TRIP should seek to answer clinical questions using the ‘highest’ quality research. In other words, if there was no secondary review then primary research or eTextbooks are better than nothing. Since then we’ve embraced this philosophy and we will continue to do so.

So what’s the problem?

In part the problem will be improved with our next planned improvement in the search algorithm. This will weed out much of the ‘noise’ within TRIP results.

However, you will still find poorer quality material (e.g. eTextbooks) mixed in with secondary reviews. I’m not actually sure that’s a problem but another approach might be to create a ‘quality slider’. What’s one of them? Well, it’s a facility that allows users to have a slider that allows them to choose what quality material they want to see. See the quality pyramid below:

A default could be set so that only the top 2 or 3 sections are searched. If an answer is found then fine. However, if not, the user can move the slider down to automatically include the lower down sections of ‘evidence’. This would make the selection of lower quality evidence explicit. An alternative would be to include everything in the main search (as happens now) and allow users to use the slider to exclude certain content.

One to ponder!

Teething troubles

I thought it was too good to be true - the new upgrade! It has come to our attention that the new upgrade has, somehow, messed up the algorithm. We're looking into it now and hopefully it'll get fixed ASAP.

It appears that this only affects the global search i.e. the initial search where all the results are combined. Clicking on a specific category (e.g. systematic reviews) brings the results more 'into focus'.

Apologies....

How much does the internet weigh?

Apparently around two ounces (60 grams) or significantly lighter if you use an alternative method. Click here for the full article.

The new TRIP

Around 11.30 today the new version of TRIP went live. But what's so different? Lots of new features most designed to help users locate the information they need quickly; these include:

• Snippets. These are the 2-3 lines of summary text from the returned document. These have been demonstrated to significantly improve the recall of appropriate search results.
• Conclusions. Where documents have conclusions we have designed our system to 'grab them' and allow them to be displayed within TRIP. This has numerous advantages including a significant reduction in bandwidth, very useful if you don't have the 'luxury' of broadband access.
• Advanced search. This has been redesigned to allow a significant improvement in usability in this function.
• Comments. We love the BMJ's 'rapid responses' so why not allow TRIP users to share their views with the wider TRIP community?
• RSS feeds. This feature allows you to 'save' a search as an RSS feed. If new material is added that appears in the top 50 search results for the same search, its outputted via RSS.
• Memory. We've doubled the capacity of our servers memory, this should bring about a rapid increase in speed.
• Redesign. A mixed blessing this one! Basically, we need to increase our income vai Google ads. As it stands not enough people are clicking on the ads so our income from these is not sufficient to pay for costs and fuel our significant development plans. By making the ads more prominent, the theory is, that more people will click on them....

I'm pretty chuffed, I hope you are too!

In around 12 hours....

...the new version of TRIP will start to be deployed. The whole deployment process will take approximately 2 hours. So by lunchtime (British Summer Time) it should be live.

PMID.US

I was sure I'd blogged this, but I can't find it - so here goes!

PubMed URLs tend to be really long. The typical URL can be http://www.ncbi.nlm.nih.gov/sites/entrez?cmd=retrieve&db=pubmed&dopt=AbstractPlus&list_uids=17540531

Using this site http://pmid.us allows you to simply add the PMID to the end of the URL e.g. http://pmid.us/17540531

If you click on the last URL you'll get taken to an article that features TRIP. In this Belgium paper clinicians searched 4 databases (one of which was TRIP) and the authors found very favourable results. I've ordered the full-text and will report more when it arrives.

Two other mentions of TRIP recently:

• http://pmid.us/17540129
• http://pmid.us/17512105