Liberating the literature

Answer engine

2012-01-15T10:58:00.001Z

Those of you who know me or have got a feel for TRIP (via this blog or using the site) will know that our biggest motivation is allowing clinicians to get rapid answers to their clinical questions. The TRIP Database has been running for nearly 15 years and we've helped shape clinical search, spawning many similar search tools (perhaps the most recent being the eye-wateringly expensive (and soon to be rebranded) NHS Evidence).
However, over the last few years my reservations about clinical search has grown. To define that further, I mean clinical search for busy clinicians (as opposed to librarians, information specialists, academics etc). Those that need a rapid answer to their clinical questions.

The current search paradigm is that users add 1-3 search terms, press 'search' and sift through the top 10-20 results to find the answer they're looking for. Below are a few problems with this (there are many more):

We know that clinicians are typically poor searchers, so there's a handicap from the start.
The way clinician's select which papers/documents to look at is problematic based on a number of factors.
From our experience of Q&A (answering over 10,000 clinical questions) we know the average number of references to answer a question is over two. So, that suggests that clinicians will need to open 2+ documents and read them to find the information they need - highly inefficient, especially if the document is long.

Using the example of NHS Evidence, they have a vast resource (£700,000+ for marketing alone. Their marketing budget is nearly 30 times higher than the entire TRIP Database budget!), a good brand, a competent implementation, part of the NHS 'family' yet have embarrassingly low search stats. I believe one of the main reasons is that search is not something that works for most clinicians. It's a paradigm, defined by Google and people seem happy to settle for it. Is it any wonder that clinician's main source of answers to their questions is to ask a colleague? One obvious reason that clinicians ask other clinicians (but there are clearly others) is that they get an answer - not 10-20 links that may answer their question.

So, to me, any solution is to go back to first principles - in this case a clinician with a clinical question. What do they want? An answer. I'm hoping that's not controversial - it seems obvious to me.

To reiterate, imagine you're a busy clinician and have a clinical question, what would you prefer:

A robust answer.
A list of 10-20 results, any number of which may contain all or part of your answer.

Is it only me that sees this as a 'no brainer'?

If you're one of the people that thinks the latter - please contact me (jon.brassey@tripdatabase.com) as I'd love to understand your perspective better

Anyway, moving on to the notion of delivering an answer - this is where it gets complicated but also interesting. I firmly believe that we shouldn't shy away from a challenge, not one that's so important as this. With the experience gained in TRIP (with search and Q&A) I actually think that the issue is manageable.

Basically, I'm planning on building a system that will take a clinical question and deliver an answer.

However, one thing for sure, it's too big for TRIP to do it on it's own. So, we've started by getting together a small group of people representing organisations who have a vested interest in getting this right. Those who share in the vision. To me, the biggest challenge will be managing the disparate bodies - from the small to the very large.

If I can pull this off, we've got every chance of making an industry-wide change for the best.

2011

2011-12-21T19:26:00.000Z

Do I need to apologise for there being such a gap (one month) between my last post and this? I've no idea!
Basically, we've been working very hard on two main areas of work:

The next upgrade of TRIP. This should appear April/May 2012 and has taken lots of work with the surveys, interviews and reading around the topic.
An answer engine. For commercial reasons I have to be vague on this for now (something I'm not comfortable with) but it'll take shape over 2012. I've identified a number of partners to get this collaboration going and our first meeting should be in February next year. While my enthusiasm is driving this (initially at least) it won't be a TRIP product, we will be one of the partners. I've come to realise that TRIP can only achieve so much on our own. But the initial partners are very strong and should help me realise my dreams.

All the above aside, I'm starting to wind down for Christmas. It's been a long year and I need it.

If you celebrate Christmas - enjoy.

Best wishes

jon

Associated results

2011-11-22T08:55:00.000Z

TRIP typically gets an overhaul once a year and ahead of changes for early 2012 we've started a series of surveys asking people what they like and don't like about TRIP. I've already posted three posts on the first round of results (see here, here and here). This allowed me to draw up a list of proposed changes which I've discussed with the TRIP techie - Phil. As a result of this discussion I started a second round of questions, to probe some of the issues raised in the first round of questions.

One area of real interest is the 'Associated results' on the right-hand side of the TRIP results page (image below).

These are resources that could be useful to a user, but are not a core part of the TRIP index. These are problematic for a few reasons:

They increase the time it takes for the page to load.
It's not scalable, in that we could put many more resources there - but it wouldn't work.
It looks messy.

So, one question I asked in the 2nd survey was "One area we're looking at are the results on the right-hand side that link to clinical trials, PubMed, BNF etc. We're thinking they may be distracting, after all you're coming to TRIP for the great set of main results. So, which of these sounds best?"

I think I was hoping that most would indicate that they don't use them and we could, perhaps, relegate them to the bottom of the results page. But, and reinforcing why it's useful to ask your users, here are the results so far:

Leave them alone, they're great and I use them all the time - 15.8%
Leave them alone, they're useful and I use them from time to time - 52.6%
Remove them completely I rarely/never use them - 2.6%
Make them less prominent e.g. at the bottom of the results so they can be used if the answer can't be found in TRIP - 28.9%

So, nearly 70% of users want them left alone! This creates a dilemma as to how to increase the number of associated results yet still use no more space. I have some ideas (e.g. use a table or buttons that shows results only when clicked) but one to be worked through with our designers.

Irrespective of the outcome - I'm really glad I asked!

Locating new content on TRIP

2011-10-28T09:34:00.000Z

TRIP aggregates thousands of new articles per month. The majority are from primary research but we also include 500-750 new articles from secondary sources e.g. systematic reviews, guidelines.

We appreciate that this is important information and have two mechanisms for helping users identify new content.

The first is the ability to carry out a search and only select content added in the last month. The image below highlights where this feature is.

The second has been our efforts to categorise each new piece of content by various clinical categories. This has been acheived in three main ways:

By publication type. For instance we assume each article in the journal Cancer is appropriate for our oncology updates.
By keyword matching. This is slightly trickier! We have over 25 clinical categories in TRIP (e.g. cardiology, oncology) and we started by creating a long list of keywords/terms associated with that clinical area. For instance, the following words are some example terms associated with cardiology: atrial fibrillation, cholesterol, hypertension. In total we have identified 334 cardiology terms (we're not claiming it's exhaustive). What we have done is cross check each document title in TRIP (NOTE: title words only) with all the category words to assign clinical categories to each document. A document can have multiple categories. For instance a document called 'Prostate cancer screening in the elderly' would be assigned to urology, oncology and geriatrics. This is not foolproof and only today I have spotted a problem. We have the word sinus associated with Otolaryngology-ENT which unfortunately returns documents with sinus rhythm in the title! However, the errors are relatively few and far between and we hope the benefits far outweigh the negatives!
Manual identification. In the case of primary care we cannot assign keywords, so this content is manually identified and typically restricted to the secondary evidence base.

You can view the new content via the links below. Note: these are updated monthly, after we carry out the manual upload of new content.

As mentioned above our methods aren't foolproof and we welcome feedback. Let me know via jon.brassey@tripdatabase.com

Making search slightly smarter

2011-10-24T10:20:00.001Z

I'm really pleased to announce a new feature to TRIP, one we've been working on for a while now. It also supports one of the major requests from the recent survey - help us refine our search (which was the 3rd most requested improvement). This new feature is not our only answer to search refinement but it's a great start.

The new feature allows users to refine search by clinical area. So, you can do a search for - say - measles and then restrict these results to neurology documents. The results will then be restricted to those articles that are related to both measles and the broad area of neurology.

How does this affect the results? The first set of 3 results (below) are when you do a search for measles while the second set of results are the top 3 results when searching for measles and restricting to neurology.

Measles (top 3 results)

Antibiotics for preventing complications in children with measles (Cochrane)
Vitamin A for treating measles in children (Cochrane)
Routine vitamin A supplementation for the prevention of blindness due to measles infection in children (Cochrane)

Measles restricted to Neurology (top 3 results)

Measles-mumps-rubella-varicella combination vaccine and the risk of febrile seizures (Pediatrics)
Measles virus-specific plasma cells are prominent in subacute sclerosing panencephalitis CSF (Neurology)
MMR vaccination and febrile seizures: evaluation of susceptible subgroups and long-term prognosis (JAMA)

But the best way to see the power is to simply give it a go and try it out. I really am pleased with it.

To help you orientate yourself, the picture below shows you were to find it!

How does it work?

Another significant piece of feedback from the survey was one of transparency - how does TRIP work. So, this seems a good start at being more transparent.

We have over 25 clinical categories in TRIP (e.g. cardiology, oncology) and we started by creating a long list of keywords/terms associated with that clinical area. For instance, the following words are some example terms associated with cardiology: atrial fibrillation, cholesterol, hypertension. In total we have identified 334 cardiology terms (we're not claiming it's exhaustive). What we have done is cross check each document title in TRIP (NOTE: title words only) with all the category words to assign clinical categories to each document. A document can have multiple categories. For instance a document called 'Prostate cancer screening in the elderly' would be assigned to urology, oncology and geriatrics.

When someone does a search on TRIP we examine all the returned documents and build a category filter based on the assigned clinical categories (most popular at the top). By clicking on a single category the documents are restricted to that category.

NOTE: the category 'Primary care' is an exception as we cannot find suitable keywords to adequately identify suitable documents. These are therefore assigned manually each month. This started relatively recently so the cohort of suitable documents is small at present.

Finally, given the nature of the methods used we do not claim that it's foolproof. The system will invariably miss some documents and include some documents that it shouldn't. Those minor irritants aside it's still really powerful - enjoy.

I hope this all makes sense, if not get in touch via jon.brassey@tripdatabase.com

An answer engine

2011-10-17T08:48:00.000Z

I gave a talk at the recent Public Health Wales staff conference (I work part-time for them) about my experience of answering clinical questions, the usefulness of evidence and the problem with search. I highlighted that clinicians want answers to their questions, not links to ten articles that might contain their answer (or part of it) - which is what search does.

As you may well know I'm passionate about answering clinician's questions and I view search as - at best - a partial solution. As TechCrunch reported in 2008:

"Despite attempts to evolve search into something more human friendly, there’s still a big hole there. As useful as Google is, it doesn’t answer questions very well"

I also heard an interview by Steve Wozniak (who co-founded Apple with Steve Jobs) and he too was talking about the limitations of search and how we need to create an answer engine.

What would be needed to create an answer engine for clinicians? TRIP Answers has a collection of over 6,000 Q&As but what would it take to make that 60,000 large?

I am very tempted to try and create an open repository of clinical Q&As and allow users (probably after a vetting process) to upload their own Q&As. Why not cut up guidelines into their constituent recommendations? Why not cut-up individual chapters in - say the Green Book - into separate Q&s?

To do this requires volunteers, co-explorers, people believing that clinician's want answers to their questions. Anyone else fancy transforming the clinical information space?

Survey: final section of analysis

2011-10-07T16:38:00.001Z

The final results from the analysis are below.

Content

Perfect - 59%
Not enough content - 16%
Too much content - 25%

How the site looks

I think it looks awful - 4%
Ok - 49%
Not bothered - 26%
Great - 21%

If you were required to login to TRIP to use it, how would you feel?

I'd stop using the site - 6%
I'm happy to do it, but make it easy - 53%
I'm happy to do so, as long as I get a better service - 25%
I'm not too keen on this idea - 16%

It's interesting to note how similar the results are compared with our early analysis (based on 345 respondents), the above is based on 518 respondents.
So, the results are all out. It now just requires a fair amount of reflection, which I'll post on the blog at a later date. I'll then plan round two of the survey - which will be trying to really focus on the issues and how we can best meet them.

Survey: more results

2011-10-07T12:25:00.001Z

The survey closed last week and with 518 respondents it's taking time to analyse them. I'm now ready to post the results from the two questions 'What's poor/weak about TRIP?' and 'What would make you use TRIP more?'. In our early analysis of 345 responses we combined these as there was significant overlap, and we're doing so again. As with the earlier analysis I'll show results for 3 categories of users (doctors, information specialists, others).

Doctors

Full text - making it easier to link to full text.
Weird results - sometimes finding strange articles in the results.
Design - main concern was clutter!
Emails - improve the design and increase the frequency.
Refine search - make it easier to refine the search results.

Information Specialists

Advanced search - improve the advanced search options
Transparency - how do we define the category filters, what sources are searched by TRIP, what synonyms do we use.
Full text - as above.
Refine search - as above.
Design - as above.

Other

Emails - as above.
Refine search - as above.
Full text - as above.
Transparency - as above.
Other database - link to a greater number of external databases.

Global

The following are results taken from all respondents, irrespective of background. This time I'm displaying the top ten.

Full text
Transparency
Refine search
Advanced search
Emails
Weird results
Design
Save search - allow users to save their searches and send updates for new hits. NOTE: this already exists but it's clearly not obvious.
Mobile friendly - possible create an app or a more dedicated mobile friendly site
Evidence slider - reinstate the evidence slider.

I'm really pleased with the results. There are loads of things for us to consider but none of them are particularly daunting. I'll reflect on these over the next few months as well as publishing the remaing results from the survey.

Thank you for taking part.

Refining search

2011-10-04T08:09:00.003Z

I'm about 40% of the way through the analysis of the TRIP survey and one thing is clear, a big issue for our users is the ability to easily refine the results. Users, after an initial search, sometimes find too many results and want to focus the results to a more manageable set of records to examine.

One idea (which I really like) was for the ability to further restrict results based on clinical cateogry. In other words a person selects 'Systematic Reviews', allow them to restrict on a publication basis e.g. only show results from Cochrane, DARE etc.

Seems reasonable to me!

Survey ended, start of analysis

2011-10-01T16:27:00.003Z

In total we had 518 results, which was a brilliant result. Early results below (it'll take me a while to analyse all the results.

Professional background

Doctor – 45.4%
Information specialist – 19.9%
Other - 9.5%
Other health professional – 8.9%
Nurse – 8.3%
Pharmacist – 7.5%
Patient/carer – 0.6%

What do you use TRIP for?

Answering clinical questions – 67.0%
Research – 57.7%
Keeping up to date – 49.8%
Other – 9.8%

Teaching, training and education were included in the 'Other' section

How do you use TRIP?

Pull (search) – 58.9%
Push (e.g. rss, email) – 7.9%
Both – 33.2%

What aspect of TRIP do you find works best for you?

The ease of differentiating between evidence types
Ease of use
Wide evidence coverage
Guideline coverage
Monthly emails
Colour coding
Linking to other databases e.g. Medline, clinical trials
PICO/Search wizard
Free
Speed

Further results will be posted when I've analysed them.

Broad versus specific information needs

2011-09-24T09:16:00.002Z

One thing I'm wrestling with is the different types of information needs of clinicians. I'm interested - at the moment - in two broad distinctions:

Broad - user typically wants background information e.g. eTextbook
Specific - user wants to answer a specific question e.g. what is the best antidepressant in pregnancy?

TRIP is set-up to answer specific questions. While we have eTextbooks they appear lower down in the search results (as they are typically of lower quality) so, for a user to consistently see them, needs to click the eTextbook filter.

But can TRIP do more for Broad information needs? Are clinicians conscious of the differences; do they approach TRIP thinking 'I need background information?'.

Perhaps we need a prominent 'show background information' button? Seems slightly clumsy to be, but plenty of time to ponder!

Survey, more meaningful results

2011-09-22T11:29:00.001Z

Earlier today we posted some preliminary results of our survey (click here). I've now had a chance to look at the main textual results and - while viewing them - I spotted the overlap to responses to the questions 'What is bad about TRIP?' and 'What features would make you use TRIP more?'.

The response has been great and has been really useful in helping to focus our minds. Below is a brief overview of the main results. I've given a global result plus broken down into 3 categories based on profession (apologies for lumping in 'Other health professionals'):

Doctors (top 3 improvements)

Access to full-text articles
Improve the advanced search
Smart phone app

Information specialists

Improve the advanced search
Clarity of the results (how the results are arrived at)
Poor, unrelated articles appearing in the results

Other health professionals

Refine search - make it easier
Improve the monthly email
Improve the Boolean search

Global results for all respondants (top 7)

Refine search - make it easier
Improve the advanced search
Access to full-text articles
Poor, unrelated articles appearing in the results
Improve the monthly email
Improve the Boolean search
Clarity of the results (how the results are arrived at)

Overall, lots of good ideas. The one worry for us is the frequent reporting of 'wrong' articles appearing in the search results. Something for us to deal with well before the next upgrade.

As ever, one big thing that was apparent from the users - the love for TRIP :-)

Survey time - early results

2011-09-22T08:41:00.000Z

We posted our latest survey yesterday afternoon and so far we've had an amazing 345 responses (if you've not done it, it's not too late https://www.surveymonkey.com/s/WMS38PY).

I'm really keen to get our users input into the next upgrade to TRIP (scheduled for early 2012) and this is the first stage survey. We asked people to leave their emails if they want to take part in further surveys and we've already got 200+. The first survey asks mainly high-levels questions (e.g. what's good about TRIP, what's bad etc). The second survey will ask more specific questions about functionality, design etc. I really can't wait. But before the second survey I have to finish reading a few books and papers on search and to fully analyses the first round of results.

A lot of the questions will require some form of textual analyses but other lend themselves to a simple numerical analysis, so some early results below:

Background (top 3)

Doctors - 44.1%
Information specialists - 18.8%
Other health professionals - 10.7%

What do you use TRIP for?

Answering clinical questions - 69.6%
Keeping up to date - 50.7%
Research - 56.5%
Other - 10.1%

How do you use TRIP?

A pull way (searching the site) - 55.9%
A push way (monthly emails, RSS etc) - 8.4%
Both - 35.7%

One issue we're looking at is the content offering of TRIP, what best describes it for you?

Perfect - 59.1%
Way too much - I get too many results - 24.3%
Way too few - I often get few and/or poor results - 16.5%

Another area of interest is the design of TRIP (how it looks). We have recently rolled out a new design and we're not convinced it's great. What do you think?

The site looks great - 22.9%
I'm not bothered how the site looks - 26.1%
I think it looks ok - 47.5%
I think it looks awful - 3.5%

If you were required to login to TRIP to use it, how would you feel?

I'd stop using the site - 5.5%
I'm happy to do it, but make it easy - 53.0%
I'm happy to do so, as long as I get a better service - 24.9%
I'm not too keen on this idea - 16.5%

So, lots of interesting results. But the biggest 'steer' will be the textual analysis of the 3 questions:

What's good about TRIP?
What's bad about TRIP?
What feature would make you use TRIP more?

Adaptive search

2011-09-16T05:37:00.001Z

Bing, the world's 2nd best used general search engine (after Google) has just announced the release of adaptive search. A brief overview can be seen via this searchengineland blog article.

Adaptive search excites me enormously and it's something we're working on at the moment. I say working on, we're working on the theory and how it might work. We need to save up our pennies and test a few assumptions before building it!

The principle is that adaptive search learns the type of person you are and alters the results based on what it has learnt about you. This means different people would see different results. For instance, a general practitioner in the UK might search for hypertension and s/he would see different results to a cardiologist based in Canada. It makes sense that they see different results as their contexts are different.

We're actually pretty confident with the theory but the biggest assumption is, for this to work, people need to login. So, we're exploring that and allowing people to login with their Facebook and Twitter accounts should make this easier.

An evidence-based TRIP?

2011-09-12T07:12:00.000Z

This may seem a strange title as many of you will feel that TRIP is an evidence-based tool already. Well, TRIP helps users find the best available evidence, I'm comfortable with that. However, there is more to this than that....

Since the new interface/functionality went live I've been tidying up loose ends, fixing a few broken bits of functionality and looking forward to the next changes to TRIP. This has coincided with a wonderful opportunity to get involved with a research grant that will explore creating a search interface based on academic theories around information gathering. This will involve creating a series of interfaces which will then be tested with a group of clinicians. So, we'll be doing research to explore various aspects of search interface design and results display. We'll be creating an evidence-base for clinical search.

I've no idea if we'll get funding but the person in charge is hugely influential so I think we've got a reasonable chance. Even if it doesn't happen I've already learnt an awful lot from just reading the background papers.

Exciting times!

Broken links

2011-08-28T11:28:00.000Z

The latest upgrade to TRIP had lots of new features easily visible to users. However, for all the external changes we have made a number of internal ones. One of the unhidden wonders has been a broken link reporter! Basically, if someone clicks on a link and it doesn't result in a valid webpage being open we now get an email telling us the link isn't working. Previously, we had no system for this and it was down to the goodwill of our users to let us know.

When the new feature went live it was truly depressing as I was personally receiving about an email every minute. However, it soon showed up that a few particular publications were causing the bulk of the issues. Within 4 days these links were fixed - causing a 90% reduction in the error messages. Now, 3 weeks later and lots of hard work, we've fixed approximately 99% of all the broken links.

For most people they will not notice a difference - but it's great to know our system's working well and long may it continue.

New version of TRIP released

2011-08-03T13:47:00.000Z

I'm very pleased to announce that we have launched the latest version of the TRIP Database a few hours ago. New features include:

Translate function. Users can translate the site and subsequent links into one of six languages (Spanish, Portuguese, Italian, French, German and Welsh).
We've added the ability to restrict results to only those added in the last month.
We've started searching one of the largest clinical trials databases.
Started extracting and displaying DOIs against each result (if they exist).
Included a separate section for clinical calculators.
Included a separate section for social media results.
The speed of the site has been improved.
New design.
Started to include patient decision aids and medical education content.
Massively increased our medical video offering, now searching over 6,100 videos.

Feel free to let us know what you think!

App for new content on TRIP

2011-08-02T11:08:00.000Z

We're building a TRIP app. Initially it'll be for the Apple iPhone and if that proves successful we'll create a version for Android. It'll actually be multiple apps, but all following the same format - highlighting the latest content added to TRIP for a given area.

We see there being two main types of app:

Clinical area - all the new content in areas such as cardiology, oncology etc. They'll be around 25 of these.
Individual clinical conditions - while we could produce unlimited, we'll start with 20-30 of the biggest e.g. diabetes, asthma, hypertension.

Each month users will be alerted when the new content is available and will be able to select to see content based on our filtering system (e.g. systematic reviews, guidelines) and then simply scroll through the results. Below is a screenshot from our prototype - and it's great!

We'll have to charge for this, but it'll be suitably cheap!

Link resolving

2011-07-26T10:32:00.000Z

For a while now we've been exploring linking to full-text articles (currently we only link to abstracts of primary research articles). It's an area I have little technical experience in and have allowed myself to avoid the issue. In the latest version of TRIP (out next week) we're extracting the DOI for journal articles (and Cochrane), but this is only part of the issue.

To link to a full-text requires an additional step - it requires a link resolver! Trying to keep things simple, it tells the system where to point the full-text request to (ie send the person to Wiley, Ebsco etc). We're looking to introduce this and I'm fairly confident we can do it! It should work like this:

We extract the DOI for an article - which we've done already!
An institution tells TRIP which full text holdings it has (e.g. NEJM, Thorax etc) and also some details of their link resolver.
A user comes to TRIP from a given institution and carries out a search and we display a full-text link to all articles the user has full-text access to. They click on the link and they've got access to the full-text (based on an authentication system).

So, to my mind, the hard work lies with the librarian who needs to tell us what full-text holdings they have! The rest seems relatively straightforward - I must be missing something.

But, bottom line, does easy linking out to full-text excite people?

The future of TRIP

2011-07-05T07:00:00.000Z

For those of you who follow TRIP you'll know that I frequently reveal our lack of money. It's something I've got used to and learnt to live with. I like to think of myself as an innovator and product developer, as opposed to a businessman. I'm not great at the latter, I don't like asking for money! However, we've survived for over ten years and are still financially viable (I was going to say strong, but that'd be a slight exaggeration)! What we've achieved with a small budget has been spectacular and part of me thinks that being kept hungry keeps the pressure on me to innovate.

But, I do occasionally fret over the lack of business development. I do wonder what we could do with a decent budget. I do have a realistic set of ideas/innovations that would make TRIP significantly better. I do think I could make TRIP the Google for medicine - the first port of call when clinicians have knowledge needs.

So, when I was approached 4-6 weeks ago by someone interested in helping TRIP on the business side, I was excited (and apprehensive). We had some great chats and interesting exchanges of emails. I'm thinking the approach may well have stalled but I'm not dispondent and that's for the following reasons:

I learnt an awful lot during the due diligence process. I received numerous pieces of advice - all free - around various aspects of business, including corporate structure. I've come out of this stronger and more confident than before.
As a result of one thread of advice I've been introduced to a venture capital firm. This relationship isn't necessarily about raising finance, it's more about exploring options and business models.

With the new version of TRIP due before the end of the month and all this excitement around business development I can't help but be excited about the future. I doubt I'll ever be a businessman but as long as I keep enjoying my business that's fine by me :-)

TILT and learning points

2011-07-01T06:49:00.000Z

TILT is our shared learning experiment which allows for clinicians to record and share learning they may have experienced. We've now had over 850 TILTs so it's getting a useful resource.

Reading today's BMJ there's an article on postural hypotension which has a great section called 'Learning points'. These are not new but it occurred to me how close TILT is to the concept of learning points. In other words, what is the take home message from the article. It's different from simply the conclusion as it's more nuanced. So, two TILTs I added (out of a total of four) are:

Whenever I make connections between concepts (in this case TILT and learning points) it makes me contemplate things. Is TILT too niche? Should we automatically grab all learning points from BMJ articles? Why not try and create a database of everything we know - so TILTs, learning points and conclusions.

These issues and othes like it represent challenges, real and important. This is why I love doing what I do.

TRIP's use in SRs

2011-06-28T12:07:00.000Z

TRIP is frequently mentioned in systematic reviews as one of the sources searched for relevant studies. In fact the mentions seems to be accelerating and below are 5 recent examples:

Relative effectiveness of clinic and home blood pressure monitoring compared with ambulatory blood pressure monitoring in diagnosis of hypertension: systematic review (BMJ)
Adequacy of reporting monitoring regimens of risk factors for cardiovascular disease in clinical guidelines: systematic review (BMJ)
Available evidence does not support use of oxalates for dentine hypersensitivity (Evidence Based Dentistry)
Effectiveness of smoking cessation programs for seriously mentally ill (Actas Españolas de Psiquiatría)
Rigour of development does not AGREE with recommendations in practice guidelines on the use of ice for acute ankle (BMJ Quality & Safety)

I can't help feeling we're doing something right!

20,000 registered users

2011-06-21T18:23:00.002Z

Quite a milestone, over the weekend we hit 20,000 registered users.

We're currently adding 1,000 new registered users every 3-4 weeks (this has been consistent over many months). We're really pleased with this. However, it does put the onus on TRIP to create an even better product/experience for registrants.

Blitter - update

2011-06-10T16:54:00.000Z

We've had lots of feedback on Blitter, mostly positive but some less so (but constructive).

As such we've re-designed it, to emphasise our view that it's principally a search tool (as opposed to a news service).

Give it a whirl and let me know what you think.

http://blitter.tripdatabase.com/

New TRIP product

2011-06-03T18:19:00.000Z

This isn't really a proper TRIP product, more an experimental one!

http://blitter.tripdatabase.com/

The idea behind Blitter is that it only includes content that an independent clinician has deemed interesting/newsworthy enough to comment about. Most clinical search tools grab all the content from a particular publisher - irrespective of the clinical usefulness of the output. So, we see Blitter as being a bottom up approach to content identification - possibly making it more useful.

In addition we have classified each contributor by their clinical interest allowing users to filter results based on the speciality of the contributor - this is possibly an important development. Why? Take the search term pain, an oncologist searching for pain would typically want significantly different results compared to a rheumatologist or a generalist. Currently TRIP and all other clinical search tools show the same results - meaning lots of 'noise'. So, allowing users to restrict the results based on speciality should make the results more meaningful.

What's happening?

2011-05-27T08:26:00.000Z

While I've not written a blog article for over a month it does not mean we've been doing nothing - far from it we're busy busy! Below are a few examples

We're currently working on a major upgrade to TRIP which I hope will be out before the end of June.
We're also working on a social media search tool we're calling evidently. We've identified a number of bloggers and tweeters who regularly post about clinical issues and we're grabbing their outputs. Why? It's an experiment but we think that if these coal face clinicians make the effort to blog/tweet about an article they've read, it suggests it's newsworthy. So why not aggregate that and have a searchable database of noteworthy articles?
We've started talking with 3rd parties about potential investments in TRIP
We've added a whole bunch of new content as well as correcting a load of faulty guideline links!

In addition we've just had our 19,000 registered users.

There's more I want to say about all the above, but too busy to be coherent!

The cost of search (3)

2011-04-26T14:09:00.001Z

A further update on the cost of search via another Freedom of Information request to NHS Evidence.

In the previous analysis (click here) I highlighted their budget was over £24 million, of which over £10 million was on content (e.g. Cochrane, BNF) and a further £2 million was on the specialist libraries (which have now ceased to exist). Therefore, removing the cost of the specialist libraries and content (a total of £12,675,971) leaves their core budget of £11.8 million (I’ve rounded up).

The FoI request highlighted that actually only 384,661 searches are carried out via the NHS Evidence site per month (or, if multiplied by 12 it gives an approximate annual search total of 4.62 million searches (I’ve rounded up). I’ve excluded both the CKS and specialist library search figures (as I have their costs as they are no longer procured).

Therefore, a cost of £11.8 million yields 4.62 million searches = £2.55 per search.

The TRIP costs have not altered and remain at 0.43 pence per search.

So, each search of NHS Evidence appears to be 593 times more expensive than TRIP.

New email format

2011-04-13T15:11:00.000Z

A while ago we asked TRIP users what they thought of the monthly TRIP emails and we got a broadly positive response. However, we've tried to listen to some of the constructive comments and have created a 'mock-up' of what a new email system might look like (click on image to make bigger).

What we've done is simplify the columns (as we're introducing new categories in the near future - namely social media) so increasing the number of columns would look even messier! To see what's new you'd click on - say - cholesterol - and that would take you to the TRIP results page with ALL the new content for cholesterol. Users can then click on any category they like to see the relevant articles.

Please let me know what you think - leave a comment or email me: jon.brassey@tripdatabase.com

Landmark clinical papers

2011-04-07T13:58:00.009Z

Below is an initial list of landmark clinial trials and/or systematic reviews. Email me (jon.brassey@tripdatabase.com) for sugestions for additional trials or how to improve this resource.

Recently Added

CRASH trial collaborators. Final results of MRC CRASH, a randomised placebo-controlled trial of intravenous corticosteroid in adults with head injury-outcomes at 6 months. Lancet. 2005 Jun 4-10;365(9475):1957-9 - In 'Trauma'
Beal SM, Finch CF. An overview of retrospective case-control studies investigating the relationship between prone sleeping position and SIDS. J Paediatr Child Health. 1991 Dec;27(6):334-9 - In 'Pediatrics'
Ioannidis JPA, 2005 Why Most Published Research Findings Are False. PLoS Med 2(8). - In 'Other' section.
Evidence-Based Medicine Working Group (November 1992). Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA 268 (17): 2420–5. - In 'Other' section.
Action to Control Cardiovascular Risk in Diabetes Study Group. Effects of intensive glucose lowering in type 2 diabetes. N Engl J Med. 2008 Jun 12;358(24):2545-59. Epub 2008 Jun 6 - in 'Endocrinology'

Anaesthesia

Mendelson, C. L. The aspiration of stomach contents into the lungs during obstetric anesthesia. Amer. J. Obstet. Gynec., 52: 191. 1946. This study described a new type of adult respiratory distress syndrome due to aspiration of gastric hydrochloric acid. PMID: 20993766 (and here and here)
Catley DM, Thornton C, Jordan C, Lehane JR, Royston D, Jones JG. Pronounced, episodic oxygen desaturation in the postoperative period: its association with ventilatory pattern and analgesic regimen. Anesthesiology 1985;63:20-8. PMID: 4014768
Cormack RS Lehane JR Adams AP Carli F. Laryngosopy grades and percentage glottic openening. Anaesthesia. 2000;55(2):184. PMID: 10755959
Bailey AR, Jones JG. Patients' memories of events during general anaesthesia. Anaesthesia 1997. 52:460-76 PMID: 9165967
Severinghaus JW, Astrup P, and Murray JF Blood Gas Analysis and Critical Care Medicine. Am. J. Respir. Crit. Care Med., Volume 157, 1998, S114-S122 PMID: 9563770
Murphy PG, Myers DS, Davies MJ, Webster NR, and Jones JG. The antioxidant potential of propofol (2,6-diisopropylphenol). Br J Anaesth 1992;68:613-618. PMID: 1319189

Cardiology

Yusuf S, Sleight P, Pogue J, Bosch J, Davies R, Dagenais G. Effects of an angiotensin-converting-enzyme inhibitor, ramipril, on cardiovascular events in high-risk patients. The Heart Outcomes Prevention Evaluation Study Investigators.N Engl J Med. 2000 Jan 20;342(3):145-53. PMID: 10639539 (and here)
Antiplatelet Trialists' Collaboration. Collaborative overview of randomised trials of antiplatelet therapy--I: Prevention of death, myocardial infarction, and stroke by prolonged antiplatelet therapy in various categories of patients BMJ. 1994 Jan 8;308(6921):81-106. PMID 8298418
Cardiac Arrhythmia Suppression Trial (CAST) Investigators. Preliminary report: effect of encainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction. N Engl J Med. 1989;321:406-412. PMID 2473403
Randomized trial of cholesterol lowering in 4444 patients with coronary heart disease: The Scandinavian Simvastatin Survival Study Lancet 1994; 344 1383-1389 PMID 7968073
Thomas R. Dawber, M.D., Gilcin F. Meadors, M.D., M.P.H., and Felix E. Moore, Jr., National Heart Institute, National Institutes of Health, Public Health Service, Federal Security Agensy, Washington, D. C., Epidemiological Approaches to Heart Disease: The Framingham Study Presented at a Joint Session of the Epidemiology, Health Officers, Medical Care, and Statistics Sections of the American Public Health Association, at the Seventy-eighth Annual Meeting in St. Louis, Mo., November 3, 1950. Framingham Heart Study
Teo KK, Yusuf S, Fuberg CD. Effects of prophylactic anti-arrhythmic drug therapy in acute myocardial infarction. JAMA 1993;270:1589-95. PMID: 8371471
Waldo AL, Camm AJ, deRuyter H, Friedman PL, Macneil DJ, Pauls JF, Pitt B, Pratt CM, Schwartz PJ, Veltri EP. Effect of d-sotalol on mortality in patients with left ventricular dysfunction after recent and remote myocardial infarction. Lancet. 1996;348:7-12. PMID: 8691967
Theroux P, Ouimet H, McCans J, et al. Aspirin, heparin or both to treat unstable angina. N Engl J Med 1988;319:1105-11. PMID: 3050522
Curb JD, Pressel SL, Cutler JA, et al. Effect of diuretic-based antihypertensive treatment on cardiovascular disease risk in older diabetic patients with isolated systolic hypertension. JAMA. 1996;276:1886-1892 PMID: 8968014
Herrington DM et al. Effects of estrogen replacement on the progression of coronary-artery atherosclerosis. N Engl H Med 2000;343:522-9. PMID: 10954759
ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2. Lancet. 1988 Aug 13;2(8607):349-60. PMID: 2899772
Effect of metoprolol CR/XL in chronic heart failure: Metoprolol CR/XL Randomised Intervention Trial in Congestive Heart Failure (MERIT-HF). Lancet. 1999 Jun 12;353(9169):2001-7. PMID: 10376614
Effects of enalapril on mortality in severe congestive heart failure. Results of the Cooperative North Scandinavian Enalapril Survival Study (CONSENSUS). The CONSENSUS Trial Study Group. N Engl J Med 1987;316(23):1429-35. PMID: 2883575
Gruppo Italiano per lo Studio della Streptochi-nasi nell'Infarto Miocardico (GISSI). Long-term effects of intravenous thrombolysis in acute myocardial infarction: final report of the GISSI study. Lancet 1987; 2(8564):871-4. PMID: 2889079
Colwell, C. W., Jr., D. K. Collis, et al. (1999). Comparison of enoxaparin and warfarin for the prevention of venous thromboembolic disease after total hip arthroplasty. Evaluation during hospitalization and three months after discharge. J Bone Joint Surg Am 81(7): 932-40. PMID: 10428124
Leroyer, C., L. Bressollette, et al. (1998). Early versus delayed introduction of oral vitamin K antagonists in combination with low-molecular-weight heparin in the treatment of deep vein thrombosis. A randomized clinical trial. The ANTENOX Study Group. Haemostasis 28(2): 70-7 PMID: 10087431
Decousus, H., A. Leizorovicz, et al. (1998). A clinical trial of vena caval filters in the prevention of pulmonary embolism in patients with proximal deep-vein thrombosis. Prevention du Risque d'Embolie Pulmonaire par Interruption Cave Study Group [see comments]. N Engl J Med 338(7): 409-15. PMID: 9459643
ENOXACAN Study Group. Efficacy and safety of enoxaparin versus unfractionated heparin for prevention of deep vein thrombosis in elective cancer surgery: a double-blind randomized multicentre trial with venographic assessment. Br J Surg 1997;84(8): 1099-103. PMID: 9278651
Turpie, A. G., M. N. Levine, et al. (1986). A randomized controlled trial of a low-molecular-weight heparin (enoxaparin) to prevent deep-vein thrombosis in patients undergoing elective hip surgery. N Engl J Med 315(15): 925-9. PMID: 3531851
Planes, A., N. Vochelle, et al. (1988). Prevention of postoperative venous thrombosis: a randomized trial comparing unfractionated heparin with low molecular weight heparin in patients undergoing total hip replacement. Thromb Haemost 60(3): 407-10. PMID: 2853459
The Columbus Investigators. Low-molecular-weight heparin in the treatment of patients with venous thromboembolism. N Engl J Med 1997;337(10):657-62. PMID: 9280815
Barritt DW,Jordan SC. Anticoagulant drugs in the treatment of pulmonary embolism. A controlled trial. Lancet 1960;1:1309-1312 PMID: 13797091
Simonneau, G., H. Sors, et al. (1997). A comparison of low-molecular-weight heparin with unfractionated heparin for acute pulmonary embolism. The THESEE Study Group. Tinzaparine ou Heparine Standard: Evaluations dans l'Embolie Pulmonaire N Engl J Med 337(10): 663-9 PMID: 9278462
Stroke Prevention in Atrial Fibrillation Study. Final results [see comments]. Circulation 1991;84(2): 527-39. PMID: 1860198
Flaker, G. C., J. L. Blackshear, et al. (1992). Antiarrhythmic drug therapy and cardiac mortality in atrial fibrillation. The Stroke Prevention in Atrial Fibrillation Investigators. J Am Coll Cardiol 20(3): 527-32. PMID: 1512329
Adjusted-dose warfarin versus low-intensity, fixed-dose warfarin plus aspirin for high-risk patients with atrial fibrillation: Stroke Prevention in Atrial Fibrillation III randomised clinical trial [see comments]. Lancet 1996;348(9028): 633-8. PMID: 8782752
Stampfer MJ, Goldhaber SZ, Yusuf S, Peto R, Hennekens CH. Effect of intravenous streptokinase on acute myocardial infarction: pooled results from randomized trials. N Engl J Med. 1982 Nov 4;307(19):1180-2. PMID: 6750403

Cystic Fibrosis

Gibson RL et al. Significant microbiological effect of inhaled tobramycin in young children with cystic fibrosis. Am J Respir Crit Care Med 2003;167(6):841–9 PMID: 12480612
Valerius NH, Koch C, Hoiby N. Prevention of chronic Pseudomonas aeruginosa colonisation in cystic fibrosis by early treatment. Lancet 1991;338(8769):725–6 PMID: 1679870
MacLusky IB, Gold R, Corey M, Levison H. Long-term effects of inhaled tobramycin in patients with cystic fibrosis colonized with Pseudomonas aeruginosa. Pediatr Pulmonol, 1989;7(1):42–8 PMID: 2505216
Gappa M, Steinkamp G, Tummler B, Von Der Hardt H. Long-term tobramycin aerosol therapy of chronic Pseudomonas aeruginosa infection in patients with cystic fibrosis. Scand J Gastroenterol 1987;23(Suppl 143):74–6 PMID: 3164515
Littlewood JM, Miller MG, Ghoneim AT, Ramsden CH. Nebulised colomycin for early pseudomonas colonisation in cystic fibrosis. Lancet 1985;1(8433):865 PMID: 2858720
Wall MA, Terry AB, Eisenberg J, Mcnamara M, Cohen R. Inhaled antibiotics in Cystic Fibrosis. Lancet 1983;1(8337):1325 PMID: 6134106
Hodson ME, Penketh ARL, Batten JC. Aerosol carbenicillin and gentamicin treatment of Pseudomonas aeruginosa infection in patients with cystic fibrosis. Lancet 1981;2(8256):1137–9 PMID: 6118579

Endocrinology

The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. The Diabetes Control and Complications Trial Research Group. N Engl J Med. 1993 Sep 30;329(14):977-86. PMID: 8366922 (also here, here and here)
Conn JW. Presidential address. I. Painting background. II. Primary aldosteronism, a new clinical syndrome. J lab clin med 1955;45(1):3 PMID: 13233623
Conn JW, Cohen EL and Rovner DR. Suppression of Plasma Renin Activity in Primary Aldosteronism. Distinguishing Primary from Secondary Aldosteronism in Hypertensive Disease. JAMA 1964; 190:213 21. and republished in JAMA 1985;253(4):558 as landmark article PMID: 3881606
Cushing HW The basophil adenomas of the pituitary body and their clinical manifestations(pituitary basophilism). Bull Johns Hopkins Hosp 1932; 50:137–195 Full text
Action to Control Cardiovascular Risk in Diabetes Study Group. Effects of intensive glucose lowering in type 2 diabetes. N Engl J Med. 2008 Jun 12;358(24):2545-59. Epub 2008 Jun 6. PMID: 18539917 (and full text)

Epidemiology

Doll Richard, Bradford Hilly A (June 26, 1954). "The mortality of doctors in relation to their smoking habits. A preliminary report". British Medical Journal 1 (4877): 1451–55. PMID 13160495
Antiplatelet Trialists' Collaboration. Collaborative overview of randomised trials of antiplatelet therapy--I: Prevention of death, myocardial infarction, and stroke by prolonged antiplatelet therapy in various categories of patients BMJ. 1994 Jan 8;308(6921):81-106. PMID 8298418
Cardiac Arrhythmia Suppression Trial (CAST) Investigators. Preliminary report: effect of encainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction. N Engl J Med. 1989;321:406-412. PMID 2473403
MRC Streptomycin in Tuberculosis Trials Committee. Streptomycin treatment of pulmonary tuberculosis. BMJ. 1948;ii:769–783. James Lind summary
Thomas Francis, Robert Korn, et al. "An Evaluation of the the 1954 Poliomyelitis Vaccine Trials." American Journal of Public Health 45 (1955), 50 page supplement with a 63 page appendix. The salk vaccine trials
Randomized trial of cholesterol lowering in 4444 patients with coronary heart disease: The Scandinavian Simvastatin Survival Study Lancet 1994; 344 1383-1389 PMID 7968073
The Editors Of The Lancet (February 2010). "Retraction--Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children". Lancet 375 (9713): 445. doi:10.1016/S0140-6736(10)60175-4. PMID 20137807.
Bonadonna G, Brusamolino E, Valagussa P, Veronesi U. Adjuvant study with combination chemotherapy in operable breast cancer. Proc Am Assoc Cancer Res Am Soc Clin Oncol 1975;16:254-and N Engl J Med. 1976 Feb 19;294(8):405-10. PMID 1246307
Thomas R. Dawber, M.D., Gilcin F. Meadors, M.D., M.P.H., and Felix E. Moore, Jr., National Heart Institute, National Institutes of Health, Public Health Service, Federal Security Agensy, Washington, D. C., Epidemiological Approaches to Heart Disease: The Framingham Study Presented at a Joint Session of the Epidemiology, Health Officers, Medical Care, and Statistics Sections of the American Public Health Association, at the Seventy-eighth Annual Meeting in St. Louis, Mo., November 3, 1950. Framingham Heart Study
On the Mode of Communication of Cholera by John Snow, M.D. London: John Churchill, New Burlington Street, England, 1855 On the Mode of communication of cholera

Gastroenterology

Miao LL. Gastric freezing: an example of the evaluation of medical therapy by randomized trials. In: Bunker JP, Barnes, Mosteller F, eds. Costs, risks, and benefits of surgery. Oxford University Press, New York: 1977, pp 198-211
Riemann, J. F., D. Schilling, et al. (1997). Cure with omeprazole plus amoxicillin versus long-term ranitidine therapy in Helicobacter pylori-associated peptic ulcer bleeding [see comments]. Gastrointest Endosc 46(4): 299-304. PMID: 9351030
Graham, D. Y., G. M. Lew, et al. (1992). Effect of treatment of Helicobacter pylori infection on the long-term recurrence of gastric or duodenal ulcer. A randomized, controlled study [see comments]. Ann Intern Med 116(9): 705-8. PMID: 1558340

Hematology

Ungley CC: Vitamin B12 in pernicious anaemia: parenteral administration. BMJ 1949; II: 1370-1377. PMID: 15397501

Infectious disease

On the Mode of Communication of Cholera by John Snow, M.D. London: John Churchill, New Burlington Street, England, 1855 On the Mode of communication of cholera
MRC Streptomycin in Tuberculosis Trials Committee. Streptomycin treatment of pulmonary tuberculosis. BMJ. 1948;ii:769–783. James Lind summary
Thomas Francis, Robert Korn, et al. "An Evaluation of the the 1954 Poliomyelitis Vaccine Trials." American Journal of Public Health 45 (1955), 50 page supplement with a 63 page appendix. The salk vaccine trials
Warrell DA, Looareesuwan S, Warrell MJ, Kasemsarn P, Intaraprasert R, Bunnag D, Harinasuta T Dexamethasone proves deleterious in cerebral malaria. A double-blind trial in 100 comatose patients.N Engl J Med 1982 Feb 11;306(6):313-9 PMID: 7033788
Abraham EP, Chain E, Fletcher CM, Gardner AD, Heatley NG, Jennings MA: Further observations on penicillin. Lancet 1941; 2: 177-88

Nephrology

Lewis EJ, Hunsicker LG, Lan SP, Rohde RD, Lachin JM. A controlled trial of plasmapheresis therapy in severe lupus nephritis. The Lupus Nephritis Collaborative Study Group. N Engl J Med. 1992 May 21;326(21):1373-9. PMID: 1569973

Neurology

The EC/IC Bypass Study Group. Failure of Extracranial-Intracranial Arterial Bypass to Reduce the Risk of Ischemic Stroke. N.Engl.J.Med. 1985; 313: 1191-1200 PMID: 2865674

Oncology

Bonadonna, G., E. Brusamolino, et al. (1976). Combination chemotherapy as an adjuvant treatment in operable breast cancer. N Engl J Med 294(8):405-10. PMID: 1246307
Rossi, A., G. Bonadonna, et al. (1981). Multimodal treatment in operable breast cancer: five-year results of the CMF programme. Br Med J (Clin Res Ed) 282(6274): 1427-31. PMID: 6784849
Bonadonna, G., M. Zambetti, et al. (1995). Sequential or alternating doxorubicin and CMF regimens in breast cancer with more than three positive nodes. Ten-year results [see comments]. JAMA 273(7):542-7. PMID: 7837388
Bonadonna, G., P. Valagussa, et al. (1995). Adjuvant cyclophosphamide, methotrexate, and fluorouracil in node- positive breast cancer: the results of 20 years of follow-up [see comments]. N Engl J Med 332(14): 901-6. PMID: 7877646
Fisher, B., M. Bauer, et al. (1985). Five-year results of a randomized clinical trial comparing total mastectomy and segmental mastectomy with or without radiation in the treatment of breast cancer. N Engl J Med 312(11):665-73. PMID: 3883167
Fisher, B., C. Redmond, et al. (1985). Ten-year results of a randomized clinical trial comparing radical mastectomy and total mastectomy with or without radiation. N Engl J Med 312(11): 674-81. PMID: 3883168
Fisher, B., C. Redmond, et al. (1990). Postoperative chemotherapy and tamoxifen compared with tamoxifen alone in the treatment of positive-node breast cancer patients aged 50 years and older with tumors responsive to tamoxifen: results from the National Surgical Adjuvant Breast and Bowel Project B-16 [see comments]. J Clin Oncol 8(6): 1005-18. PMID: 2189950
Fisher, B., A. Brown, et al. (1997). Effect of preoperative chemotherapy on local-regional disease in women with operable breast cancer: findings from National Surgical Adjuvant Breast and Bowel Project B-18 [see comments]. J Clin Oncol 15(7): 2483-93. PMID: 9215816
Canellos GP, Anderoson JP, Propert KJ, etl. Chemotherapy of advanced Hodgkin's disease with MOPP, ABVD vs. MOPP alternating with ABVD. N Engl J Med 1992;327:1478 PMID: 1383821
Fisher, B., N. Wolmark, et al. (1988). Postoperative adjuvant chemotherapy or radiation therapy for rectal cancer: results from NSABP protocol R-01. J Natl Cancer Inst 80(1): 21-9 PMID: 3276900
Wolmark, N., H. Rockette, et al. (1999). Clinical trial to assess the relative efficacy of fluorouracil and leucovorin, fluorouracil and levamisole, and fluorouracil, leucovorin, and levamisole in patients with Dukes' B and C carcinoma of the colon: results from National Surgical Adjuvant Breast and Bowel Project C-04. J Clin Oncol 17(11): 3553-9. PMID: 10550154
Okawara, G., J. Rusthoven, et al. (1997). Unresected stage III non-small-cell lung cancer. Provincial Lung Cancer Disease Site Group. Cancer Prev Control 1(3): 249-59. PMID: 9765750
McKelvey EM, Gottlieb JA, Wislon HE, et el.: Hydroxyadaunomycin (Adriamycin) combination chemotherapy in malignant lymphoma. Cancer 1976; 38: 1484. PMID: 791473
Tirelli, U., D. Errante, et al. (1998). CHOP is the standard regimen in patients > or = 70 years of age with intermediate-grade and high-grade non-Hodgkin's lymphoma: results of a randomized study of the European Organization for Research and Treatment of Cancer Lymphoma Cooperative Study Group. J Clin Oncol 16(1): 27-34. PMID: 9440719
Fisher, R. I., E. R. Gaynor, et al. (1993). Comparison of a standard regimen (CHOP) with three intensive chemotherapy regimens for advanced non-Hodgkin's lymphoma [see comments]. N Engl J Med 328(14): 1002-6. PMID: 7680764
Miller, T. P., S. Dahlberg, et al. (1998). Chemotherapy alone compared with chemotherapy plus radiotherapy for localized intermediate- and high-grade non-Hodgkin's lymphoma [see comments]. N Engl J Med 339(1): 21-6. PMID: 9647875
Stadtmauer EA, O'Neill OA, Goldstein LJ et al., Conventional-Dose Chemotherapy Compared with High-Dose Chemotherapy plus Autologous Hematopoietic Stem-Cell Transplantation for Metastatic Breast Cancer. N Engl J Med 2000;342:1069-76. PMID: 10760307
Einhorn LH, Donohue J. Cis-diamminedichloroplatinum, vinblastine, and bleomycin combination chemotherapy in disseminated testicular cancer. Ann Intern Med. 1977 Sep;87(3):293-8. PMID: 71004
Devita VT Jr, Serpick AA, Carbone PP. Combination chemotherapy in the treatment of advanced Hodgkin's disease. Ann Intern Med. 1970 Dec;73(6):881-95. PMID: 5525541
Piro LD, Carrera CJ, Carson DA, Beutler E. Lasting remissions in hairy-cell leukemia induced by a single infusion of 2-chlorodeoxyadenosine. N Engl J Med. 1990 Apr 19;322(16):1117-21. PMID: 1969613

Ophthalmology

Optic nerve decompressive surgery for nonarteritic anterior ischemic optic neuropathy (NAION) is not effective and may be harmful. The Ischemic Optic Neuropathy Decompression Trial Research Group [see comments]. JAMA 1995; 273: 625-632 PMID: 7844872

Pediatrics

Beal SM, Finch CF. An overview of retrospective case-control studies investigating the relationship between prone sleeping position and SIDS. J Paediatr Child Health. 1991 Dec;27(6):334-9. PMID: 1836736

Respiratory

The TORCH trial. Calverley PM, Anderson JA, Celli B, Ferguson GT, Jenkins C, Jones PW, Yates JC, Vestbo J; TORCH investigators. Salmeterol and fluticasone propionate and survival in chronic obstructive pulmonary disease. N Engl J Med. 2007 Feb 22;356(8):775-89. PMID: 17314337 (and here)
The UPLIFT study. Tashkin DP, Celli B, Senn S, Burkhart D, Kesten S, Menjoge S, Decramer M; UPLIFT Study Investigators. A 4-year trial of tiotropium in chronic obstructive pulmonary disease. N Engl J Med. 2008 Oct 9;359(15):1543-54 PMID: 18836213 (and here)
Crane J. Pearce N. Flatt A. Burgess C. Jackson R. Kwong T. Ball M. Beasley R. Prescribed fenoterol and death from asthma in New Zealand, 1981-83: case-control study [see comments]. Lancet. 1(8644):917-22, 1989 Apr 29. PMID: 2565417

Surgery

An evaluation of internal-mammary-artery ligation by a double-blind technic. Cobb LA, Thomas GI, Dillard DH, Merendino KA, Bruce RA. N Engl J Med. 1959 May 28;260(22):1115-8. PMID: 13657350
Comparison of internal mammary artery ligation and sham operation for angina pectoris. Dimond EG, Kittle CF, Crockett JE. Am J Cardiol. 1960 Apr;5:483-6. PMID: 13816818
POISE Study Group. Effects of extended-release metoprolol succinate in patients undergoing non-cardiac surgery (POISE trial): a randomised controlled trial. Lancet. 2008 May 31;371(9627):1839-47. Epub 2008 May 12. PMID: 18479744

Trauma

CRASH trial collaborators. Final results of MRC CRASH, a randomised placebo-controlled trial of intravenous corticosteroid in adults with head injury-outcomes at 6 months. Lancet. 2005 Jun 4-10;365(9475):1957-9. PMID: 15936423

Urology

Roos NP, Wennberg JE, Malenka DJ, et al. Mortality and reoperation after open and transurethral resection of the prostate for benign prostatic hyperplasia. N. Engl J Med. 1989;320:1120-1124 PMID: 2469015

Women's health

International randomised controlled trial of acetazolamide and furosemide in posthaemorrhagic ventricular dilatation in infancy. International PHVD Drug Trial Group. Lancet. 1998 Aug 8;352(9126):433-40. PMID: 9708751
Routine vs selective episiotomy: a randomised controlled trial. Argentine Episiotomy Trial Collaborative Group. Lancet. 1993 Dec 18-25;342(8886-8887):1517-8. PMID: 7902901
Klein MC, Gauthier RJ, Jorgensen SH, Robbins JM, Kaczorowski J, Johnson B, Corriveau M, Westreich R, Waghorn K, Gelfand MM, et al. Does episiotomy prevent perineal trauma and pelvic floor relaxation? [published erratum appears in Online J Curr Clin Trials 1992 Sep 12;Doc No 20:[54 words; 1 paragraphs]. Online J Curr Clin Trials. 1992 Jul 1;Doc No 10:[6019 words; 65 paragraphs]. PMID: 1343606
Chalmers TC. The impact of controlled trials on the practice of medicine. Mt Sinai J Med 1974;41:753-759 PMID: 4547818
ACTOBAT Study Group. Australian collaborative trial of antenatal thyrotropin-releasing hormone (ACTOBAT) for prevention of neonatal respiratory disease Lancet, 1995; 345:877-882. PMID: 7707809
Lucas MJ, Leveno KJ, Cunningham FG. A comparison of magnesium sulfate with phenytoin for the prevention of eclampsia. N Eng J Med 1995 Jul 27; 333(4):201-5 PMID: 7791836
Czeizel E, Dudas I. [Prevention of the first occurrence of anencephaly and spina bifida with periconceptional multivitamin supplementation]. Orv Hetil 1994;135:2313-7 PMID: 7970646
Hulley S, Grady D, Bush T, Furberg C, Herrington D, Riggs B, Vittinghoff E. Randomized Trial of Estrogen Plus Progestin for Secondary Prevention of Coronary Heart Disease in Postmenopausal Women. JAMA 1998: 280(7); 605-613. PMID: 9718051
Writing Group for the Women's Health Initiative Investigators. Risks and Benefits of Estrogen Plus Progestin in Healthy Postmenopausal Women. JAMA 2002;288:321-333. PMID: 12117397 (and full text)

Other

Ransohoff DF, Feinstein AR. Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. N Engl J Med 1978: 299(17); 926-30 PMID: 692598
Rosa L, Rosa E, Sarner L, Barrett S. A close look at therapeutic touch. JAMA 1998: 279(13); 1005-10 PMID: 9533499
Is the placebo powerless? An analysis of clinical trials comparing placebo with no treatment. Hrobjartsson A, Gotzsche PC. N Engl J Med 2001: 344(21); 1594-602 PMID: 11372012
Ioannidis J, Cappelleri J, Lau J. Issues in Comparisons between Meta-analyses and Large Trials. JAMA 2002: 279(14); 1089-93. PMID: 9546568
Stelfox HT, Chua G, O. Rourke K, Detsky AS.Conflict of interest in the debate over calcium-channel antagonists. N Engl J Med 1998: 338(2); 101-6 PMID: 9420342
Monto AS. The disappearance of Reye's syndrome--a public health triumph. N Engl J Med 1999: 340(18); 1423-4. PMID: 10228195
Ioannidis JPA, 2005 Why Most Published Research Findings Are False. PLoS Med 2(8). PMID: 16060722 (and full text)
Evidence-Based Medicine Working Group (November 1992). Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA 268 (17): 2420–5. PMID: 1404801 (and full text)

Contributors
@rostrevorrose, Professor Gareth Jones, @mikey3982, TrustTheEvidence, Steve Simon, Justus Krabshuis, Ben Djulbegovic, David R. Rovner, Pablo Alonso, Carlos Cuello

evidently.so

2011-04-03T09:30:00.000Z

What has TRIP got to do with Somalia?

The truth is very little. However, at the start of this month we secured the domain name evidently.so (really http://www.evidently.so - but don't click there as there's nothing to see just yet). The .so domain is the country code extension for Somalia. I purchased it on a whim, I had no idea we'd get it or what we'd use it for. I just liked the phrase - it fits in with the 'evidence' concept and is an easy to remember name.

But we've got it and the more I reflect on it the more pleased I am with it. I think it's got all the potential to be a strong brand.

As for what to do with it, I'm currently thinking it might be a good home for our 'new evidence' service - which is currently just a vague concept. One to ponder, but if you have any suggestions?

Experiment with new cardiology content

2011-03-27T16:44:00.000Z

I've been thinking how useful all the new content added - each month - to TRIP is. We typically add between 400-700 articles each month manually (these are typically secondary evidence type articles), of these about 60% are recent publications and the rest from previous years. In addition to this manual update we typically add around 3-5,000 new articles automatically (via various mechanisms).

I typically view the manually added content as being the 'best stuff'! So, can we use this 'best stuff' in a meaningful way? I believe so, but have no idea how! Identifying it's easy - displaying it is more problematic. So, in an effort to help me understand the issues I've decided to experiment! Therefore, I've taken March's new content and extract all recent articles that relate to cardiology - 52 in total. And below is my first effort at presenting them in a structured form (it's a pragmatic structure - if that's not obvious!) - please let me know what you think!

Lipids

Mipomersen - Familial hypercholesterolemia - third or fourth line (National Horizan Scanning Centre)
Rosuvastatin and the JUPITER trial. A critical appraisal (DTB, Navarre)
Continue to follow NICE guidance when prescribing statins (NPC)
Ezetimibe/simvastatin may reduce CV events in advanced CKD: but is it better than simvastatin alone? (NPC)
Limited observational evidence for CV risk reduction with exenatide (NPC)
Managing lipids (National Prescribing Service)
Protection Against Nephropathy in Diabetes with Atorvastatin (PANDA): a randomized double-blind placebo-controlled trial of high- vs. low-dose atorvastatin (Evidence Updates)
The cost-effectiveness of C-reactive protein testing and rosuvastatin treatment for patients with normal cholesterol levels (Evidence Updates)

Stroke

Prognostic Risk Stratification Instruments for TIA (Washington University Emergency Medicine Journal Club)
Elevation of serum magnesium may improve clinical outcome after aneursymal subarachnoid haemorrhage (BestBETs)
Current evidence does not support the use of a negative D-dimer to rule out suspected pulmonary embolism in pregnancy (BestBETs)
Stroke Prevention in the Setting of a Patent Foramen Ovale: A “Hole” in the Evidence (Clinical Correlations)
Neurothrombectomy devices for the treatment of acute ischemic stroke: state of the evidence (DARE)
An assessment of the incremental value of the ABCD2 score in the emergency department evaluation of transient ischemic attack (Evidence Updates)
Computed tomographic angiography and venography for young or nonhypertensive patients with acute spontaneous intracerebral hemorrhage (Evidence Updates)
Association between obesity and mortality after acute first-ever stroke: the obesity-stroke paradox (Evidence Updates)
Very early mobilization after stroke fast-tracks return to walking: further results from the phase II AVERT randomized controlled trial (Evidence Updates)
The safety of proton pump inhibitors and clopidogrel in patients after stroke (Evidence Updates)
iScore: A Risk Score to Predict Death Early After Hospitalization for an Acute Ischemic Stroke (Evidence Updates)

Myocardial Infarcation

CV Risk

Clinical Rules

Prognostic Risk Stratification Instruments for TIA (Washington University Emergency Medicine Journal Club)
An assessment of the incremental value of the ABCD2 score in the emergency department evaluation of transient ischemic attack (Evidence Updates)
The Randomised Assessment of Treatment using Panel Assay of Cardiac Markers (RATPAC) trial: a randomised controlled trial of point-of-care cardiac markers in the emergency department (Evidence Updates)
Comparing the diagnostic performance of 2 clinical decision rules to rule out deep vein thrombosis in primary care patients (Evidence Updates)
Genetic warfarin dosing tables versus algorithms (Evidence Updates)
Predictive accuracy of SYNTAX score for predicting long-term outcomes of unprotected left main coronary artery revascularization (Evidence Updates)
The cost-effectiveness of C-reactive protein testing and rosuvastatin treatment for patients with

Surgery

Other

NICE guidance. Is it inaccessible?

2011-03-24T21:56:00.000Z

This is such an crucial video NICE guidance. Is it inaccessible? It's been produced by a general practitioner from South Wales (Anne Marie Cunningham). Although Anne Marie focuses on NICE guidance the issues affect other guidelines and many other sources of evidence.

The challenges Anne Marie raise are important and people interested in EBM/EBHC/EBP need to understand the problems front-line clinicians face. When Anne Marie can get the information she needs quickly and with little effort we'll be in a better place.

However, we're miles away from there and the challenges are daunting!

Word cloud of new content on TRIP

2011-03-18T12:49:00.000Z

Each month we add add content via two main ways:

Automatically - this uses various mechanisms (e.g. RSS)
Manually - I go and look at various sites, typically those dealing with secondary evidence and grab any new content.

For this month (March 2011) I've just uploaded the monthly manual content. This consisted of approximately four hundred articles dated 2011 and four hundred from 2010 (or earlier). As I'm looking at ways of allowing users to more easily view new content I've experimented with a wordcloud using Wordle. As many people have trouble viewing these I've embedded a screen grab below. To view it as large size you'll need to click on it.

NOTE: I've edited the words (taken from the document titles only) to exclude certain non-clinical terms (e.g. trial, versus, randomized).

16,000 registered user

2011-03-14T06:03:00.000Z

We've just had our 16,000 registered user of TRIP, that's 1,000 new registrants in 3 weeks.

Our 16,000 user was a student from Fiji!

Facebook update

2011-03-08T08:06:00.000Z

Here are a few recent Facebook posts (to see our full Facebook page click here). Remember, the Facebook posts are typically small posts that are perhaps not 'worthy' of a blog post (but feel free to contradict me!)

The BNF is now live on the TRIP Database (February 12th)
Just about to add 300+ clinical guidelines from the Australian guidelines portal (February 15th)
Just added records from two sites aimed at undergraduate medical students: Almostadoctor & MediVids (February 17th)
Highlighted how good TILT was at highlighting significant new research (February 25th)
Work on updating the systematic review filter that auto-searches PubMed (February 28th)
Reported on the positive response to a training session I ran for a NHS organisation and offering to run more courses - just ask (March 2nd)
I'm looking to introduce OpenURL into TRIP and gradually understanding the issues/complexity (March 3rd)
Exploring hooking TRIP up to Google Translate (e.g. Spanish & Welsh) (March 7th)

We've been busy!

Monthly emails from TRIP

2011-02-08T08:51:00.000Z

When users register on TRIP they can record keywords of interest and/or clinical areas of interest. By doing this we send them a monthly email with new content that matches their interests. This information is displayed in the following format (click on image to enlarge):

I'm not a huge fan of this look as I think it's too complicated. I raised this with Phil (the TRIP techie) and he wasn't convinced by some of my ideas - so he suggested I ask the users what they thought! The results are below...

When the monthly email arrives from TRIP, do you

Ignore it – 2%
Skim read it – 35%
Follow some of the links – 62%
I never receive them – 1%

The TRIP emails currently contain links to all new content. We are considering including the "top" 2 or 3 articles in each subject/interest area - highlighting them (title, publication and URL) in the email. What do you think?

Sounds good, yes please – 77%
Sounds good, but I'm worried it would make the email too long – 15%
No thanks, there's already too much information – 8%

Overall, how would you rate the emails

Great – 22%
Good – 56%
Ok – 17%
Poor – 5%
Awful – 0%

What do you most like about the current emails?

short, infrequent
They are useful for concise
They are well-organized and easy to use
All wanted information is very clear arranged and I can grasp on more details with only one klick.
Helps me with current awareness derived from a credible source! I often suggest my clients sign up for these emails directly

What do you most dislike?

tough to see the significance without ANY detail
Some features didn't appear to work, specifically, I've clicked and chosen some of the links and have tried both the email and the send to rtf file but neither of those features have worked for me.
Not enough information to help me decide whether to click on links
Too wordy. Bullet points with links if the item looks interesting
There is a lot of unrelated information in the links. Articles come up that have nothing to do with the topic.
No information about the top articles in the emails compels readers to click on the multiple links. If article titles are listed, like Table of Content alerts from Journals, would be more user friendly.

We also asked for additional comments and got a number - including some significant love and fondness for TRIP. A few raised issues of accuracy of the results (an issue we're aware of an looking into). The issue of primary care content raises it head yet again. The content for the emails is automated and I can think of no way of automatically deciding if an article is suitable for primary care or not.

Overall, it appears that most people are happy with the emails (I'm surprised) but a number would like additional information displayed to give a flavour of the content. In the above example, with 12 interests and over 3,000 links - how do you select 4-5 that give a flavour?

These are all challenges that we need to deal with. Fortunately, as people are broadly happy with the emails there is no huge pressure - so we'll include it in our next upgrade (in 2-3 months).

But if anyone cares to make a suggest we'd love to hear from you...

Contextual discovery & TRIP

2011-02-03T09:29:00.000Z

A while ago I read this article on TechCrunch Marissa Mayer’s Next Big Thing: “Contextual Discovery” — Google Results Without Search and it inspired me. It helped me discover a recurring theme in my thoughts (and on this blog) that search is useful for known unknowns. The flipside of that is unknown unknowns, something you didn't know you didn't know. In the above article Mayer talks about pushing information to people. She talks alot about location but it doesn't need to be restricted to that.

Currently on TRIP we have the ability for people to record their clinical area of interest and/or specific keywords of interest. The specific keywords works reasonably well. You say you're interested in stable angina, and we email you every article that is about stable angina - simple and the feedback is positive. Move to the wider clinical areas of interest (e.g. cardiology) and things get more problematic as it's so broad.

But what if you're like one of the people who doesn't register or if they do they don't record any interests? Fine you use TRIP, it works well and that's arguably enough.

But I want TRIP to do more.

If we encourage people to login we can start to note the articles they click on and start to create clinical hot spots of interest. When they next login to TRIP (or we email them) we show them new articles that match there previous interests. They don't need to search. There are a few issues (e.g. does previous searching predict future interests?) but I feel this is an avenue worth pursing. I'm actually confident we can create these clinical hotspots and use that to the benefit of our users.

But what do you think? Please comment to let me know.

The cost of search (2)

2011-01-21T09:31:00.002Z

In the middle of last year I attempted to draw conclusions about the difference in costs between TRIP and NHS Evidence (click here for the full blog post). Since then I've been trying to understand the costs of NHS Evidence and have managed to arrive at the following figures (NOTE: I've had to use Freedom of Information requests which are tedious see here & here. I welcome NHS Evidence correcting any figures below):

Total budget - £24,438,000
Content costs (BNF, Cochrane) - £10,675,971
Remaining costs (This includes the budget for the specialist collections which I understand is approximately £2,000,000 for all 34) - £13,762,029

Of those remaining costs, here's a flavour of the spending:

Consultants and other temporary staff - £2,813,258
Staff costs (the 41 employees) - £2,863,742
Marketing - £730,000

Using the same search figures as before = 15,811,716 searches at a cost of £13,762,029 (excluding content costs from the total budget). BTW since I quoted that figure a few people have suggested that actual figure for number of searches is too high as it includes search figures for other databases managed by NHS Evidence. However, a specific question, via Freedom of Information, refutes this - so I will use this higher figure.

Each search on NHS Evidence costs 87.04 pence

TRIP on the other hand has now reduced its costs (on the TRIP Database) and will this year run on a maximum of £35,000. Using the same search figures as before (8,058,648) this equates to a cost per search of 0.43 pence per search.

Therefore, each search on NHS Evidence costs 202 times more than on TRIP.

Why does this bother me? A number of reasons:

The NHS is facing massive challenges to the budget yet NHS Evidence appears immune.
NHS Evidence distorts the market and TRIP suffers.
NHS Evidence does not appears to be engaging with librarians (that's certainly the feel I get from conversations and emails). The recent decision to give the specialist collections to non-librarian consortia reinforces this feeling.

But the main one is that I don't feel they offer a superior service to TRIP. I would go as far as to say that I feel TRIP is significantly better at supporting real, frontline clinical staff. However, I still stand by my main conclusion of last year that search isn't the answer to properly supporting clinicians to practice evidence-based healthcare.

With regard to these latter points I'd love to have the funds to test this. NHS Evidence got any spare change?

UPDATE: We've had an email from someone who used to work on the predecessor of the NHS Evidence - the National Library for Health (NLH). They point out that the old budget for the NLH was approximately £9,000,000. The only significant difference between the two (in relation to funding content (is the £5,000,000 paid to the BNF. So, comparing the old NLH and the new NHS Evidence there is a difference in funding level of around £10,000,000. They would like to know what does that extra money get you?

I've no idea, perhaps NHS Evidence can tell us?

UPDATE 2: I tweeted about this post and subsequently saw this tweet from Ben Goldacre (@bengoldacre) he of Bad Science fame:

A search on NHS Ev costs 200X one on TRIP: I find TRIP better

The latest evidence in TRIP

2011-01-08T15:07:00.000Z

Below is a copy of an email I sent to one of the design teams we use, I'm hoping it's self-explanatory. Feel free to comment on the idea and possible approach.

In TRIP we add around 4-5,000 new articles per month and currently this gets added to the index. In a way this hides it. I feel we could make it much more visible and accessible.

I'm increasingly aware that to search you need to know what you're looking for ie a known unknown. I want to create a space where people can browse for content without really knowing what they're looking for, so more like unknown unknowns.

A few points:

I'd like users to be able to select the time period (from 1 month to 12 months) perhaps some sort of visual system e.g. slider.
I'd like them to be able to select the grade of evidence (we have the slider in the results filter, which could be used/aopted).
I'd like to create some specialist areas e.g. view all the latest cardiology articles.
But I'd also like them to be able to search e.g. show latest articles on cholesterol
But I'd also like them to start with ALL latest evidence and burrow.

I'd hate for it to be a simple list - so boring. I want it to be immersive, I want them to enjoy exploring the data. I'd like it to be visual. I quite like the idea of using a word cloud to help navigate. But is that good? Old fashioned?

I'm still at very early stages of my thoughts and am quite keen to have some input from a design perspective.

Do you fancy doing a little bit of work on this? I'm not expecting proper designs, perhaps some outlines? I'm not saying this should be design led but I think it could help guide my thoughts on how/if it develops.

Welcome to 2011 and a request for new content ideas!

2011-01-02T15:25:00.000Z

I hope everyone had a good new year.

In 2010 we added a number of blogs to TRIP, for instance:

We're fairly selective about which blogs to include and have identified a few more to add in the near future. However, we're hoping to hear from users of TRIP with suggestions of any blogs to include. A few - loose - criteria:

High quality
Content is clinically focused and clinically usable
Aimed at health professionals

Let me know if you can think of any.

Also, we're always looking out for high-quality content. So, any non-blog sites you find useful, that fit the criteria above, then feel free to let me know.

TRIP in 2010

2010-12-27T07:11:00.000Z

It's a bit of a tradition to do some sort of review of the year. This year I'm combining this with my love of the data visualisation site Wordle. Wordle takes blocks of text and displays them in a manner that allows you to instantly see which words are used more frequently that others. The more often a word is mentioned the bigger the word is displayed. I'm hoping this makes sense!

For the review of 2010, I've taken the titles of all the documents manually uploaded to TRIP during the year and given it to Wordle to display, the results are below. The graph should expand when it's clicked, but if not the full word cloud can be viewed here.

RIS and Background Knowledge Boxes

2010-12-15T06:25:00.000Z

RIS is the file format for reference management software and we've been asked on a number of occasions if TRIP can support this format. In other words, allow users to select search results in TRIP and export them - via RIS - into their reference management software package. We're delighted to say this has now been released and users can select it via the drop-down menu at the top of the search results (see image below - click to make bigger). NOTE: TRIP does not have access to all meta-data so the export will be for partial results only.

Another bit of work we've been doing is looking to 2011 (see previous post). I'm very keen to redesign the results page to make the results more useful. One feature that we've spent a lot of time creating has been the background knowledge boxes (these appear at the top of many of the search results that link to eTextbooks linked to the search terms - see image above). However, we've started to look at how popular this feature is and yesterday it was used 30 times. This seems relatively little, so perhaps we need to consider if such a modestly useful feature deserves such prominence.

It's always the battle, balancing the demands on precious screen space between results and 'added value' features.

Feel free to comment on any of the above topics...

The next upgrade to TRIP

2010-12-11T08:15:00.000Z

Now TILT is out and seeming to be operating well it's time to turn our attention to the next upgrade to TRIP. So far we've got a number of suggestions, including:

Better support for languages other than English.
RIS export (we actually hope to get this out before the main update).
Linking in to content on the BNF.
Better differentiation of content types (e.g. evidence, images, patient information).
Introduction of new content types.
Integration with TILT.

There are others but the above are some of the the main ones.

I'm guessing that if you're reading this then you're likely to be a regular user of TRIP. How can we make it better? What new features would you like to see? Any gripes that annoy you about TRIP?

Analysing TRIP searches: COPD

2010-12-02T15:37:00.001Z

We've been doing some analysis looking at what users are looking for when they search TRIP. Below are two visual representations of this, using COPD as an example.

The top image shows all the additional search terms used when people have searched for COPD. In other words, if someone searched for 'COPD and exacerbations' we have analysed the auxiliary terms, in this example it's exacerbations. NOTE: Click on image to make bigger.

This second image uses a similar process to highlight the documents users actually went on to visit.

Help support TRIP

2010-12-01T15:37:00.002Z

I'm guessing, if you're reading this that you like TRIP and feel what we do is important.

If that is correct can you please read our request for donations.

Thank you in advance.

jon

Adding a TILT button to other sites

2010-11-24T12:38:00.000Z

You can see in the top right of this blog a TILT button, if you press it it opens up TILT with the URL embedded - making it very easy to TILT about a particular page. We've added it to the actual blog, but there's no reason it couldn't be added against every article that appears on a website.

Either way, it's very easy! Simply paste the following code into your site and you're TILT enabled...

<script type="text/javascript" src="http://tilt.tripdatabase.com/scripts/refer.js"></script>

The impact of TRIP: Part 2

2010-11-23T09:57:00.000Z

Earlier this month we posted results of our survey of users which revealed that 40.77% of searches improve patient care. For the full methodology read the post - click here.

Since it's start in 1997/8 TRIP has been searched over 51 million times.

So, 40.77% of 51 million = 20,793,000 times TRIP has helped improve patient care.

Is the answer 20,793,000? Probably not!

But there are good reasons why this figure is too high and reasons why it is too low!

Reasons why the figure is too high

Methodology weak: relatively small sample size, self-selecting participants (?more likely to be favourable to TRIP), non-validated questionnaire.
TRIP has improved. The figure of 40.77% assumes that TRIP has always been as good as it is now while it's likely to have been relatively poor to start with. However, to be fair to TRIP it has been consistently good for a number of years and the bulk of our searches have been in these latter years.

Reasons why the figure is too low

TRIP has actually been searched more than 51 million times. The 51 million refers to people coming to the http://www.tripdatabase.com/ site. We allow 3rd party sites to search TRIP and return results in their sites. We have a number of these for examples in clinical portal sites, EMRs etc. Also, we do various other automated information support systems (such as displaying related articles against 3rd party clinical articles).
It assumes that the information gained via TRIP is only used once. If each clinician searches TRIP based on a patient-query I imagine the knowledge gained is used in dealing with subsequent patients.

Irrespective of an accurate figure, I doubt anyone can question that TRIP has had a significant impact!

All slideshare for TILT in one place

2010-11-20T09:28:00.000Z

For my ease I'm placing all the TILT slideshares in one place:

Another slideshow - on appraisal

2010-11-19T13:46:00.000Z

TILT: Use for appraisal

View more presentations from JonBrassey.

TILT: How to make it a social experience

2010-11-19T09:46:00.002Z

Making TILT social

View more presentations from JonBrassey.

A basic introduction to TILT

2010-11-18T19:30:00.000Z

Does this work ok?

Your first TILT

View more presentations from JonBrassey.

TILT is out there

2010-11-17T07:34:00.000Z

I've been mentioning TILT in this blog over the last few months. It was actually live last week, but we started publicising it yesterday. You can see it at http://tilt.tripdatabase.com. If you're already registered with TRIP then you can log into TILT using your TRIP details.

What is TILT? At the simplest level it's a way for an individual to record their clinical learning (we encourage simple snippets of information). However, the real beauty is that this learning is shared within the community. There are lots of nuances associated with TILT, too many to post here now (while I'm still pushing hard to get TILT 'out there') but I'm sure I'll add more thoughts over the coming months and hopefully years.

Go to TILT, interact and let me know what you think.

The impact of TRIP

2010-11-12T06:15:00.000Z

Earlier this week we created a short survey to try and understand what impact TRIP might have on patient care. There were 4 questions:

On average, how often do you search TRIP each month?
Do you ever use TRIP for helping you manage your patients or yourself?
On average, what percentage of your searches are related to patient care?
Of those searches that you use for improving patient care or your own care what percentage do you find TRIP actually helps?

We linked to the survey from two places: As a link from the TRIP Database and also from our various social media 'outlets' - Facebook, Twitter and this Blog.

The results were very similar across both data collection methods. For the sake of clarity I'll show an average result and in brackets place the separate results from TRIP and then from the social media:

14.4 searches (17.2, 11.5)
79.1% (74.2, 83.9)
72.9% (74.3, 71.5)
70.7% (72.3, 69.0)

So, what do all these figures mean?

Firstly, the methodology is far from ideal, so the results are speculative/dubious. However, I feel reassured that the figures have been relatively stable from very early on in the data collection.

If we wanted to work out the impact of say 1,000 searches here's how I've approached it:

1000 searches of which 79.1% are from users who manage patients or are patients themselves = 791 searches
Of these 791 searches, 72.9% relate to patient care = 576.6
Of these 576.6 searches, 70.7% help improve patient care = 407.7

In other words 40.8% of searches on TRIP result in improved patient care. That's significantly higher than our estimates and also, what I think is equally significant, is that 70.7% of users actually find TRIP is useful for patient care.

What is clearly needed is some more research (I typically hate seeing that in the discussion of a research article) to try and get a more accurate figure for TRIP's impact and I'd welcome any comments on the most appropriate methodology.

But, in the interim, I'm happy (with the above reservations) to say that 40.8% of searches on TRIP help improve patient care

The impact of TRIP

2010-11-09T16:07:00.000Z

I'm really trying to understand the impact of the TRIP Database. If you care about TRIP please take this very brief survey http://www.surveymonkey.com/s/JJKH25W

Thank you in advance.

Reflections on Evidence 2010

2010-11-04T07:32:00.000Z

At Evidence2010 I was a aware of a number of themes (in reverse order):

The whole publishing model is wrong.
Clinicians needs support in using the evidence ie what to do once they read a trial and know an intervention works, how do they actually use or administer it!
Conflicts of interest (COI) are widespread. Pharma is an obvious COI but also in academia there are significant issues.
There's no more money - the healthcare systems in 'the West' have no more money to keep pumping into healthcare.
Need to work more closely with patients to help make them better decision makers.
Clinicians need to say this phrase more often - I don't know.
There's an awful lot of love for TRIP :-)

Obviously, I'm very pleased about the last one. Overall, a very good, thought-provoking conference!

Top 30 consultations in primary care

2010-11-03T06:25:00.000Z

I had the pleasure of attending the Evidence2010 conference. One role I undertook was to tweet about the presentations, in a way sending a summary of the presentations. One talk, by Paul Glasziou, highlighted the diversity of conditions a GP will see (compared with specialists). He reported that 30 'conditions' accounted for 50% of consultations. A number of people wanted to see the 30 conditions and Paul has sent me a spreadsheet. It's actually 32 (not 30), so here goes:

Hypertension*
Upper_respiratory_tract_infection
Arthritis—all*
Diabetes,_non-gestational*
Depression*
Lipid_disorders
Osteoarthritis*
Back_complaint*
Immunisation—respiratory
General_check-up*
Asthma
Oesophageal_disease
Acute_bronchitis/bronchiolitis
General_immunisation
Contact_dermatitis
Anxiety*
Gastroenteritis*
Female_check/papsmear*
Sleep_disturbance
Urinary_tract_infection*
Sprain/strain*
Medication/script
Sinusitis
Solar_keratosis/sunburn
Cardiac_check-up*
Ischaemic_heart_disease*
Oral_contraception*
Pregnancy*
Malignant_neoplasm_skin
Acute_otitis_media/myringitis
Results_tests/procedures_NOS
Viral_disease,_other/NOS

Interestingly (!) the top 10 account for 27%

Facebook update

2010-10-31T06:14:00.000Z

I highlighted a while ago that I often add small comments to Facebook that I didn't consider worthy of a blog post. So, continuing the theme started then, a few highlights of comments I've made on Facebook.

I've updated all the Royal College guidelines - nearly 1,000 documents.
Asked about how to raise money to help to keep TRIP going and improving!
Highlighted a very nice net website - theNNT.
Exposed the new TILT logo to the world (see below)
Asked if people like the Q&A site Gas Exchange.
Announced that we've as good as secured funding for an African clinical search tool.
Not TRIP related, but what I do part-time for the NHS, is the Q&A service called ATTRACT. One feature we've just released in called tripe.
Finally, we're starting to look to improve our links to educational content and we're starting that process!

As mentioned above - the TILT logo!

Bing likes Facebook

2010-10-13T20:21:00.000Z

Facebook is massive, Microsoft is massive, Bing (Microsoft's search engine) is less than massive. Bing want to change this and they're hoping that a tie-in with Facebook will help. See this Techcrunch article for further details. As the article states:

"Starting today.... you will automatically start to see links that your friends have “liked.” These will appear in a separate module, with related social links called out. The example Microsoft gives is if you are searching for San Francisco steak houses and one of your friends liked Alexander’s Steakhouse in San Francisco, that would appear as a result along with the name of your friend."

I like this idea a lot. It's dependent on a number of variables coming together, but at a high-level I like this.

I particularly like it as I can see something similar working in the world of health. What's better for a GP searching for an article on the diagnosis of prostate cancer than one that a trusted colleague has approved? How about, using the extended social graph to identify experts in the field of prostate cancer - what do they like? Finally, what does the evidence have to say (as shown in the current TRIP search)?

However, this also requires a number of variables coming together. However, if Facebook/Bing paves the way, then why not? An important component for this to work is a clinician's functioning social graph - where can that comes from?

Let's hope TILT works!!

TILT: you'll be amazed at what you don't know

2010-10-12T06:51:00.014Z

The launch of TILT (which stands for Today I Learnt That) is fast approaching and we're pulling together the design and the actual programming - so time for a bit more information. First the logo, which we love!

At the simplest level TILT is an easy way for users to record any learning they may have experienced. This was designed principally around clinicians who need to record this sort of information for revalidation/recertification. Although, from our pilot, it became obvious that clinicians like to record their learning even if it's not specifically for revalidation.

However, the real beauty and strength of TILT is that the 'TILTs' are viewable by other clinicians. As in 'real' life clinicians can learn from each other.

Some example TILTs include:

PPIs associated with 1) fractures 2) LRTIs 3) C diff
There's not enough evidence to recommend racecadotril in children with acute diarrhoea
I now (belatedly) appreciate the differences between the old UK90 growth charts and the new WHO charts (and the rationale behind the switch)

The clinicians in the pilot really enjoyed themselves and reported lots and lots of learning. For me, the biggest joy was seeing the way clinicians transformed complex issues/articles into nuggets of learning. From an information perspective this is gold dust.

The pilot involved around 10 active users and generated hundred's of TILTs in 6 weeks. If we have 100 active clinicians what sort of impact will that have? What about 1,000?

As one clinician fed back "you'll be amazed at what you don't know"

10,000 registered users, official and proper!

2010-10-05T16:31:00.000Z

After a few false alarms we finally got the 10,000 registered users. Now these figures have been audited - I'm very very happy with this figure. I've done some analysis and the following are the top 25 countries where registrants come from:

UNITED KINGDOM
UNITED STATES
SPAIN
AUSTRALIA
CANADA
MEXICO
CHILE
ITALY
BRAZIL
PERU
INDIA
IRELAND
SAUDI ARABIA
ARGENTINA
COLOMBIA
NETHERLANDS
NEW ZEALAND
IRAN (ISLAMIC REPUBLIC OF)
EGYPT
INDONESIA
ISRAEL
BELGIUM
THAILAND
CHINA
PANAMA

TRIP truly is international! And here's a selection of countries where we only have a single registrant:

AMERICAN SAMOA
ANGOLA
ANTARCTICA
BURKINA FASO
CONGO
EAST TIMOR
ERITREA
GUADELOUPE
KOREA DEMOCRATIC PEOPLE'S REPUBLIC OF
LAO PEOPLE'S DEMOCRATIC REPUBLIC
LUXEMBOURG
MONACO
MYANMAR
NIEU
PITCAIRN
SAINT KITTS AND NEVIS
ST. HELENA
SURINAME
SVALBARD AND JAN MAYEN ISLANDS
TAJIKISTAN

I love running TRIP and this sort of milestone and analysis make me very proud. I think the next milestone will come relatively quickly via Facebook, where we have 494 'fans' - only 6 to go!

Oops, we made a mistake

2010-09-30T07:57:00.000Z

On August 4th we announced that we had 10,000 registered users. After some internal checks we now realise this was wrong.

The reason for the discrepancy was a number of spam accounts. We became aware of these about 6 weeks ago (and introduced a CAPTCHA system to prevent future spam accounts) and have since been tracking down these accounts. We now feel we have a pretty accurate figure for actual accounts and as of now, we have 9,873 registered users.

Keeping TRIP Going!

2010-09-28T08:39:00.000Z

As you know money is always in short supply at TRIP. Our ideas always outstrip our income!
We've discussed asking for donations in the past and we're still tempted. However, might asking people to sponser/adopt a search term be better? For instance, someone could sponsor 'diabetes' and everytime there is a search for diabetes a message appears somewhere on the results page 'This search has been sponsored by..............'.

What do people think? Alternatively, has anyone else got any suggestions to increase our income?

Remember, TRIP is free to access, has made a massive contribution to healthcare (over 50 million searches) and has no significant backers. All the income we get has to come through small, separate contracts where we can get them.

We'd like to think the many users of TRIP might like to help support us.....

Any specific ideas can be added via comments or emailed directly to me at jon.brassey@tripdatabase.com

Facebook, blogger and twitter

2010-09-24T09:33:00.000Z

The above applications are widespread and TRIP has a presence on them all! They all offer a way of TRIP to reach out to users and those with a shared interest. It's the blog that gives me the greatest worry, as I feel I should blog more regularly. There's no external pressure, I just feel I should blog once a week.

However, there lies the problem - I see blogs, Facebook and twitter in a different light. Our Facebook presence gets the most attention and I think that's due to the size of the posts (so I post more) and the possibly more interactive audience on Facebook (where it's very easy to interact - just press a button).

I see the blog as a place for publishing longer pieces, twitter for very small and Facebook in between. This medium size suits me as I'm not a big writer, if I can say something in 50 words I'll do so, as opposed to some who'd prefer to use 500.

But, perhaps I need to precis what I've written on Facebook, here in the blog, so the blog audience doesn't 'miss out'.

So, since the last blog article I've 'Facebooked':

I went and gave a hands on workshop at a CEBM training conference. Once nice bit of feedback, from a clinician was - 'before I found TRIP I was ploughing the field with a trowel, now I'm using a tractor'.
Reported on a meeting with the lovely Glycosmedia.
Revealed that we've now got 3,172 medical videos in TRIP.
It looks like we're very close to secure an African version of TRIP, which I'm very excited about and follows on from our crowdsourcing of evidence for the developing world.
We had a little twitter experiment with the European Respiratory Society.
Highlighted that a training course was being undertaken in Chile as we had 20 registrations from Chile in the space of ten minutes.
Finally, reported on a 'spring clean' of all the UK Royal College's clinical guidelines to fix broken URLs and to ensure the content was actually fresh.

Having summarised the Facebook activity I think it works quite well! It's good to see a summary of our activity. Now, I must rush - I've got a meeting with our techie (Phil) and designers about our new project - TILT!

Visibility of features on TRIP

2010-09-12T06:57:00.001Z

I was recently invited to attend a workshop held by the Centre for Evidence-Based Medicine in Oxford. This wasn't a formal demonstration of TRIP, it was more an informal chat to allow people to ask about TRIP various aspects of TRIP e.g.

How often is the site updated? Answer: most content is weekly or fortnightly and the rest on a monthly basis.
What is involved in updating the site? Answer: Most content is automatically added via various clever bits of technology. However, for a number of sites I still have to visit them monthly and manually add the content.
How does the boolean work? Answer: Use the advanced search to show you - as this builds the boolean for you.

Then, a person who had used TRIP for years said they weren't aware of this feature. Over the course of the session people highlighted they hadn't seen:

Advanced search
PICO search
Free-text search
Sources searched by TRIP
Synonyms - the list of synonyms used for a search are displayed at the foot of the results page (see below)

It's an interesting issue/problem. It's trying to get the balance right between making features visible and not over-powering. I'm really not sure how to resolve this! Possibly create a monthly 'Feature' area which would allow me the ability to focus on a given feature. One to ponder!

One very memorable quote was from a clinician who said:

'before I found TRIP I was ploughing the field with a trowel, now I'm using a tractor'

NHS Evidence and non 'evidence based' claims!

2010-09-06T16:58:00.000Z

I've not mentioned NHS Evidence much and it's a coincidence that I mention them twice in 6 weeks. I last blogged about them then (click here) to highlight that their searches are massively more expensive than TRIP's.

My interest in NHS Evidence goes back longer than 6 weeks! I have contacts (and had contracts) with the predecessor to NHS Evidence (National Library for Health) and was keen to see how NHS Evidence would change things. Unfortunately, I was disappointed by their lack of ambition and innovation. Although, I'm still hoping to be proved wrong and they'll do something special - clinicians (and patients) deserve it (especially with their budget of nearly £25 million).

But, the reason for this post, relates to some advertising NHS Evidence produced earlier this year. I saw it in the middle of March in the Health Service Journal. In that advert they claimed two things which I felt were inappropriate:

The most dependable clinical and non-clinical information online
answer all your clinical and non-clinical questions

It makes me shudder that this got past any internal processes. As a gateway for evidence surely they should be able to substantiate any claims they make. Both claims, to me, were absurd. I could go on, at length, as to why the claims are so foolish and if people are unclear then please contact me separately (jon.brassey@tripdatabase.com) and I’ll explain.

After seeing these adverts I felt quite strongly that NHS Evidence shouldn’t make such statements and therefore complained to the Advertising Standards Authority. I recently received a letter telling me that NHS Evidence has confirmed that the advert has been withdrawn and that they will not repeat the claims.

It’s a small victory.

But here’s a challenge to NHS Evidence – fund some comparative research and compare yourself to other products. Not just TRIP, but DynaMed, NHS Scotland’s rather nice Knowledge Network and others. Help everyone learn from the strengths and weaknesses of others. If it helps we’ll part-fund it (we’ll give 0.5% of our budget if you match that with 0.5% of your budget).

As it happens in another freedom of information request NHS Evidence (where I asked about any comparisons they make with other search engines) they reported:

"we don’t compare our performance against other services”

This last quote is quite simply scandalous.

We're still here!

2010-09-05T17:15:00.000Z

I sometimes feel there is a pressure to blog, to write something, just to show people we're still here and thinking of them! While things may appear quiet (we've not blogged, twittered or Facebooked much) we're still very busy. However, much of the work is behind the scenes as we gear up the development of our TILT project. I've been busy finalising the specifications and have as good as signed off on these. TILT is pretty much in stealth mode so what can I say....

It's a way to record and share learning
While search focuses on known unknowns (you need to know you don’t know something before searching for it) TILT will focus as much on unknown unknowns as known unknowns!
When I talk about search, in TILT, I'm not referring to document search - something far more useful and concise.
At it's core is clinical usefulness, it's a bottom-up approach.

Well, I hope that's whetted people's appetites.

It'll launch, probably as a beta towards the end of October.

Changing tack, if you're on Facebook please visit our page (TRIP on Facebook) and 'Like' us :-)

Monthly update

2010-08-20T12:39:00.000Z

Another month and another manual upload of 600 plus records. TRIP actually imports more records than that, the majority come via automatic methods.

In addition to the monthly upload we've completely revamped (due to URL changes) the following sites:

Diabetes UK
British Thoracic Society
British Association of Dermatology
British Society for Rheumatology
European Association of Urology
Washington University Division of Emergency Medicine Journal Club

A busy month, but well worth the effort.

Ten thousand registered users

2010-08-04T06:17:00.001Z

I'm very pleased to announce that TRIP now has over 10,000 registered users.

The vast majority of these users are health professionals enrol and there's pretty much an even split between doctors, information specialists and other health professionals. The users are from all around the globe, highlighting TRIP's global appeal and reach.

To celebrate we'll be rolling out a significantly enhanced e-mail alert system. Previously, if a user had recorded, say, 'prostate cancer' as an area of interest they would simply get an e-mail reporting how many new articles matched this keyword. Following feedback we have improved this so the results are broken down into EB Synopses, Systematic Reviews, Guidelines (pertinent to your country), Core Primary Research and Extended Primary Research (see below)

We'll be sending out the latest evidence upgrade in the next day or so.

Developing world initiative

2010-08-03T12:43:00.000Z

The number of articles suitable for the developing world (click here for further information) continues to grow and appears to be speeding up. As of today we have identified 288 articles. It's not a massive number, but a review of these reveals a high level of accuracy - so I'm happy that things are moving in the right direction.

We never envisaged this initiative being 'the' answer, but part of something bigger. In relation to that we have a number of ongoing discussions and if/when these come to fruition we'll announce them here.

The cost of search

2010-07-30T14:34:00.001Z

NHS Evidence was created to replace the NHS's National Library for Health. I was disappointed when it was released as it appeared little more than a clone of TRIP. I was hoping for something more innovative to help support clinicians better and also to push the search agenda along - there is nothing like robust competition to stimulate innovation.

I have undertaken a number of Freedom of Information requests and have found out some interesting facts (click here to see the requests and responses).

What I found is as follows:

The budget for NHS Evidence in 2010/11 is £24,438,000
The number of searches this year is 15,811,716 (based on the average monthly figures for February to April 2010 multiplied by 12)
This means a cost per search of 154p

By comparison, this is how TRIP stacks up:

The budget for 2010/11 (for our search engine) - £45,000
The number of searches this year is 8,058,648 (worked out using the same figures and method as above.
This means a cost per search of 0.56p

Therefore, each search on NHS Evidence is 276 times more expensive than TRIP!

Why I raise this is not to rubbish NHS Evidence, although the figures are unflattering. It's more a feeling that it highlights the limitations of search. Is this the law of diminishing returns? Is search really the answer to clinical uncertainty? I'm increasingly convinced that search is not the solution, irrespective of how much money you throw at the problem.

I'd love to have any comment on the above, are my conclusions sound? Do the figures add up?

Clinical questions and StackOverflow

2010-07-23T09:34:00.000Z

Answering clinical questions is at the heart of the TRIP Database. We've been answering clinical questions for over 10 years and answered well over 10,000. Most of these questions have been answered using a standard methodology:

Receive and clarify the question
Conduct a search
Read and appraise the articles
Write and answer
Post answer on the website

This can frequently be time-consuming and therefore relatively expensive. Search systems, even the TRIP Database, doesn't answer questions - it returns articles that may help the clinician answer their question.

There are alternate models of answering questions and from the research evidence the most consistent source of answers for clinicians is their own colleagues. But, there are half-way houses and my current favourite is facilitated by a service called StackOverflow which is a Q&A service for those involved in IT. The company behind StackOverflow have recently been increasing the availability of the underlying software to power other Q&A systems and this was featured, today, on the ReadWriteWeb blog.

One of the first non-IT uses for StackOverflow that I was aware of was GasExchange, a Q&A service for anaesthesia (check it out, it's great). It's been running for many months and has an active community. I'd love to see this sort of thing happen for other specialities - including primary care.

An alternate results page

2010-07-16T13:21:00.000Z

I spend a lot of time pondering how best to serve up the results on TRIP. Currently all results are merged with the ability to filter the results by category (systematic reviews, guidelines etc).

But, there are clearly different types of content:

Research evidence - systematic reviews, synopses, guidelines, primary research etc.
Background knowledge - eTextbooks.
News
Patient information

The mock-up below (click to enlarge) shows a way of making this distinction clearer. Let me know what you think!

Health in the developing world

2010-07-09T13:25:00.000Z

TRIP has become increasingly interested in improving access to the best available evidence for the developing world (see our crowdsourcing initiative). In launching the initiative I've managed to meet a number of interesting people.

Yesterday I had the pleasure of meeting the editor of the Africa-Health journal who's knowledge and experience of the issues facing health professionals in the developing world was impressive. We talked about a number of potential new projects but as ever funding is an issue.

I recommend readers of this blog, especially those in the so-called 'developed world' to spend some time reading the journal - it highlights the issues facing health professionals in these areas and gives a sobering perspective.

Excelencia clinica

2010-07-01T12:41:00.000Z

Excelencia clinica (ec.net) was launched in December 2008 as a collaboration between TRIP, the Spanish Ministry of Health and the Iberoamerican Cochrane Center. ec.net was a bilingual (Spanish:English) clinical search engine modelled on the TRIP Database.

Our contract to work on ec.net finished yesterday and the work is now being carried out in-house. It has been a great experience working with the Iberoamerican Cochrane Center (our main point of contact) and it has been a pleasure seeing the site grow and to develop.

We wish them well for the future and hope the growth of ec.net continues.

Medical Images on TRIP

2010-06-21T12:10:00.004Z

Our latest medical image update has occured and we've now got 113,876 images in our index.

For those of you who've not used our medical image search before we return the four top medical images for a given search term on the main results page. We then allow users to 'See more images' and as you'll see from the image below we automatically import Google images to enhance our own offering. Altogether our medical images are starting to look pretty solid.

New server

2010-06-14T18:41:00.002Z

Wow, I'm impressed!

Last week I posted that we're moving servers and TRIP is now on there. For a few days it'll redirect to www.tripdatabase.co.uk but that's only while we muck around with some DNS issues.

Why am I impressed?

It is super, super quick - amazingly so.

I'm very, very happy.

Moving servers

2010-06-11T12:24:00.003Z

Just a quick note that we're in the process of moving servers. We've been using the same servers for a number of years and one in particular is getting ancient and needs replacing. So, instead of buying new servers we've decided to move to the industry leader - Rackspace.

Our sites will be sitting on a brand new, high specification server. We'll also get better backup and support and most importantly we won't get charged extra if we're well used (we currently get penalised if we exceed a particular bandwidth figure).

The transfer is currently happening with TRIP Answers sitting on the new server and the main TRIP Database should follow as soon as the whole database (90 Gb) has transferred over. Once on the new servers we'll need to alter something called the DNS (the internet's address book) and then everything will be fine and much faster. In the interim you'll note that certain addresses re-direct to temporary addresses on the new server (for instance www.tripanswers.org points to http://tripanswers.tripdatabase.co.uk/). This should be no reason for concern.

TRIP in the future

2010-06-09T06:59:00.002Z

It's been a while since I last posted, it's been a busy reflective time at TRIP. We've been preoccupied with our new project called TILT as well as looking at our business model. At the same time we're questioning the 'worth' of search (which overlaps with our TILT work).

Looking at the business model is difficult at the best of times and this is especially difficult when the people at TRIP are not natural business people. We're good at innovating and developing products, but turning these into commercial products has always proved difficult. At the heart of this is how to make 'search' profitable. In the mainstream world of Google there is the advertising model which works wonderfully for them. However, our efforts at TRIP to use such a model have never really delivered, even with our hundreds of thousands of searches per month.

We've been in a few discussions with bigger companies recently (and these are ongoing) looking at working more closely with them to benefit from the better business brains. It's very early stages but we're hoping that more formal arrangements will be forthcoming, giving TRIP a firm foundation to carry on and develop. One of these developments is to move on from the current state of search.

I've posted a few times on the limitations of search but will briefly recap here. Imagine if search engines had not been developed and you got 10,000 clinicians in a room and asked them what product would they like to see if they had a clinical question. I would be surprised if a single one would say they wanted to see a screen of 10-20 results which may contain the answer to their question - based on searching with 1-3 search terms related to their clinical question. Yet, this is the current situation. While TRIP is a great search engine, search is limited.

The next stage of TRIP is to try and move on from the above paradigm and TILT is our approach. Without giving too much away, TILT seeks to extract clinically useful content from a wide range of sources. So, you may have a research article which contains multiple nuggets of evidence/knowledge. Why leave them hidden in the document? So, much of TILT is about deconstructing evidence into constituent parts. However, there is another very important layer above that. The layer is how to decide what nuggets are clinicially useful/relevant.

We're mid-way through piloting our approach and we're very close to being able to deliver on the above. How our approach is initially received and adopted will decide it's success and having decent partners to help market and approach TILT will be invaluable. We're also looking to content providers to work with us and we're currently in discussions with a few groups we'd like to have on-board at launch.

After over ten years working in search and information support for clinicians I'm getting really excited as TILT has the potential to move significantly closer to where we need to be in helping clinicians practice better healthcare.

An evaluation of TRIP (sort of)

2010-05-26T09:50:00.002Z

I've just seen this rather nice paper about TRIP. The article is from an Italian publication, so all I can go on is the abstract. As such I'm not sure how they searched TRIP and what other databases were involved. Irrespective of that it's quite pleasing!

Bibliographic research of efficiency tests: analysis of the validity of the meta-database
Galbiati G, Casati M.
Prof Inferm. 2010 Jan-Mar;63(1):3-8.

Aims: The use of a meta-database as a first approach to bibliographic research can be just as efficient asinterrogating single data-bases of the litature. The advantages and drawbacks of the two strategies are compared .
Method: A comparison of the results obtained using an identical interrogation made using the TRIP meta-database and different single databases (15 of guide-lines, 4 of systematic reviews, 3 prevalently consisting of primary studies) made it possible to analyse these methods as well as to study 4 meta-databases and identify the most efficient one.
Results: Using the same MeSH terms in both strategies, the following results were obtained: 204 publicationsusing TRIP and 475 using different databases. Evaluation demonstrated the pertinence of 142 (69,6%) of the 204 found using TRIP compared to 185 (38,9%) of those elicited by single data-bases.
Conclusions: The TRIP meta-database yields a lower number of documents but with a higher degree of pertinency, meaning that the researcher employs less time finding pertinent documents. With respect to the traditional approach, beginning research by testing the efficiency of the TRIP meta-database proved advantageous.

Facebook Questions

2010-05-24T04:48:00.002Z

I have a real passion for getting answers to people.

I have a passion for search, but search is of limited value in answering questions. The typical search engine returns 10-20 results which the algorithm believes will answer your question(s). Imagine if search engines didn't exist and you got 100 doctors in a room and asked them what features would they like to see in a system to answer their questions. Do you think any would say 'give me 10-20 results to articles which may answer my question' - but here we are.

Anyway, all that preamble is due to news that Facebook is working on Facebook Questions, see this TechCrunch article for more info - this'll be interesting.

Monthly update

2010-05-16T08:42:00.002Z

I've just finished the monthly update to TRIP with 668 new articles added manually and 2,500+ automatically.

When reviewing the manual uploads I often come across themes and this month the stand out, for me, was wound care. We've answered lots of questions on wounds over the years and the lack of evidence if obvious. So, three new reviews are very welcome:

1) Water for wound cleansing (Cochrane)

Conclusion: There is no evidence that using tap water to cleanse acute wounds in adults increases infection and some evidence that it reduces it. However there is not strong evidence that cleansing wounds per se increases healing or reduces infection. In the absence of potable tap water, boiled and cooled water as well as distilled water can be used as wound cleansing agents.

2) Wound cleansing for pressure ulcers (Cochrane)

Conclusion: We identified three small studies addressing cleansing of pressure ulcers. One noted a statistically significant improvement in pressure ulcer healing for wounds cleansed with saline spray containing Aloe vera, silver chloride and decyl glucoside (Vulnopur) when compared with isotonic saline solution. Overall, there is no good trial evidence to support use of any particular wound cleansing solution or technique for pressure ulcers.

3) Silver-Releasing Dressings in Treating Chronic Wounds (Swedish Technology Council)

Main conclusion: The scientific evidence is insufficient to determine whether silver dressings differ from dressings without silver in terms of effects on the percentage of healed wounds, wound size, pain, quality of life, percentage of infections, and use of antibiotics in treating chronic wounds. The reason is that too few studies of sufficient quality are available. The studies reviewed have not identified serious side effects or complications related to silver dressings, but they were not designed to study this specifically.

TILT

2010-05-06T16:44:00.002Z

A further update on TILT - it's generating some very interesting outputs. For those of you unsure of what the TILT project is - it's an area allowing clinicians to record and share learning. Some examples of the learning we've received so far:

Ropinirole for Parkinson's disease can cause ankle swelling as a side effect
For every 1% increase in omega-3 intake, HDL levels rose by 2.5 mg/dL
DISH is Diffuse Idiopathic Skeletal Hyperostosis and is failry common in men and represents calcification of ligaments tendons etc principally around the Thoracic spine. It has a classical 'dripping wax' appearence on plain radiography. There may be an increased risk of diabetes and poss heart disease in patients who have DISH
Medial arch pain localised to bone rather than soft tissue may indicate an accessory navicular bone! usually unmasked after a twisting injury
Both antiseptic cream and silver nitrate cautery are likely to be effective in preventing recurrent idiopathic epistaxis in children Cautery is indicated when there are obvious telangiectatic blood vessels

Lots of feedback from users, many reporting very positive examples of learning from others and others impatient to see it being taken out of 'proof of concept' and some serious development undertaken.

So far very, very interesting

Update on TILT

2010-05-04T06:38:00.002Z

TILT (see previous blog) has been running for a little over a week and so far so good.

We have had around 50 recorded instances of learning and crucially people have learnt from others. In other words they have seen someone elses recorded learning and reported they have learnt from that. Therefore, it appears that:

People will record their own learning.
People will allow the sharing of this learning.
Other people can learn from other people's learning.

The pilot still has another 4(ish) weeks to run so we'll see how things develop. After that a decision will need to be made about if we adopt TILT or not. If we do we'll need to learn from those involved in the pilot. However, the biggest challenge will be scalability - if we have 10,000 doctors recording one item of learning per week that's 500,000+ items per year. So, the issue of findability is vital. Thankfully, we have a number of ideas to help deal with this - if we proceed.

It's not too late to volunteer to take part, let me know via jon.brassey@tripdatabase.com

Open Access Education Initiative

2010-04-26T06:11:00.002Z

TRIP's recent work around supporting access to the evidence for low-resource settings (click here) has exposed TRIP to all sorts of new contacts and some amazing people/resources that are already involved in this field. The Open Access Education Initiative (Peoples-uni) is a great example and was designed to help build Public Health capacity in developing countries.

It uses open-source materials and a dispersed group of volunteer course developers, tutors and infrastructure support, to provide low-cost education for capacity building in Public Health. Over the past two years, it has been able to deliver an educational programme to those who cannot afford overseas student fees charged by most universities. The Peoples-uni has developed a set of course modules, which can be taken individually and used to gain a Certificate, Diploma (and soon a Masters) in Public Health.

Visit the site today (Peoples-uni) and learn much more.

TILT

2010-04-23T06:24:00.002Z

stands for Today I Learnt That and is based on observations on how people have used the reflective elements of TRIP CPD. When you look at an article via the CPD/CME link (under each article) it opens the article with a reflective toolbar at the top of the page. This prompts the user to answer three questions:

Why are you looking at this article?
What did you learn from this article?
How will you apply this in practice?

What we observed was how powerful and useful the middle answer tended to be. This was typically a clinician giving their account of the learning they had undertaken from reading the article. It struck us how powerful this information would be if shared.

TILT allows a user to record any learning they have undertaken (not restricted to TRIP articles). It might be any article they've read, a conversation with a colleague - any clinical learning. This is then recorded as a learning log. But the real beauty is that this learning can be shared with other users. In other words, they can learn from your learning and vice versa. Already, after just 3 days and around 10 recorded 'learnings' two clinicians have already learnt from others contributions.

Currently, this is a proof of concept model and at the end of the testing period (4-6 weeks) we'll try and get a feel for the 'worth' of the model and make a decision on whether to take it forward or not.

If you'd like to get involved let us know via jon.brassey@tripdatabase.com

Liability and the health librarian

2010-04-22T07:34:00.001Z

While I'm not a librarian I'm clearly heavily involved in information. I'm also aware that a number of librarians read this blog, therefore I feel I should highlight the above article, written by Dean Giustini (click here to read the article).

Essential reading.

8,000 users

2010-04-18T05:25:00.001Z

18 days after hitting 7,000 users we make it 8,000, at this rate we should hit 10,000 sometime in June.

Aardvark

2010-04-17T05:59:00.004Z

I mentioned Aardvark in a previous post. Basically, it allows you to send questions to people who might know the answer. They seek to send questions to a users 'extended social network'. I'm not sure how this is defined, but it's probably based on the 'friend of a friend' principle. Google bought Aarvark a while ago (surely a vote of confidence) and have just started to roll it out on YouTube (click here for post).

I'd love this sort of service on TRIP and we could launch something similar. However, for this to work well we need to have a good idea where people's expertise lies. Currently, the only details we consistently have on users is what their broad area of interest are (e.g. cardiology, oncology). If there are 100 questions covering cardiology topics - what's the best way of getting them sent to people who may know the answer? If I had some expertise in, say, stable angina I would find it very off-putting to be asked to help answer questions on cholesterol, stroke, hypertension etc. What would be much better would be to send the user only questions relating to stable angina.

So, to do this requires some sort of profiling. In other words recording and trying to understand a user's habits on TRIP and making assumptions based on their search terms, click-throughs etc. And, if we had a decent profile, as well as forwarding appropriate questions to them (assuming they'd be predisposed to answering them) we could arguably push new research, conferences, even jobs of interest.

It's a big task to undertake and requires users to consistently login (to help assign behaviours to an individuals profile). However, I think it's well worth the effort.

TRIP on Facebook

2010-04-15T05:49:00.003Z

A quick update on the TRIP Database Facebook page - it's doing really well.

We've currently got 233 'fans' (a term I dislike in this context) and we're having conversations all the time. I suppose, as I've said before, it's another medium to reach out and 'meet' our users and to hear what they want.

If you're not signed up already - go and visit and become a 'fan' today.

Open access

2010-04-08T08:35:00.002Z

The juggernaut that is 'open access' continues to gain pace (or as they say in this Nature News article 'The push to open up scientific knowledge to all looks set to go into overdrive').

The article is a good introduction to the topic so I recommend you read it.

TRIP thrives on linking to high quality open access content but we need to embrace it even more. I can't help thinking that auto-searching PubMed Central (not forgetting the UK equivalent) would be a good 'next step' for us.

But can we help in other ways? If you have any ideas, let us know.

TRIP Answers

2010-04-02T08:16:00.002Z

I've just been looking back and TRIP Answers has only been around for 16 months (but still had well over 1 million page views), it seems like it has been with us for much longer. Perhaps 16 months is a good time for a 'makeover', which is what we've done with the site. The changes include:

Re-designed homepage to make it more dynamic.
Simplified the look and feel of each answer.
Introduced a 'related articles' panel which pulls back articles from the main TRIP Database which are similar to the actual Q&A. This is especially useful for older Q&As as it helps to show if there is any new research.
Significantly improved the search, it now uses the same system as found on the TRIP Database.

Lots of other smaller changes to improve the site.

Go take a look and let us know what you think.

Usability testing.

2010-03-31T13:47:00.002Z

When we relaunched TRIP last November it was with the help of Minervation, an Oxford-based company who specialises in 'evidence based' websites, knowledge management, information support and usability testing. They carried out some usability testing of our test site.

I blogged about how painful a process it was, but ultimately worthwhile.

They've just released a case-study of their work with the charity CancerHelp. Well worth a read!

7,000 users

2010-03-31T11:18:00.001Z

This morning we hit 7,000 registered users.

The TRIP and Low Resource Initiative: Update

2010-03-30T08:00:00.002Z

We launched the TRIP initiative to highlight content suitable for low-resource settings less than 3 weeks ago (click here).

I'm pleased to see that already 96 articles have been approved (clicked on twice!) to form a small sub-set of articles, this includes 20 systematic reviews and 6 guidelines. A sample of document is shown below.

Guidelines for the prevention of Malaria in South Africa. South African Department of Health
Intrarectal quinine versus intravenous or intramuscular quinine for treating Plasmodium falciparum malaria. Cochrane Database of Systematic Reviews
Cost-effectiveness analysis of antiretroviral drug treatment and HIV-1 vaccination in Thailand. NHS Economic Evaluation Database
Efficacy of iron-fortified whole maize flour on iron status of schoolchildren in Kenya: a randomised controlled trial. Lancet
The use of misoprostol in obstetrics and gynaecology. BJOG
Cost effectiveness of high-risk HPV DNA testing for cervical cancer screening in South Africa. Gynecologic Oncology

You never know when you launch an initiative if it'll suceed. It's far too early to say if our 'low resource' initiative will - but we couldn't really have expected a better start.

Videos for TRIP

2010-03-29T15:53:00.004Z

I came across a free screencast site called Jing which allows to make screencasts/videos.
I've been looking out for this sort of software for a while, so I was keen to give it a go and here is my first attempt.

I'd love to hear feedback!

TRIP Database on wikipedia

2010-03-26T14:01:00.001Z

There's a wikipedia article on the TRIP Database (click here).

If you're involved in wikipedia - help improve it.

TRIP and Facebook

2010-03-26T09:52:00.002Z

We've gone and set up a Facebook page for TRIP, click here to visit - and become a 'fan'!

It's still not clear the advantage of being on Facebook, but it is another medium to connect with people, which is probably reason enough. But, if people are more aware of the potential for TRIP on Facebook - please let us know.

Other Facebook pages include:

TRIP & OSCAR

2010-03-21T06:16:00.004Z

TRIP is interested in ensuring clinicians have easy access to the best available evidence. That's why we're thrilled to see TRIP available to the users of OSCAR Canada. As the wikipedia article states:

"OSCAR McMaster is a web-based electronic medical record (EMR) system initially developed for academic primary care clinics. It has grown into a comprehensive EMR and billing system used by many doctor's offices and private medical clinics in Canada and other parts of the world. The name is derived from where it was created and an acronym; OSCAR stands for Open Source Clinical Application and Resource and McMaster refers to McMaster University, where it was developed. It enables the delivery of evidence resources at the point of care."

I''m particularly thrilled to be on OSCAR as it's an open source project.

Below are some screenshots (click to make bigger) from the first round of integration, a few flaws to be ironed out, but nothing major.

It's not our first time on an electronic medical record (or sometimes called an electronic health record) - see our October 2009 post TRIP and Electronic Health Records (EHR). I love this integration with 'point of care', bringing the evidence closer to where care is delivered.

Using TRIP to help identify content suitable for resource poor settings

2010-03-10T07:45:00.004Z

We’re very pleased to announce the launch of the above initiative, generously supported by the BUPA Giving scheme.

We’ve blogged about the above initiative for a while (click here or here). The basic problem being that users in resource poor settings (for example, a rural district hospital in Africa or South Asia) frequently do not have access to the latest health care technologies e.g. the latest diagnostic bit of kit or expensive new medicine. However, much of the evidence is directed at the newer, more expensive, interventions. So, users from these settings have to overcome the additional problem that much of the evidence is not relevant.

As of today under each link on the results page will have a link ‘Developing World?’ If a registered user of TRIP believes the article is appropriate (see definition below) for such setting we encourage them to click on the link. If two separate people click on the link the article is then deemed suitable; these articles will form a sub-set of data in TRIP. Users of TRIP will then be able to search TRIP and then (via a tick box at the bottom of the ‘Filter your search’ box) be able to select only those articles in the subset.

We encourage TRIP users to get involved by either tagging articles or to help spread the word of this project!

A few additional pieces of information:

The term ‘Developing World’ is problematic and controversial. We have used it as it is a widely recognised term which we hope will aid adoption, the reality is that the alternative ‘Low resource?’ carries less meaning. However, we welcome input on the term used (send comments to jon.brassey@tripdatabase.com).

The working definition of an article suitable for this initiative is ‘any clinical evidence that can be implemented in primary care and small-district-hospital settings with basic drugs and equipment.’ As the subset of the database is developed, we expect users to discuss and refine the definition and criteria for inclusion. This highlights the fluid nature of this whole project.

Phew, all the definitions, caveats out of the way.

The bottom line is that we want to make access to the evidence easier for those from poorer setting – please help!

Liberating the literature

Answer engine

2011

Associated results

Locating new content on TRIP

Making search slightly smarter

An answer engine

Survey: final section of analysis

Survey: more results

Refining search

Survey ended, start of analysis

Broad versus specific information needs

Survey, more meaningful results

Survey time - early results

Adaptive search

An evidence-based TRIP?

Broken links

New version of TRIP released

App for new content on TRIP

Link resolving

The future of TRIP

TILT and learning points

TRIP's use in SRs

20,000 registered users

Blitter - update

New TRIP product

What's happening?

The cost of search (3)

New email format

Landmark clinical papers

evidently.so

Experiment with new cardiology content

NICE guidance. Is it inaccessible?

Word cloud of new content on TRIP

16,000 registered user

Facebook update

Monthly emails from TRIP

Contextual discovery & TRIP

The cost of search (2)

The latest evidence in TRIP

Welcome to 2011 and a request for new content ideas!

TRIP in 2010

RIS and Background Knowledge Boxes

The next upgrade to TRIP

Analysing TRIP searches: COPD

Help support TRIP

Adding a TILT button to other sites

The impact of TRIP: Part 2

All slideshare for TILT in one place

Another slideshow - on appraisal

TILT: How to make it a social experience

A basic introduction to TILT

TILT is out there

The impact of TRIP

The impact of TRIP

Reflections on Evidence 2010

Top 30 consultations in primary care

Facebook update

Popular papers

Bing likes Facebook

TILT: you'll be amazed at what you don't know

10,000 registered users, official and proper!

Oops, we made a mistake

Keeping TRIP Going!

Facebook, blogger and twitter

Visibility of features on TRIP

NHS Evidence and non 'evidence based' claims!

We're still here!

Monthly update

Ten thousand registered users

Developing world initiative

The cost of search

Clinical questions and StackOverflow

An alternate results page

Health in the developing world

Excelencia clinica

Medical Images on TRIP

New server

Moving servers

TRIP in the future