Trip Database Blog

One thing I learnt while studying the diffusion of innovations and social networks was that the greater the uncertainty the more likely we are to turn to people for advice/reassurance.

Two recent personal experiences highlight this phenomenon. Firstly, I was looking for places to go on holiday. There were a number of companies offering the type of holiday I wanted (sailing/activity) and from multiple locations. All were broadly similar in cost, had similar weather and facilities. So, to help me decide I took to TripAdvisor and located all the potential targets and chose by looking at user ratings and comments.

The second experience relates to me buying a new camera (which broke while on my holiday) I wanted a particular type of camera and to help me decide I went to Amazon and again looked at user ratings and feedback. In the end the newest version of the camera I wanted had pretty poor reviews, so for now, I’ve decided not to buy a replacement and to simply rely on my relatively good mobile phone camera.

I’ve been reflecting on this theme as recently, two separate users of Trip have floated the idea of introducing such a feature in Trip. I posted the idea on our Facebook page over the weekend and idea was received quite positively (based on a small number of responses).

I like the idea as it can help give context to the research, give different perspectives and perhaps help highlight potential problems with the evidence. There are associated problems such as potential bias, inaccurate comments etc. But I’m sure these negatives can be mitigated for, with some thought.

So, what might a rating/comments feature look like? I have my ideas, which I’ll highlight below but I’m really keen to obtain feedback from you. This feature, if it is to be released, will not happen till next year – but it’s useful for me to reflect on ideas.

One thing that is essential is that the system is easy to use and understand. I would like it to be more than a binary ‘good or bad’ or ‘thumbs up or down’. On both TripAdvisor and Amazon I like to look at reviews by score. So, for those who gave a holiday/product a low score, why was that and vice versa for high scores? So, I think it requires a numerical scale and both TripAdvisor and Amazon use a 1-5 scale (although, someone pointed out that people tend to gravitate to the middle).

When people have scored an article we should offer them the ability to comment. We could suggest a structure to comment against (e.g. what did you like about the article, what did you dislike etc) but I think the more formalised you make it the less it’ll be used. So, I favour a free-text response.

The results would need to be displayed somehow but I’ll not give that much thought now, I think our designer would be the best to advice on this. Needless to say it needs to be clear and easy to understand. I also like the idea of being able to sort results by rating (currently, on Trip, you can sort results by relevance and date).

There are still questions to be explored (apart from the big one of do people think it’s a good idea), such as:

• Will it be used? I’ll explore this further with our users over the next few months (after we release the new version of the site – due imminently).
• What will the score represent? I think one can over-analyse. But, I think it indicates a user’s view/opinion on a paper and what this represents is individual to the person. With TripAdvisor and Amazon the comments help explain their rating and you take the points that you feel (as a consumer of the ratings) are important to you.
• Do we ‘seed’ the scores? We could create a starting score for each document to get the ‘ball rolling’. We could create a score based on the quality of the publication and how often it has been viewed. As people submit ‘real’ ratings the seed-score diminishes in worth.

I often get excited by new ideas and this is no exception. If ratings/comments take off I think it will impact how users consume evidence. It’ll also deliver something that is hugely beneficial, yet potentially immeasurable – value!

Finally, now you’ve read this, please do our two question survey – click here.

I forget how horrible it is, testing the site.  There is the worry about how users will take to it (few people like change), will people like the new features,etc.  Oh yes, the hard work and worry that you’ve missed something!

I’m reassured that the core of Trip – our content and search system are broadly untouched – so users will still get the same results.  The new design, a number will not like but after a few searches I have no doubt that these will be forgotten. 

I’ve previously blogged about the changes (click here) but there is an additional change which you might find interesting – results thumbnails.  I’ve even done a screencast to show how it works (click here) which gives a sneak preview of the new site.

I’m hoping it’ll all be finished and launched by mid-October (so two weeks). The ‘to do’ list of fixes is currently at around 35 and a major headache that it looks awful in Internet Explorer 7 (an old browser, but heavily used in the UK’s National Health Service).

If you want to help test the site then email me – jon.brassey@tripdatabase.com

I had a great meeting with Carl Heneghan at the Centre for Evidence Based Medicine yesterday.  I went with an agenda which Carl skillfully dispatched and then turned everything on it’s head.  I believe I’ve recovered.

One issue we touched upon was new research and how to keep up to date with the latest evidence. At Trip we aggregate the World’s evidence.  If there’s new high-quality research it should be (and probably is) in Trip.  But Trip is a search engine.

Chris (my partner in Trip) is a GP and he uses Trip to answer clinical questions.  To him this is what he wants Trip to do and he’s very happy with how it works.

Carl, on the other hand, wants to know what’s new and what to take notice of (as well as searching for answers).

So, what can we do to help?  We have our monthly emails, but they’re far from perfect.  There are oftern way too many results (especially in primary research).  So, one potential approach is to just highlight new research from the categories of secondary evidence (systematic reviews, synopses and guidelines) and key primary research (big 5 internal medicine journals plus the output from ‘EvidenceUpdates’). If we take the example of diabetes, below is the complete list (from last month) of articles with diabetes in the title (an important distinction) from the ‘top’ sources:

• Different intensities of glycaemic control for pregnant women with pre-existing diabetes, Cochrane Database of Systematic Reviews
• Continuous glucose monitoring systems for type 1 diabetes mellitus, Cochrane Database of Systematic Reviews
• Momordica charantia for type 2 diabetes mellitus, Cochrane Database of Systematic Reviews
• Interventions for pregnant women with hyperglycaemia not meeting gestational diabetes and type 2 diabetes diagnostic criteria, Cochrane Database of Systematic Reviews
• Self-monitoring of blood glucose in patients with type 2 diabetes mellitus who are not using insulin, Cochrane Database of Systematic Reviews
• Saxagliptin (Onglyza) – type 2 diabetes mellitus, All Wales Medicines Strategy Group
• Type 2 diabetes: newer medicines and insulin analogues, WeMeReC
• Exenatide – Type 2 diabetes mellitus, Canadian Agency for Drugs and Technologies in Health – Common Drug Review
• Reno-protective effects of renin-angiotensin system blockade in type 2 diabetic patients: a systematic review and network meta-analysis, DARE
• Type 2 diabetes – reducing cardiovascular risk, National Prescribing Service Limited (Australia)
• Empagliflozin for type 2 diabetes mellitus, Health Technology Assessment (HTA) Database
• Review finds little evidence for Complementary and Alternative Medicines in diabetes, The Diabetes Elf
• Review:  insulin pumps may improve quality of life and glycaemic control in adults with type 1 diabetes, The Diabetes Elf
• Peri-operative diabetes management guidelines, Clinical Practice Guidelines Portal, Australia
• Linagliptin rejected by the NHS in Wales for use in type 2 diabetes due to lack of evidence of efficacy, The Diabetes Elf
• Moderate physical activity is associated with lower mortality in people with diabetes, The Diabetes Elf
• Diabetes accounted for 8.9% of prescribing costs in NHS England in 2011/12, The Diabetes Elf
• Better evidence needed on the effectiveness of tailored interventions on self-management type 2 diabetes, The Diabetes Elf
• Review:  more evidence needed for metformin in children with type 2 diabetes, The Diabetes Elf
• New guidelines for the management of diabetic retinopathy, The Diabetes Elf
• Review finds weak evidence favouring moderate blood sugar targets in diabetic women during pregnancy, The Diabetes Elf
• Intensive blood pressure control prevents strokes in diabetic patients with hypertension, The Diabetes Elf
• Eating disorders are common and problematic in adolescents with type 1 diabetes, The Diabetes Elf
• Review:  children and adolescents with type 1 diabetes can have the same quality of life as those without it, The Diabetes Elf
• Welsh NHS recommends saxagliptin as an option in type 2 diabete, The Diabetes Elf
• Comparison of Two Creatinine-Based Estimating Equations in Predicting All-Cause and Cardiovascular Mortality in Patients With Type 2 Diabetes, Diabetes Care (EvidenceUpdates)
• Concentration of Cystatin C and Risk of End-Stage Renal Disease in Diabetes, Diabetes Care (EvidenceUpdates)
• Maternal Efficacy and Safety Outcomes in a Randomized, Controlled Trial Comparing Insulin Detemir With NPH Insulin in 310 Pregnant Women With Type 1 Diabetes, Diabetes Care (EvidenceUpdates)
• Intervention Costs and Cost-Effectiveness of a Successful Telephonic Intervention to Promote Diabetes Control, Diabetes Care (EvidenceUpdates)
• Short-term continuous glucose monitoring: effects on glucose and treatment satisfaction in patients with type 1 diabetes mellitus; a randomized controlled trial, Int J Clin Pract (EvidenceUpdates)
• Diabetic neuropathy: clinical manifestations and current treatments, Lancet Neurol (EvidenceUpdates)
• Hemoglobin A1c Versus Oral Glucose Tolerance Test in Postpartum Diabetes Screening, Diabetes Care (EvidenceUpdates)
• Combined intensive blood pressure and glycemic control does not produce an additive benefit on microvascular outcomes in type 2 diabetic patients, Kidney Int (EvidenceUpdates)
• Curcumin Extract for Prevention of Type 2 Diabetes, Diabetes Care (EvidenceUpdates)
• The cost-effectiveness of substituting physicians with diabetes nurse specialists: a randomized controlled trial with 2-year follow-up, J Adv Nurs (EvidenceUpdates)
• Performance of HbA1c as an Early Diagnostic Indicator of Type 1 Diabetes in Children and Youth, Diabetes Care (EvidenceUpdates)

A great collection of evidence.  As I’ve not used it for a while, I used Wordle to create this word cloud (click to make bigger):

But what’s the best way of Trip using this data, not just for diabetes, what about hypertension, acne, hay fever?  We want to find a great way of highlighting new evidence. It needs to look good and make people want to engage with it…!  Carl highlighted what the BMJ are doing with their portals e.g. diabetes portal, not as something to aspire to but an example of one approach.

If you’ve ever seen Flipboard for iPad, I like that approach but no idea if it’d work.  It might be that we use our – soon to be released – timeline.  But if anyone has any ideas – please let me know.

We’re getting fairly close to the launch of the new TRIP, hopefully within the next 4 weeks – depending on testing.  We are very busy behind the scenes and there is even more work to come.  We’ve just reviewed the 7th iteration of design.  Each time we get a set of designs we need to review them and comment.  I believe version 8 will be the version we approve.  Once they have been ‘signed-off’ we’ll be busy plugging the new and old functionality into the design and then on to proper testing!

The work on the new site started last year and included a large review of our users (click here for the main results).  Our users have been great since then feeding back on various questions I’ve had.  One user (via our Facebook page) even helped formulate a – sort of – tagline of ‘find evidence fast‘, a very simple yet powerful explanation of what TRIP does.  And I cannot forget the wonderful donations we received that have helped make this all happen! 

In conjunction with user input has been a significant amount of work on my part exploring the literature on information retrieval and search.  So, what can we expect?  In no particular order (and there are more as those listed below):

• Redesign.  We’ve used a great designer who’s worked wonders. The old site was looking old and had suffered from new design/features simply being bolted on.  While it’s still a search engine it has been completely redesigned – even the logo will change.  IT IS LOVELY.  I appreciate some people will be put off as it’s a huge change, but people will get used to it and love it.
• PICO search.  As well as the ‘usual’ search we have introduced a PICO search to help users formulate focused searches. 
• Clear demarcation of content types.  While TRIP is principally about evidence, we also have videos, images etc in our index.  We’ll be making this more explicit and easier to use.
• Filter move.  Currently the filter by evidence type is on the left-hand side, we’re moving it to the right.  The most important aspect of TRIP are the results, so they should be in the most prominent position!
• Improved filtering – allowing easier filtering by clinical area, year and even the ability to restrict the results to a single publisher.
• Cited.  We’re starting to automatically explore citations.  If you find an article you like, we’ll link to other articles in TRIP that have cited it (typically a systematic review or guideline).  This is new and experimental – so expect mixed results!
• Important papers.  For any search on TRIP we’ll explore the citations found in the results and see which articles appear most frequently – surely a sign of importance to the search terms.
• Login.  While not compulsory we’re going to encourage people to login.  This serves numerous functions, some obvious now and others for the future.  But, immediately, when you search we’ll also include previous papers you’ve viewed – as we know people often revisit the same papers, this makes it much easier. 
• Starred.  Like and article or think it’s interesting – well star it and we’ll save it for you in your timeline….
• Timeline.  This is where we record your activity on TRIP: searches, articles viewed, articles starred, any new content that we think is of interest to you (based on your recorded keywords of interest).  There’s a twist in the timeline that will make it special.  The Timeline might lead to much bigger things.

TRIP was good before, it will now be better and I think we’re getting close to greatness.

We’ve been working very hard on the next version of TRIP and we’re getting there!  There are two main components of work, functionality and design.  These are performed by separate people (a techie and a designer) and eventually brought together by the techie.  We’re hoping to go live in the next 4-6 weeks.

We had a first round of designs about 2-3 weeks ago and had the latest version yesterday.  So, it was a hectic day going over them and giving feedback to the designer.  But the design is close to being signed-off.  It’s actually a complete overhaul of the site, including a new logo.  It will look and behave radically differently, it’s a significant step forward for TRIP.

See below a small sneak preview of a component of the results page (click on image to increase the size).  Some hidden for tease value and some hidden as they need altering. I hope you enjoy it 🙂

I’m doing some work (well, thinking really) around clinician similarity and information needs.

Basically, if a UK general practitioner does a search for diabetes the intention/information needs are likely to be different than, say, a Brazilian endocrinologist. Yet, TRIP returns the same results.

If we created a similarity score (based on profession, interests and geography) we could we show the results as per normal but also have an area ‘Clinicians like you who searched for diabetes looked at these articles…’. 

We could also introduce something similar at an article level – ‘Clinicians like you who looked at this article also looked at these…’.

In a way, it’s using the experience of previous similar users of TRIP to filter and hopefully improve search results.

What do people think?

As part of the new upgrade to the site we’re experimenting with a number of new features. An ‘Important Papers’ feature is a side-effect of one of our efforts, but what’s that?

When you search TRIP our algorithm is designed to show the best, most research, which is great.  However, much of the latest research is built on significant ‘historical’ papers in that field, for the time being we’re calling them ‘Important Papers’ (happy to take suggestions for other names).  These are important papers associated with the main search results.

To illustrate what I’m talking about, take an example search for ‘warfarin anticoagulation’, on the main TRIP the top results can be seen here (the top result being: Comparative Effectiveness of Warfarin and Newer Oral Anticoagulants for the Long-term Prevention and Treatment of Arterial and Venous Thromboembolism (Veterans Affairs Evidence-based Synthesis Program Reports 2012)).  Using our ‘Important Paper’ feature, the top 3 results are:

• Placebo-controlled, randomised trial of warfarin and aspirin for prevention of thromboembolic complications in chronic atrial fibrillation (Lancet, 1989)
• Oral Anticoagulants vs Aspirin in Nonvalvular Atrial Fibrillation An Individual Patient Meta-analysis (JAMA, 2002)
• A risk score for predicting stroke or death in individuals with new-onset atrial fibrillation in the community: the Framingham Heart Study (JAMA, 2003)

We’re using a number of measures to infer importance including citations and social media.

Also, to be clear these results will be presented away from the main results (as a side-feature).

Will this feature be a hit?  I’ve no idea.  The results are certainly interesting and user feedback will decide if it stays.

For a number of years I’ve been pondering the numerous relationships contained within TRIP.  These are numerous and a few examples include:

• Relationships between articles e.g. via citations, by being in the same publication
• Relationships between users e.g. linked by clinical interests, by geographic location
• Relationships between articles and users e.g. looking at the same articles

I can’t help feeling there is value in these links and I do not mean in the financial sense.

Take a really simple example (click on image to enlarge it)

This is an imaginary search undertaken by 5 users and a line signifies which papers they looked at.  We can deduce some things:

• Papers six and seven weren’t liked.
• Paper two looks the most popular
• Users 2 and 3 appear similar/close (both looking at two of the same papers)

Now, if we add an extra level of data:

In this image the rounder reddish boxes signify doctors while the green boxes signify nurses.  Do these inferences seem reasonable?

• Paper five is really suited to nurses while papers one (to a point), two and three are more ‘doctor’ focused
• Paper four has mixed interest.

Imagine if you can add extra detail (different types of doctors, different geographic location) and lots of data (something we have lots of in TRIP) you might be able to generate a really powerful system.  Could it inform search results?

What do people think? I’ve really simplified things to make a point and I doubt the data will ever be as clear cut. 

Are you in for a treat?  I’d like to think so.

I had a long but rewarding meeting with Phil (our genius techie) and Reuben (our new and wonderful design guy) to thrash out the final details of the next upgrade to TRIP.  I am so excited by the proposed developments which are:

• 3 major innovations – for me really really important developments
• A handful of significant improvements
• A load of minor improvements
• A design overhaul

I’m being deliberately vague with the details for now.  But as things develop and designs get drawn, features get available I may well share them here and on Facebook (if you didn’t know we had a Facebook page, click here).

No firm timetable but I’d like to think it’ll be ready in 3 months.

Also, I’ve started planning the next upgrade, the one after this one, but much depends on how the innovations from this one take off.  If you have any suggestions then feel free to let me know.