Trip Database Blog

I’ve already posted a breakdown of stats for the site (see 2015 in numbers).  The highlights for me are:

• 3,700,000 – page views
• 900,000 – individual sessions spent on the site
• 650,000 – number of users of the site
• 4,560,603 – number of minutes spent on the site
• 350 – mentions in journal articles
• 881,280 – number of times Trip helped improve patient care in 2015

Apart from our continued impact on health care related to search we made large strides towards financial security with the launch, in May, of the Freemium business model.  Uptake has exceeded our expectations with a significant increase in income.  However, we are far from secure, so any suggestions around additional income streams and/or sponsorship opportunities – I’m all ears!

The one slight mistake we made, in moving to Freemium, was to insist on users logging in to use Trip, which resulted in a significant user backlash.  We listened and removed that ‘feature’ and that resulted in a large increase in usage towards the end of the year.

We’ve been making some great progress in relation to our work around clickstream data.  For instance:

• Clickstream data and results reordering. Still experimental but very solid groundwork for future developments.
• The search safety net

A new site, rapid-reviews.info was launched to help separate out the Trip related content (on this blog) from the content related to rapid/systematic reviews.  Talking of which, my post ‘A critique of the Cochrane Collaboration‘ reached over 20,000 views last year.

As part of our work with the EU-funded KConnect project we introduced some very nicely integrated multi-lingual search function.  It’s currently restricted to French, German and Czech but more languages will come online in 2016 including Spanish.

But what else can we expect in 2016?:

• A big announcement around our freemium business model in the next week or so.
• A significant redesign of the site, hopefully ready by February.
• A first draft of our answer engine concept which is looking really exciting.
• Further work on rapid reviews including working on understanding important regulatory data and improvements to the Trip Rapid Review system.
• The rollout of a new system to help cope with broken links.
• Further development, related to our clickstream data, around analytics and insights e.g. A new analytic ‘toy’ and Searching for hypertension.

Running Trip is full of interest, lots of excitement and helping support care with high-quality evidence across the globe is the icing on the cake.  One thing I’d like to see this year, apart form continuing financial security, is to have a greater academic presence/impact.  I’m not even sure why, I think I’d just enjoy it (based on interactions over the last few years).

I will sign off now and wish you all a happy 2016.

Some statistics from 2015

• 3,700,000 – page views, up slightly on 2014
• 900,000 – individual sessions spent on the site
• 18% – increase in users in November and December compared with 2014. I can only assume this was down to our removal of the requirement to sign-in to use Trip!
• 650,000 – number of users of the site
• 4,560,603 – number of minutes spent on the site
• 76,010 – hours spent on the site
• 178,000 – number of most recent registered user.  This suggests the actual number of registered users is 142-160,000 registered users.  This represents an increase of nearly 40,000 last year
• 350 – mentions in articles
• 83% – users accessing the site via a desktop computer (mobile 11% and tablet 6%)
• 46% – users accessing the site with Chrome, the most popular browser (IE 20% and Safari 16%)
• 22% – users from the USA followed by UK 13%, Spain 8%, Canada 5% and Australia 4%

The above figures are based on users actively coming to the website and does not include an increasing large number of people searching Trip via our webservice. So, if we assume an average of 2 searches per session and assume 20% of the searches coming via webservice (obviously the assumptions makes accuracy problematic) that equates to 2,160,000 searches in 2015. From previous analysis we have estimated that 40.8% of searches help improve patient care.  Therefore, probably the most important statistic of the lot:

881,280 – number of times Trip helped improve patient care in 2015.

Historically I have used this blog to write about Trip developments as well as expanding on my own work around rapid reviews.  This can sometimes be appropriate, for instance when we highlight a new tool on Trip to make reviews more rapid.  However, sometimes the articles have nothing to do with Trip and more to do with me (Jon Brassey) as an individual.  It was one of the reasons I started the Rapid Reviews website.

My intention is to post these articles on that site but I will continue to link to these articles if I feel they are substantial and offer value/benefit to Trip users.  One such article, is Why do we do systematic reviews? Part 4, possibly my firmest critique of the current ways we undertake systematic reviews.  This is linked to the need for rapid reviews.

This is probably my last post this side of Christmas, so if you’re ready – and that way inclined – have a wonderful time.

Every now and then it’s important to take a step back and look at how the site is performing.  I’m not talking about the search results/algorithm, more the user interface. I am broadly happy with the design (logo, colours etc) but I’m less convinced that the site is configured optimally to ensure users can easily find the information they need.

So, we’re exploring that and making some headway.  Don’t expect it to look radically different, but hopefully it’ll be a delightful surprise.

It should be ready early 2016, but that depends on user testing of the new interface.

Feel free to make suggestions as to bits you don’t like…!

I love clickstream data – the data we collect as people use the site.  We know – anonymously – the search terms used and articles viewed.  We have just developed the ability to ‘mine’ this data; nearly 100 million items it!

Below are two images, both taken from searches for hypertension.  The first is for all the data we have from 2009 (tens of thousands of searches for hypertension) and the second is from 2015 only (still thousands).  What we’ve done is taken these searches, removed the term hypertension and created a wordcloud of the remaining words.  These words being the additional terms users used when searching for hypertension.  So, if a user searched for hypertension pregnancy, we simply chart the pregnancy term.

Searches since 2009

Searches in 2015

They look fairly similar but from a cursory analysis five terms are more prominent in 2015, those being:

• placebo
• aspirin
• patient
• enalapril
• obesity

Interesting for sure, but more than that?

Broken links are never great but unfortunately they are unavoidable.  They typically happen when a website either removes an article or has a redesign and moves all the old links to new ones.  Users, naturally, get frustrated and Trip has to do better.

We currently have a broken link detector – which sends an alert for each broken link.  In reality it’s crude in that it ‘decides’ a link is broken if it takes longer to load than five seconds.  If a user is on a slow internet connection or a site is running slowly – an alert is sent.  So, we get lots of false positives, making the system poor.

But, recently we spent a great deal of time analysing these and found a large number of sites with problems and have completely revamped these links.  So, the true positives should fall dramatically.  However, moving forward, we’re planning a new broken link system.  If we detect a broken link (or one we suspect is broken) we will re-try it a number of times over the next 24 hours and only after a few tries will it be considered a true positive and an alert created.  The alert will contain additional tools to make it much easier to amend the index.  While no system will be perfect, we hope that this system will help to significantly reduce the problem of broken links.

Earlier this month I asked our users their thoughts on logging in to use Trip.  With hundreds of responses the results were pretty evenly split, half didn’t mind logging in and half thought it was a pain and made them less likely to use the site.  There were two main reasons given for not liking logging in (apart from difficulty remembering passwords):

1. Health professionals, often jumping from computer to computer, in a busy clinic and simply not having the time to login.
2. Information specialists trying to demonstrate Trip to users and finding it highly problematic to get all the students to register.

As a result of the feedback, and my own disquiet at forcing login, we have rolled back the login screens and users can get unhindered access to the free site, as of now.

As an extension to the easier access we are currently working on two other changes:

1. Making registration/login not necessary for institutional subscribers who use IP authentication.  Our system will detect the user is from a subscribing institution and they will get seamless access to the Premium Trip.
2. Trip has the ability to link to your institutions subscription journals full-text (as opposed to the PubMed abstract). Where we have the IP details of the institution we will seamlessly insert the links to these full-text. This feature is open to both free and Premium users. If you belong to an institution who subscribe to journals and would like to take advantage of this feature then let me know.

These two additional features will be ready in the next two weeks.

Thank you all for your feedback, we asked, you replied in your hundreds, we listened and acted pretty quickly!

Trip users are amazing – in less than 48 hours of releasing the survey we had 1,0001 responses, at which stage SurveyMonkey closed the survey saying we’d reached the limit!  Apologies if you feel your voice hasn’t been heard, if that is the case email me (jon.brassey@tripdatabase.com), I’d love to hear from you. Given your generosity of time I thought I’d share the initial results highlights…

The top 5 professions represented in the survey

• Doctor – secondary or tertiary care
• Doctor – primary care
• Librarian
• Other
• Researcher/scientist

75% of respondents have been using Trip for more than a year with 35% using it for longer than 3 years.

I asked about the most important features relating to our content and these are the top 6 responses (those that were highlighted by more than 30% of the respondents):

• Largest single searchable collection of ‘evidence-based’ content
• Largest global collection of clinical guidelines
• Many more systematic reviews than Cochrane
• Content is from around the globe, for example USA, UK, Canada, Australia, New Zealand, France, Germany, Japan, Singapore, South Africa
• Selected collection of PubMed’s leading clinical journals
• Database of over 500,000 clinical trials

I also asked if there were many surprises – and there were lots of responses.  The main one being the lack of awareness of our image and video collections.  We clearly need to work hard on getting that message out.

I asked about the most important key features of Trip, the following are all those that polled over 30%:

• Easy filtering of results to restrict to evidence types e.g. systematic reviews, guidelines
• Monthly alert of new evidence linked to your interests
• PICO search interface
• Order the results by quality, relevance or date
• Easy/Friendly interface with no steep learning curve
• Advanced search interface
• Colour coding scheme to make it easier to highlight high quality evidence

Our users seem keen to be alerted to clinical trials, jobs, conferences and books (most polled over 50% approval).

We asked about a Trip Evidence Service and most thought it was a good idea.  However, only 11% thought they would be able to find the money within their organisation.  But I’m encouraged as 11% is still high, given our large user base.

Most people appeared to be broadly supportive/understanding of our need to move to a freemium business model.

I listed a number of possible new premium features and those that polled greater than 20% (only the top 3 were higher than 30%):

• Add in additional full-text articles
• Creation of an ‘Answer Engine’ giving you instant answers to your clinical questions
• PICO+. Based on the popular PICO search make it more user friendly and powerful
• A ‘Help’ feature so if you can’t find what you need you can ask the wider Trip community
• Providing education points based on your time using Trip
• Improved emails highlighting evidence that is more likely to be useful to you
• Introduce a ‘People who looked at this article, also looked at these articles’ features to highlight related articles
• Improved export of records

Due to us using colour we asked about colourblindness and 3.2% said they were colourblind.  I’ve no idea how that compares to the wider population.  nearly 30% of the users reported “I am not colour blind and I was not aware that you used colour to help highlight the quality of the results”.  So, another communication challenge for us.

Finally, in looking through the ‘Any other comments’ section I was completely overwhelmed by the messages of love and support.  Knowing that makes my work so much easier.

We are planning to make significant changes to Trip in early 2015.

An important aspect of this is better understanding our users; how they use Trip and what features they value.  In addition we’re keen to explore attitudes to various proposed changes.

This is a really important survey so can you please take 5-10 minutes to go through the 14 questions.

Click here to take part in the survey

Thank you.