Monday, October 05, 2015

Search safety net

The search safety net is a novel feature to help improve searching; helping users not miss important papers.  I wanted to explain it - simply - but have failed on that score.  It's important so I hope you can make sense of what I've written.  If you have any questions, my email is

After a search you will see a new 'Search Safety Net' button

If you click that it'll bring up a list of related search terms.  It does this by looking at the top 250 search results and analysing the search terms people have previously used when clicking on these results.  This works on the notion that a single document can be clicked on after numerous searches.  For instance, in the example above search terms might have been 'prostate cancer screening', 'MRI screening' etc.

The next section of the search safety net happens AFTER you're conducted your search and found a number of documents you like AND looked at (or simply clicked the 'check box' to the left of the result).  If you click on the Search Safety Net button again you see three columns of results:

The first column is closely related articles, the second is other related articles and the third is the related search terms.  The latter column is similar to the description of related search terms above, but is based purely on the documents clicked (as opposed to the top 250 results).  However, to understand the process behind the other two columns you need to understand a clickstream data.

Paper 1 ---------- Paper 2 ---------- Paper 3

In the above there are three papers (1-3).  A user, in the same session, clicks on Paper 1 and Paper 2, therefore we can make a link between the two.  Another user might click on Paper 2 and Paper 3, again making a link.  So, Paper 1 is connected to Paper 2 (a single step, using network language) while Paper 3 is two-steps away from Paper 1.  We have this data for all articles in Trip.

Slightly simplifying things (!) the first column is the most popular related articles based on documents that are one step away from the documents clicked.  So, we look at all the articles clicked by the user and pull back all the documents that are one step away, displaying most 'popular' at the top.  The second column are all the documents that are two steps away.  This is likely to find less focused results, but the occasional really interesting study that might have been missed.

Two important issues:
  • For this to work requires clicks, if the documents you've looked at has no clicks, then you'll get no results.
  • This is being released as a 'beta' bit of software, as in we're still developing it.  At present it is available to both free and Premium users of Trip.  However, this is likely to change in the near future.

Friday, September 25, 2015

Logging in to use Trip

Earlier this month I asked our users their thoughts on logging in to use Trip.  With hundreds of responses the results were pretty evenly split, half didn't mind logging in and half thought it was a pain and made them less likely to use the site.  There were two main reasons given for not liking logging in (apart from difficulty remembering passwords):

  1. Health professionals, often jumping from computer to computer, in a busy clinic and simply not having the time to login.
  2. Information specialists trying to demonstrate Trip to users and finding it highly problematic to get all the students to register.
As a result of the feedback, and my own disquiet at forcing login, we have rolled back the login screens and users can get unhindered access to the free site, as of now.

As an extension to the easier access we are currently working on two other changes:

  1. Making registration/login not necessary for institutional subscribers who use IP authentication.  Our system will detect the user is from a subscribing institution and they will get seamless access to the Premium Trip.
  2. Trip has the ability to link to your institutions subscription journals full-text (as opposed to the PubMed abstract). Where we have the IP details of the institution we will seamlessly insert the links to these full-text. This feature is open to both free and Premium users. If you belong to an institution who subscribe to journals and would like to take advantage of this feature then let me know.
These two additional features will be ready in the next two weeks.

Thank you all for your feedback, we asked, you replied in your hundreds, we listened and acted pretty quickly!

Wednesday, September 16, 2015

Restricting search results by clinical area

Just over three weeks ago I published Clinical area tagging of documents which highlighted a really useful but fairly neglected part of the site.  In short it's a system that tags documents, by clinical area, as they are added to Trip.  There are multiple clinical areas e.g. cardiology, urology, oncology.  Users can then search for an item of interest and restrict the search results to a given clinical area.

The motivation for this came, many years ago, from a Professor of Anaesthetics I wanted to demonstrate Trip to.  After two weeks of use they reported back, saying that the results were poor.  Further investigation reveals their interested in awareness under anaesthesia and they had searched for 'awareness'.  If you repeat the search yourself (click here) you'll see very few of the results are related to anaesthesia.  However, if you restrict a search of awareness to anaesthesia (click here) the results are really focused and would have impressed the Professor much more!

We've recently overhauled and significantly enhanced the tagging process making it even more powerful.  Give it a try and let me know how you get on.

Below is a brief screencast to show you how to use it.

Finally, for those interested in the mechanism of action around the tagging of documents it's fairly simple.  We have a list of terms associated with each clinical area.  So, words such as cholesterol, hypertension, statins, angina are associated with cardiology.  The number of words used per area varies, but in some clinical areas it's well over one hundred. If any article in Trip contains any of these words in the title it's tagged with the appropriate area.  So, an article on hypertension in children, would be tagged as both cardiology and pediatrics.  Due to the nature of the process it can't be assumed to be perfect, but it is usually very powerful. 

Thursday, September 10, 2015

Search safety net (or 'what have I missed?')

We're continuing to discover uses for the clickstream data Trip has and the new use is the search safety net - which we're currently testing.

The idea is that as you search Trip we note which articles you've clicked on and then, using clickstream data, predict other articles that might be related, or those you might have missed (hence the name). See the video embedded below.

There are two issues/problems/challenges:
  1. Lack of data. This only works on clickstream data, so it requires clicks!  Very new articles or obscure articles will not have the data.  As it happens, in the tests we've done, it's been - broadly - really good.  But when we roll it out, it's something to consider.  
  2. User experience.  This is the biggest challenge is how will users interact with the 'service'?  In other words where do we put the results?  Do we automatically show them somewhere on the results page?  If so, will that annoy people who don't want the service?  Alternatively, we create some sort of 'search safety net' button which would require a user to click the button.  This means many people will simply not see it and miss out.
Once we solve the user interaction side of things, we'll roll it out.  In the interim, if you want to give it a try (and be one of the first people to use it) then drop me a line (

Friday, September 04, 2015

Trip Tips - starring items

This is a really simple tip to help you save documents you think are important or you want to come back to.

You need to sign in and simply do a search and 'star' any item you'd like to save.  In the image below you can see the position of the stars, to the left of the article titles.  Once starred you can, at any time, click on the 'Starred items' link at the top of the page to go back to previously starred items.

If you're still unsure watch the quick video below.

Friday, August 28, 2015

Are you a luddite?

Many will be familiar with my post A critique of the Cochrane Collaboration, it's been the most viewed article ever published on this blog.  Continuing in the theme was Some additional thoughts on systematic reviews and more recently Evidence, hourglasses and uncertainty.

They all point to the current methods employed in systematic reviews (as exemplified by Cochrane) being a mess! As a type of summary, a few problems:
  • They can't be relied upon to be accurate
  • They're financially costly
  • They're typically out of date
  •  Significant opportunity cost
More evidence that my perspective is correct is shown is the presentation Does access to clinical study reports from the European Medicines Agency reduce reporting bias? submitted to the Cochrane Colloquium in Vienna.  The conclusions:

"Unpublished clinical study reports held by EMA may be a useful source to reduce outcome reporting bias."

It's a testimony to Cochrane's openness to allow such 'dissent' to be published.  It's a dissenting view as the current Cochrane methods rely almost exclusively on published journal articles; unpublished clinical study reports - virtually unheard of. But that's Cochrane; on one hand a business trying to maximise it's business model but on the other a collection of individuals doing there best to improve methods.  As an outsider, I find this tension fascinating. Why? Because the best one can say for Cochrane's methods (alongside most other SR producers) is that they are likely to produce 'ball park' accuracy - in most cases you simply cannot rely on the methods to produce results you can trust.  And this is where the tension comes, if they were being transparent, they would say 'buy the Cochrane Library, most of the SRs are likely to be out of date and many are likely to be innaccurate' which even I can see is not great for sales.

But this bring me nicely to diffusion of innovations - an area I studied for the PhD I never completed!  The Wikipedia article summarises it nicely by saying its " a theory that seeks to explain how, why, and at what rate new ideas and technology spread through cultures." The spread of an innovation is characterised as an S-shaped curve:

In relation to systematic review methods, where are you on this scale?  If, like me, you think there are serious problems by relying on published journal articles you're probably at the innovator side of things.  If, however, you think current methods are great and there is no need to change you may well be a laggard.  But, the majority (those that are feeling uneasy) are likely to be in the middle.

Thursday, August 27, 2015

Clinical area tagging of documents

Around 6 weeks ago I wrote the article Logging in to Trip which provoked much negative comment. I have clearly not rushed to reply properly but that's because I've wanted to properly think through the issues.  In short, the reason for asking people to log-in meant we knew more about the users and could therefore improve the service we deliver.  As I see it there are two connected issues of concern:
  • Logging in - it's a pain for users.
  • Profiling users and altering the results accordingly.
One suggestion (from Paul, see comments in previous post), in relation to the second point, is to focus on our refine by clinical area feature.  This is an already available system which tags documents by clinical area.  So, an article titled 'Cholesterol and the elderly' would be tagged as cardiology and geriatrics.  If a user did a search for, say, cholesterol the above document would be returned in the results, alongside many others.  But if they decided to refine by geriatrics, the above document (alongside others tagged with geriatrics) would be moved to the top of the results.

Below are two images showing how it currently works:

The differences are clear and shows the potential for the system.  For me, it's about helping users find the answers they need really quickly.  An anaesthetist, interested in 'awareness', would find the results of the 'normal' Trip disappointing but by selected anaesthesiology all the results are relevant.

As mentioned above this system is available already and can be accessed as shown below:

It's not particularly developed and we could definitely improve it:
  • Machine learning to improve the document tagging
  • Better user interface so it's more apparent and more intuitive to use.
Ultimately, it might serve a similar role as profiling users in improving the search results.