Wednesday, April 09, 2014

Evidence-based tweeting

Or possibly more appropriately, evidence-based dissemination of research evidence.

For a while now we have been tweeting new research added to Trip in 8 topic areas, those being:
For each account, we send a topic specific tweet with the title and the corresponding URL of an article recently added to Trip. They have been really successful with over 2,500 followers since the start of the year (with little publicity) and tens of thousands of clicks (on links to the articles).  

In the last month we've been monitoring the activity quite closely to better understand usage.  We have typically been tweeting articles throughout the day at fairly even time intervals and seeing how many clicks we're getting.  Doing so has allowed us to graph the most popular time for people clicking on links, which we have graphed below (click on graph to expand).  The data is based on hour time-periods when we have received more than ten clicks in that hour.

So, there's a clear pattern with the evening (UK time) being the most popular. 

We have used this information to see if we can better focus the tweets we send to coincide with the greater likelihood of being clicked on.  So, this means the following:
  • Weekdays - a tweet between 9-10am then some tweets after 6pm, with a concentration between 8-11pm.
  • Weekends - a tweet around 11am then some tweets after 3pm, with a concentration between 9-11pm.
We've uploaded the tweets for the next 6 days and I will report back to see if this 'evidence-based' targeted tweeting makes any difference!

Thursday, April 03, 2014


We have a manual system for handling synonyms in our search.  This means that if someone searches for IBS we automatically search for irritable bowel syndrome.  I'm currently undertaking a review of these synonyms, a long-winded and problematic process - but well worth doing.  However, I'm stuck and would like people's views (email me via  Currently, we have three separate 'collections' of synonyms:

paediatric, paediatrics, pediatric, pediatrics, infant, children, infants, infancy, child, childhood, kid, kids, preschoolers, childrens, children’s

infant, babies, baby, child

newborn, neonatal, neonate, neonates, newborns, neonatology

There is clear overlap.  But there is a precise answer and a pragmatic one.  In other words, you need to put yourself in the shoes of a searcher.  So, if they search for children and cough is it reasonable to drop in the synonym paediatric ie also search for paediatric and cough?  I think it is.

But, and this gets a bit harder - where is the overlap between newborn with infant and with children?

The easiest solution is to lump the top two 'collections' together and leave the third 'as is'. 

But I'd welcome opinions!

Wednesday, April 02, 2014

Trip and CPD/CME

I've posted before about Trip's educational merits.  But, in a nutshell, using Trip to help answer clinical questions is undoubtedly educational.

Many health professionals around the globe are required to demonstrate that they are keeping up to date with the latest evidence and the requirements vary widely from country to country.  Trip is very keen to help support this and to date we have two main ways:
  • The timeline, this records all activity on Trip (search terms used, articles viewed) and can be exported for inclusion in educational portfolios
  • Reflective toolbar.  This is little used but allows a user to open a document and answer reflective questions about it.  This is recorded and is exportable.
But, we want to improve our educational support but require help from our users - hence this post.  It has been prompted by me seeing, for a UK-based educational activity, a company offering 1 CPD credit.  The notion is that if an activity takes an hour they get 1 CPD credit.

Might such an approach be useful in Trip?

For every article read do we assign a CPD credit?  Do we also allow, via the timeline, the ability to reflect on an article to gain extra CPD credit?  So, you might read an article and gain 0.5 CPD credits and if you then record your reflections it goes up to 1 CPD.

So, if the above impacts on your professional life please let me know what you think (via  If we get it right it'll be a huge benefit.

Thursday, March 13, 2014

Colours and Trip

I have recently received an email from a librarian in Sweden asking about the use of colours in Trip.  Hopefully this post will answer her questions and also prove useful to others.

A small bit of background first: when someone is presented with a list of results the mind attempts to make rapid sense of them; to better understand them.  The mind uses heuristics to minimise the cognitive load, the lighter the load the better for the user.

It is for that reason I introduced a colour coding system in Trip.  While everyone might know that Cochrane, NICE and AHRQ are likely to be higher quality (using secondary review methods) what about other, smaller, lesser known publishers that we include in Trip?  For instance, a user might not have heard of BestBETs and therefore - falsely - relegate the 'worth' of that article.  Similarly, many users might feel that preeminent journals such as NEJM and The Lancet must be high-quality.  However, within Trip we only count these as medium quality as they have only gone through peer-review (itself a highly flawed quality control system).

In other words there is a hierarchy, based on likely quality of article/publisher, within Trip.  Quality is perhaps an over-simplification, it's more robustness and transparency of method (but the two are closely linked).  This hierarchy features heavily in our search algorithm (the technique we use to decide on the order of articles on the results page), which favours higher-quality, secondary review type evidence

In Trip we use the colour system to reflect the quality hierarchy.

In the image above (click to enlarge) you can see two areas where we use colour:
  • On the right-hand side, associated with the refine/filtering system. As you can see next to Evidence-based Synopses, Systematic reviews there are green 'flashes' and as you move down the quality hierarchy the colours change.  At the bottom we have the yellow for eTextbooks.
  • On the left-hand side, associated with each result is a colour flash.  In the top result, it's an AHRQ publication, which has a green flash - indicating that it's likely to be high quality.  Under that are articles from the journals Breast cancer and Anticancer research, both of which are primary research journals.  They have orange flashes as they are controlled trials.
So, the two sides of colours are linked and should help users quickly locate the content that best suits their needs.

Friday, March 07, 2014

Primary research only search in Trip

I thought I would share this tip.

Via twitter I was asked by @AliceMBuchan if it was possible to restrict the results to all the primary research articles.  Currently, we have four separate primary research 'sections':
For secondary research we have a filter that allows a user to select 'All Secondary Evidence' but nothing similar for primary research.  While I'm not sure there is demand for an 'All Primary Research' button I've been able to create a hack.

If you follow this link it shows such a search for prostate cancer.  Simply remove the term prostate cancer from the search and add the search term(s) of interest.

Friday, February 28, 2014

The impact of Trip in 2013

In 2010 I attempted to estimate the impact of Trip, the results can be seen here, but the important point was an estimate that 40.77% of searches improve patient care.  Within that post I discussed why the 40.77% might be too high and why it might be too low - there are good reasons on both sides.

In 2013 Trip was searched 4 millions times.  So, 40.77% = 1,630,800.

Bottom line: in 2013 Trip helped improve patient care over 1.6 million times.

Oh yes, we were also mentioned in over 350 systematic reviews and/or articles (see here).

Wednesday, February 26, 2014

Article social networks, meaning and redundancy

This is very much a 'thinking aloud' post.

In October last year I posted Structure in Trip an article that described the social networks of articles in Trip, based on clickstream data. The analysis allowed me to produce graphs like the one below (based on the clickstream data of people searching for UTI.

The structure is clear and I've labelled a few, the most prominent being UTIs and cranberry (in the bottom left of the graph). 

I'm increasingly of the opinion that this can be used to speed up the review process and also improve the search experience (but search is for another day). In social network analysis there is a view that within a cluster there is a lot of duplicated information.  If you think about your social networks your close friends probably know lots of the same things as you - this duplicated information/knowledge about birthdays, addresses etc.  I can't help feeling this is likely in clusters of articles.  So, take the cluster of UTI and cranberry there's probably a lot of duplicated information (background information I would have thought).  But there is also lots of unique information (e.g. each set of results will be unique).  Then the conclusions are probably split into three main types - positive, negative, uncertain/ambiguous.

So,  as a precursor to more in-depth work I simply took the articles and created a word-cloud (I did do some editing to remove terms I felt were unhelpful):

And this is the thinking aloud part - I'm not sure if that's helpful or even useful.  It's pretty.  But I really don't think it's hugely helpful as it stands.  However, it does take me further along with my thinking, I think!  Perhaps adding some specialist semantic analysis would be helpful.

Now, and this surely constitutes a world first - an instant systematic review.  I've posted a few times about five minute systematic reviews, but if we could identify clusters, extract the RCTs and automatically put them into our rapid review system the result is an instant review.  Well, I did just that (manually, but it could be automated to make it truly instant).  There were 18 articles in the UTI and cranberry cluster.  Many of these were review articles but there were also 4 placebo-controlled trials.  Placing them in our rapid review system gives the following result:

So, our system gives it a score of -0.12, so I would say that the results show no clear evidence for or against the effectiveness of cranberry juice in UTIs. One point, one of the trials is for a highly-specific population 'patients undergoing radiotherapy for cancer of the bladder or cervix' so ideally would be excluded. 

How does our score of -0.12 compare with the Cochrane Cranberries for preventing urinary tract infections?  Within their conclusion, they report:

"..cranberry juice cannot currently be recommended for the prevention of UTIs"

Lots of things to reflect on.  I would say our conclusion is pretty much the same as Cochrane's.  Ours is based on far fewer trials. This, in part, reflects the fact that our cluster was only based on a small sample of all our data, so a full analysis would highlight more trials.  But I think the principle is there, instant systematic reviews - easy!

I'm hoping I'll look back at this post in 2-3 years time and laugh at how basic the analysis is - reflecting a significant leap forward in my work.