Trip Database Blog

These last few months have been hard work!

• Mid-March we released the Controlled trials filter in Trip.  500,000 trials, all easily searchable and incorporated into Trip.
• Also in mid-March I presented at EvidenceLive 2013, where I gave a talk under the interesting title ‘Anarchism, Punk and EBM’.  The broad thrust can be read in my A critique of the Cochrane Collaboration blog article. This has now been read nearly 4,000 times.
• In early April I set about recruiting for the new Trip Advisory Board. It’s a 20 strong group of clinicians and information specialists from around the world.  This group will become active in the near future and will help advise me on the way forward for Trip.
• I had some good fun creating the Trip Pinterest account. I view this as a simple way of posting pictures which I can easily link to!
• More recently I have been concentrating on the latest upgrade to Trip. The highlights being better access to full-text articles, a ‘developing world’ filter, integration of DynaMed and case reports and a host of other minor changes.  The full-text feature is a personal triumph as it has taken me so long to figure out how it’s done!  Successive surveys have shown better full-text access as a priority so, finally, to be able to help is wonderful.  As I write this, we have 199 institutions signed up and over 350 individuals have aligned themselves with their institution – not bad for 3(ish) days.  Currently, individuals from Barts Health NHS Trust, NHS Scotland, King’s College London, Academisch Medisch Centrum and University of British Columbia are the biggest institutional ‘subscribers’
• We’re often asked about promotional material and we’re very close to getting some leaflets produced.  On our Pinterest account you can see the final designs.

Not a bad two months for Trip.

But, there’s no resting and other plans are taking shape:

• Clinician similarity is something I blogged about in 2012.  Fairly quietly I’ve been working quite hard on this and have recently received funding to work with the University of Glasgow.  We’re hoping to have initial results of that in the next 2-3 months.
• Reporting even earlier than that will be another project I’ve been working on – near instant reviews.  Trip funded phase one and we received a grant to work on phase two.  This is really exciting as the phase one results were so promising.  At the end of this phase we should have a tool for people to experiment with.  The principles are sound, the technology looks good but I can’t help feeling acceptance will be the hard part!

The above are the two main projects I’m working on.  But that leaves future projects and this is where the advisory board will help.  A few example projects that I’m keen to explore:

• Patient interface – very excited by this
• Mobile interface
• Better publicity
• Creating of a decent business model, which may include a freemium Trip
• Improved full-text access, improve our initial offering
• Further improve the timeline experience
• And a handful more speculative/spectacular ideas

Happy days!

It’s only been a few days but already lots of people and institutions are taking advantage of our link-outs to full-text.  Today we went through the 150 institutions barrier, which astounds me, well it would if I wasn’t so frazzled adding them to our system!

But we have some early adopters and so far the top institutions (by way of users signing up) are:

• University of British Columbia and NHS Scotland – both with 6 users
• King’s College London – with 5
• NHS Wales, University Hospitals Coventry and Warwickshire NHS Trust Library and Knowledge Services, Academisch Medisch Centrum,Barts Health NHS Trust, Macquarie University, Bond University, Leiden University Medical Center, University of Otago – all with 4

We’ve released a bunch of upgrades to the site, some really powerful others simply useful!

The screengrab below (click on it to enlarge) highlights the major changes.

Full-text links: We’ve used two methods for this.  Firstly, we’ve started cross-checking our records with PubMed Central and linking accordingly.  Secondly, we’re working with institutions to allow the easy linking between Trip and the institutions full-text holdings.  For this to work a user needs to alter they profile (via the ‘Setting‘ button), about half-way down there are a series of drop-downs, select your institution from there and it should work straight away.  If your institution is not there then send us an email (jon.brassey@tripdatabase.com) and we’ll tell you the simple steps needed.

DynaMed integration: Click on the DynaMed tab and you’ll see the results.  Access to the actual content is only available for those with subscription access – alas we do not provide that!

Controlled trials database: This has actually been out for a while, but I’m including it here as it was planned with the rest of these changes and is a fairly recent addition.  Click here for further details.

Case Reports: Working with BioMedCentral’s Cases Database we’re really pleased to see this interesting collection added to the site.

Developing World Filter: Working with a slightly modified filter from a Norwegian Cochrane site we have created a specific and sensitive filter. If you would like to know the difference then email us via the email above.

Minor changes

• Ability to delete items from the timeline
• Move from eternal scrolling on timeline to pagination
• Number each result
• Ability to change password

In February I posted an article discussing ‘the near instantaneous meta-analysis‘. In a nutshell – is there a way to very rapidly combine the results of multiple-trials?

Since then we have been quite busy working on this project, helped by some external expertise and a recent research grant.  Trip funded phase one, a proof of concept phase that allowed me to appreciate the challenges, limitations and opportunities that our approach presented.  The results were great and since then we have been awarded a grant to move forward to phase two.

Phase two will create a working model for people to use. This will be quite a simple solution and will only be aimed at synthesising placebo-controlled trials (more complex, comparator trials will form phase three).  The working model will work as follows:

1. User will use a modified search box telling us the condition and the intervention (e.g. acne and antibiotics).
2. Our system will search just the controlled trials portion of Trip to identify suitable trials.
3. We will then analyse these and present a score (more below on the scoring system).
4. We will then have an area that explains the results, how we arrived at them and the ability for the user to alter certain aspects.  This last bit is important as we’re relying on machines to ‘read’ the documents and extract pertinent information.  This is unlikely to be foolproof and while the system ‘learns’ it’ll need some feedback from users.  But, if the user does make alterations we will then re-analyse based on the updated information.

Quite simple really and steps 1-3 will take less than a second (we hope).

As for the score, that’s an interesting area and we’re sure it’ll change over time.  But the thinking at the moment is – what is most clinically useful?  After all, our audience for this will be practicing clinicians, not academics.  As such we’re thinking that an effect size is not particularly great/intuitive.  I really like the Clinical Evidence system for rating interventions e.g. ‘Likely to be beneficial’, ‘Unknown effectiveness’.

However, I’m also struck by the systems used by Amazon and TripAdvisor to rate articles.  A product/holiday is given an overall score but you can easily see how the score is arrived at.  When I use these I always look at the reasons people have given for 1 or 2 stars (ie people who have rated the item poor).  Whichever system we use we’ll make it very easy for users to differentiate good and bad aspects of an intervention.

Hopefully, phase two will be released in 6-8 weeks, probably not on broad release to start. This will be a gentle release, to a few people to start with.  This will allow us to alter the algorithms, allow for further machine learning etc.

I see this whole ‘instant review’ system taking a minimum of four phases.  Hopefully, if phase two works as well as we think it will, funding will follow to allow us to move to phase three.

In a meeting yesterday we discussed the concept of institutional ‘accounts’ with regard to our soon to be released full-text link-outs. In this scenario an institution (hospital, university etc) would give us some information to allow their users to link directly from Trip to the full-text.  For this to work the user needs to tell us they work for the particular institution.

But, once we have the information, can we be more useful?

Having all the evidence in Trip is great, but often local policies, guidelines etc are really important to consider. So, why not allow the user to search all the evidence in Trip but also be able to see if their institution has something to say on the issue.  It makes perfect sense to me.  But this is non-trivial and requires a number of issues to be dealt with:

• The institution needs to agree to this.
• How do we get local content into Trip.
• What format, web-based in easy but Word documents creates an extra level of complexity.
• Once we have the content in Trip, how do we maintain it being up-to-date?

The latter point is probably the biggest issue.  It’s fine to have initial enthusiasm to add content, but to go back every now and then and maintain up-to-dateness is a harder proposition. What happens if the enthusiast leaves?

That worry aside, doing a check every 3-6 months isn’t a big deal and we could arguably build some alert system saying ‘You have not checked your content for 3 months, please go and check now’.  I’m guessing we could even automate something that checks all the links every now and then.

One to ponder and to ask the advisory board about, when we get the latest upgrade out of the way.

UPDATE: One thought has struck me, why not create a spider to go and grab all the content on the site (the way Google does).  This would be no work for the institution but requires all documents to be on a public-facing website (so no intranet content).  Also, it grabs everything (e.g. see this MIT example) which might introduce some noise!