Search

Trip Database Blog

Liberating the literature

Author

jrbtrip

Advanced search

I just had a really interesting conversation (with a long-term user), where the deficiencies of our Advanced Search were highlighted! In a way it was not a big surprise, we’ve known we’d need to improve it for some time. So, how to move it forward?

PubMed’s Advanced Search:

Is this the best advanced search out there? Any alternatives we should consider? Any particular problems with PubMed’s approach?

We’re listening

Pyramids are back

In the recent redesign we removed the evidence pyramids in the results panel, after some feedback we have just reinstated them:

The evidence pyramid concept is explained in this diagram and accompanying text:

Evidence pyramids are not perfect, for instance a poor systematic review is no match for a good RCT. But our evidence pyramids are useful in attempting to convey the potential level of evidence for each result. We use 5 levels in our simplified pyramid and they are colour coded to match the narrative description of the record (e.g ‘systematic review’ is green or ‘key primary research’ is dark blue) and that corresponds to the same colour in the filter results column:

In short, these are a simply tool to help convey the potential robustness of the evidence.

More guidelines

As you may be aware Trip has the largest dedicated collection of guidelines and it’s important to us that we continue to develop this valuable resource.

Following on from the new guidelines added in May we have just added a further 6 sites.

NOTE: If you find a guideline not in Trip then please let us know – it’d be great to hear from you.

Update: New developments on Trip

Just under a month ago we posted a list of four potential improvements to the site (click here for details), alongside an option for user to vote on which they’d like to see. The results so far are as follows:

I’m interpreting this as a clear vote for the first idea (improved search) and then the next two are roughly tied. So, how will this translate in to action? Here’s the plan:
  • Improved search: We have asked a tech/data team to come up with options for improving the sensitivity/specificity of the search. From there we will aim to implement the changes ASAP.
  • Search support: We’re really excited to see this do so well. We need to have a planning session to explore what this might look like, this will require our techie and design teams to come together. This is likely to be after we’ve rolled out the improved search.
  • Quality scoring: We have already done all of the background work on the guideline quality scoring and we can roll this out when we get developer capacity. After that we have already completed a quality scoring method for systematic reviews, we just need to spend some time verifying that.

All very ambiguous with the dates – as we still have the indexing to finish (see What next? for further details) but while we wait for developer time we’re working hard in the background to bring these to you as quickly as possible.

New developments on Trip

What next for Trip? After over a year if back end development we’re looking to the next set of developments for Trip. Below are our current ideas and we’d like some help on which to take forward and/or prioritise.

IDEA ONE – improved search sensitivity/specificity

Currently Trip allows two main ways of governing search sensitivity. The default (sensitive) is to search title and text. The more specific mode is ‘title only’ (matching search terms to words in the document title).

Problem – sometimes the default can be overly sensitive returning low relevancy results or, if using the ‘title only’ too specific.

Proposed solution – To introduce three levels of searching:

  • Title only
  • Title and full text
  • Title and abstract

The default could be to the mid-level ‘title and abstract’ (but we would introduce an easy mechanism to ‘show more’ or ‘show fewer’ results). We believe this will be a sweet spot of relevancy (for most users) and reduce the sometimes overwhelmingly high number of results.

IDEA TWO – start to use citation data

Document linkage can be very useful in easily identifying articles that are related to the ones you are interested in. We exploit these linkages – in the form of clickstream data – in our SmartSearch and related articles features. This is fine but is limited in certain respects.

Problem – identifying linked articles is problematic and means users have to look through many results to find articles similar to the ones they have found already. Users may give up after a page of results missing lots of really good articles on the next pages.

Proposed solution – to introduce citation data.

Citation data would dramatically increase our ability to link articles. This could enhance our SmartSearch feature. So, if a user clicks on articles #1 and #5 we could identify more, closely related articles, that are relevant to their search intention. Another use could be highlighting the relationship between RCTs and systematic reviews. So, if you find an RCT we could link it to systematic reviews it might have been used in. Conversely if you find a systematic review we can highlight the individual RCTs.

IDEA THREE – quality scoring

Problem – Trip is an EBM search engine and we like to connect users to the best available evidence. However, just because something is linked to from Trip it doesn’t mean it’s not without flaws.

Proposed solution – we currently, using RobotReviewer, automatically assess if RCTs are likely to be of low risk of bias or not. We have done most of the work to assess guidelines for how evidence-based they are. We have also started some preliminary work to assess systematic reviews for bias and could develop this further.

IDEA FOUR – search support

Currently Trip allows users to search and then restrict the search by evidence type, clinical area, year etc. But is this set-up in an efficient way and does it support users to find what they need?

Problem – users might not be able to find the documents they needs as quickly as possible.

Proposed solution (1) – introduce an additional category of filters based on clinical question type e.g. therapy, diagnosis

Proposed solution (2) – introduce a step-by-step search guide to really allow users to focus their results. It might have the following steps:

  • User types search terms
  • We potentially suggest additional terms and user finalises search
  • We ask if the user wants a particular type of evidence
  • We then ask if there is a particular recency they’re interested in
  • If it’s a query about diagnosis, therapy
  • If there’s a particular clinical area

At each step we show the number of results and when the user is happy we show the results. It’s sounds long-winded but it should be quick and should deliver really focussed results.

So, it’s now over to you to let us know what you think. Please use the poll below (or let us know any alternatives you may have):

Help us to make Trip better

The new site is out, getting positive reviews and is very stable – this is excellent. There are a few teething issues which we’re dealing with (95% of them have already been fixed). So, our attention turns to the next steps. The immediate area we’re working on is the indexing (as laid out here (with an explanation as to what indexing is)).

The step after that – which needs considerable planning – is to work on improvements to the site. To that end we want to try a new approach – to convene a number of small, online, group sessions with users. These are likely to be grouped by user type e.g. health professional and information specialist. These sessions would allow us to better understand how users access evidence, the problems that they encounter and also for us to ‘pitch’ our current ideas for improvements. The subsequent discussions should allow us to better plan the next steps.

As a flavour of our thinking, these are some of our current ideas:

  • Introduce citation data in to Trip.
  • Improve the search – introducing greater control over the sensitivity and specificity of the search.
  • Add more document grading e.g. guidelines and systematic reviews.
  • Create specific filters for topics such as therapy, prognosis and diagnosis.
  • Introduce a search wizard to allow an easy way to find the documents users need.

Love or loathe those ideas, here’s your opportunity to influence. Send an email to jon.brassey@tripdatabase.com if you want to take part.

Updating Trip

We have just finished manually updating Trip for this month.

Trip gets content via three main routes:

  • Automated via mechanisms such as APIs and RSS. Once set up this requires no effort from Trip it just grabs new content on a daily or weekly basis.
  • Semi-automated. A number of organisations supply us their new content via email (typically a .csv file) in the correctly formatted way. We then add this to the third way of updating content:
  • Manual. Over a week, always in the middle of the month, Trip manually visits around 300 sites to manually find new content added since the previous month.

This month we just uploaded 650 records from the semi-automated and manual method – the majority via the manual way. Of these 86 were dated from 2020 or earlier and the rest were from this year. Of the older one these were typically from us having to updated outdated URLs or if we found new sources of evidence. The records from this year covered a large number of sources e.g. NICE, AHRQ, IQWiG, WHO, NIHR, ACOG, NCCMT, FDA, EMA, Prescrire, PHE, PHW, SBU (so many initials)!

In addition we manually tag all articles, from 2021, if they’re useful for primary care (automatically tagging articles for primary care isn’t great). Some examples include:

While it’s hard work it’s always great to see so much wonderful evidence being added to Trip.

What next?

It’s been just over ten days since the new site went live. It has been remarkably smooth. There are a small number of issues but the site appears stable and there have been no critical issues. So, we now look to the future and this is a brief update on that.

Firstly, we have a list of around ten issues that need fixing. These have been prioritised and we’ll work through those as quickly as we can. Only a few of these I’d class as significant.

Secondly, we need to work on the indexing! Trip has three main parts:

  • The actual search component – the Trip brain, this is working wonderfully and is very modern (so, no updating needed)!
  • The interface – this is how people interact with the site. It’s the design, the interactive bits etc. It’s what we’ve spent the last 12 months re-writing.
  • Indexing – the search component needs to search content. This involves:
    • Publications being added, either manually or automatically. This is minimal information such as the title of the document, the URL and publication date.
    • Spidering – we send a spider (AKA webcrawler or a bot) to the URL and grab the content.
    • Processing – our system adds the content to the search component but also does a number of other bits and bobs e.g. tagging the articles by clinical content

Our indexing system is as old as the interface was (some bits being nearly twenty years old) so we’re completely rewriting this to make it better (quicker, cheaper etc).

Thirdly, planning the next upgrades to the site. We’ve got a few projects that are coming close to delivery e.g. guideline grading. However, there are some significant projects we’re thinking about and we need to decide which ones to take forward and for that we need to engage with our users. In the past we’ve relied on web-surveys and these have been really helpful. However, for this next step we’d like to have a more engaged process – one where we can have more of a two-way dialogue. To that end we’re interested in setting up a series of online small group calls. We’d briefly discuss our ideas and then engage with the group to unpick our ideas, what sounds good, what sounds bad and how we might implement these ideas. If you’re interested in taking part (one hour session) please let us know via development@tripdatabase.com.

Paradox of choice

I was delighted to recently receive an email from a prominent EBMer who declared himself a fan of Trip! However, his email was more about a teaching session he ran on EBM sources of quick answers and the feedback from the students.

He said the more EBM focussed the more they liked Trip however for others Trip was less favoured. He suggested this was because they had an extra cognitive load (in short, more effort) in selecting the resource most likely to be useful to them. Other sources (the likes of UpToDate) tend to give one main result – so there’s a lack of choice, search and click. For Trip its:

  • Search
  • Look at results (hopefully not having to look at too many)
  • Click

While that extra step seems fairly minor, clearly the more time-pressured you are the more of a big deal this is.

This is really important and interesting feedback. One for Trip to reflect on but if you have any thoughts, please let me know.

Blog at WordPress.com.

Up ↑