Barb, one of our volunteers on our Twitter accounts, commented a while ago about seeing some strange results on Trip, so I asked her to send any news ones she found to help me understand what was going on.  She was looking for new articles in Trip that are returned for the search ‘immunisations’.  Many were fine but a few weren’t, for instance:

  • Multiple sclerosis: management of multiple sclerosis in primary and secondary care
  • Health visiting
  • Economic Evaluation of Complex Health System Interventions: A Discussion Paper
  • British Guideline on the management of asthma
  • Developing and Evaluating Methods for Record Linkage and Reducing Bias in Patient Registries

Now, these are not specifically about immunisations but they’ll all make reference to it.  For instance the top result has the following:

Vaccinations
1.4.2 Be aware that live vaccinations may be contraindicated in people with MS who are being treated with disease-modifying therapies.

We return all the results that match the search terms (and/or synonyms).  However, our algorithm is designed to emphasise those results which are more relevant.  So, ordinarily, if you do a search with lots of results the relatively irrelevant results don’t appear (well they do, but not till way down the results).  However, if you look for things with few results (perhaps an unusual condition or you heavily restrict the results) you are more likely to see ‘strange’ results.

So, what can we do? I see three options:

  1. Leave it ‘as is’ and hope people don’t get put off by the occasional result they find strange.
  2. We allow users to set a relevancy cut-off themselves.  Each search result gets a score from 0 to 1 (with 1 being very relevant) and every result that matches the search term gets at least 0.0001 and therefore can be shown in the results.  We could give users a ‘slider’ to allow them to chose what cut-off they want, So some might chose 0.1 while others might chose 0.3.
  3. We effectively borrow a concept from PubMed’s Clinical Queries which has a narrow and broad search.  The narrow search returns fewer results, they’re more relevant but you may miss a few (it’s a specific search) while the broad search gets more results but more irrelevant results (it’s a sensitive search).  So, in effect, Trip currently does a highly sensitive search.  You can see the effects in PubMed for a broad and narrow search for prostate cancer screening:

 My ‘gut’ instinct is the third option.  We, at Trip, experiment to try and arrive at a reasonable relevancy cut-off which is introduced by default on all searches. On the result’s page we highlight that the search is narrow and to make it broad simply press a button.

Feedback please and thank you – again – Barb for the input 🙂