Given the recent interest in guidelines and Trip we’ve had a number of new conversations with users from the USA.  I’ll reproduce (well, slightly edited) one below as I think it’s broadly useful for others to understand a fairly common search problem. In this case the person highlighted how frustrating searching for guidelines can be, so here’s my response:

 

Firstly, as previously mentioned, guidelines are particularly problematic as they typically long and cover a broad range of topics. This creates two different problems – missing them and noise!

Missing them

So, you might have a guideline ‘The diagnosis and management of hypertension‘ which covers loads of areas. It might mention gestational hypertension as a chapter, but nothing more. If you searched for gestational hypertension this guideline would not feature highly as gestational is not mentioned in the title (our system favours matches in the title). So, it appears low down in the search results (even though that chapter might be the best result there is for the user). This is frustrating and we have a few things we’re going to try in the near future to improve this! But, your problem is more the noise…

Noise

Take one of your example search safety mechanical ventilation, when you search on the main Trip (so no refinement) you get the following results:

I’ve highlighted the scores – which relate to how relevant the document is to the search terms (BTW only I get to see those scores). But when you refine to USA Guidelines it gets noisy:

You’ll see, from result 3 the relevancy goes right down. In fact, only 2 (of 85 USA guidelines we return) look reasonable.

This comes back to the size of guidelines. They’re long and invariably the search terms appear within them – but they might mention safety, mechanical and ventilation all in different contexts. But as they include the terms they are counted as ‘hits’ and returned. Search can be stupid!!

The thing that we think we’ll introduce is a relevancy cut-off so we could say only return documents with a relevancy score above 0.1, 0.2 (we’d need to test) but allow users to ‘see ALL documents’ – if they want the noise!

Advertisements