Trip Database Blog

Our previous post highlighted the fact that many of our top searches are for single concepts e.g. asthma, pregnancy, aspirin. In some situations this sort of search is fine but in others it might be imprecise.

If you use simple searches can you answer the following question please..:

If there is another reason, not listed above, please let us know. This can be done via comments or email: survey@tripdatabase.com.

UPDATE: please take our simple poll to help us understand simple searches (if you do simple searches)

One observation we’ve made over the years has been that many of the top searches (by frequency) are for single concepts e.g. asthma, diabetes, aspirin. This seems quite non-specific as the user possibly doesn’t want to know about, say, asthma generally. Perhaps they want to know about asthma diagnosis, asthma in children etc.

At Trip we want to help users get better results faster and we could take the above issue and try to solve it using techniques such as query expansion. However, the fact is we don’t fully understand the issue and we’ve suggested a solution (query expansion) straightaway! I have said the search is non-specific and the user doesn’t want to know about asthma generally. But there are many possible reasons for using such broad terms, for instance:

• They just want to get a feel for the literature
• They are new to the site and want to experiment with using the site
• They are not confident in creating searches that mirror their intention e.g. they may be wanting to know what is the most appropriate diagnostic tests for asthma in children and are not sure how to make a suitable search

There are probably other reasons as well – not really sure. So, leaping into a solution seems foolish. So, we should ask our users! My idea is that for a set number of searches (say every 10th search) we have a pop-up to take the user to a questionnaire to better understand their reasons for using Trip and if we note the search terms used we can cross-check that and see if there is a problem and from there what might the solution be.

I even asked ChatGPT to suggest some Qs and it came up with these:

Q1 Search Intent Understanding:
“What is your primary objective when conducting a search with us (e.g., browsing for general information, seeking specific data or studies, looking for treatment guidelines)?”

Q2 Search Term Selection:
“How do you decide on the search terms to use? Please select all that apply: a) Based on the specificity of the information needed, b) Using terms familiar to me, c) Repeating terms from educational or professional materials, d) Other (please specify).”

Q3 Experience with Search Outcomes:
“How often do the search results meet your expectations in terms of relevance and specificity? Always, Often, Sometimes, Rarely, Never.”

Q4 Challenges in Formulating Search Terms:
“What challenges, if any, do you encounter when deciding on which search terms to use? Lack of knowledge on the topic, uncertainty about which terms will produce the best results, other (please specify).”

Q5 Interest in Search Assistance Features:
“Would you find it helpful if the search engine offered suggestions or guidance on refining search terms to improve result specificity? Yes, somewhat helpful, No, not necessary.”

These are really interesting questions (thank you ChatGPT) and I think that scrolling through the results will be incredibly useful. So, when we roll out the survey (probably in the next month or so) please consider completing it.

A simple tip to help you get results from one publisher. To start with do a search, this is one for prostate cancer, and the results look like this (with snippets turned off):

Note that the tope results are all from different publishers! To restrict to articles just from Cancer Care Ontario, simple click on the publication name:

And the results look like this:

We hope this was helpful and if you’d like any more insights as to how to get the best from Trip then leave a comment or email me: jon.brassey@tripdatabase.com.

The Internet Archive is great and I found the first ‘grab’ of the Trip Database here. It lists the 25 publications we covered back then:

Very few are still going, I think only these are still operating:

• Cochrane
• Evidence-Based Medicine
• SIGN

In those days these sites contributed around 1,100 links and search was by title only.

We mentioned the possibility of a new LLM project improving the way we present the latest evidence to our users. In that post we mentioned asking our designer to mock something up and the first draft is below. Some things to consider:

• It’s early days so should not be considered final
• The content is for the clinical area of primary care
• If we go ahead with this project it can be full-automated or we may seek clinical area specialists to act as editors, both routes have advantages and disadvantages

Let me know what you think, either in the comments or directly: jon.brassey@tripdatabase.com. NOTE: We have had to ‘cut up’ the single, large image of the mock-up, into smaller images – so the blog displays them at a decent size….

We’re undertaking a rapid review to answer the client’s question “Is there high-quality evidence to support the use of EMDA MMC in combination with BCG therapy for non-muscle invasive high-risk bladder cancer (NMIHRBC)“. It’s an interesting question and I thought it might be nice to show how we use Trip to help answer the question.

Understanding the Q: EMDA = ElectroMotive drug administration, MMC = Mitomycin C and BCG = BCG vaccine.

Initial search: I keep things simple and adjust if the initial search is problematic. In this case I searched (EMDA OR electromotive) AND (MMC OR Mitomycin) AND BCG AND (“bladder cancer” OR NMIHRBC). In this there are 4 elements:

• EMDA OR electromotive – covers the first part of the search
• MMC OR Mitomycin – using the abbreviation and the term ‘Mitomycin’. I didn’t add the C as it doesn’t seem necessary, in this case. If we got lots of results I could always replace ‘Mitomycin’ with “Mitomycin C” (note quotation marks to use it as a phrase search)
• BCG – As above there seems no real need to add the term ‘vaccine’
• “bladder cancer” OR NMIHRBC – bladder cancer might seem to be too vague (why not search for the fuller term ‘non-muscle invasive high-risk bladder cancer‘? Well, again, if it’s problematic – and we got too many results – I could always add it to make the search more specific

This is what it looks like via the Advanced Search:

So, the search generates 24 results. I clicked on 14 articles that looked particularly relevant and then used the Connected Articles feature (BTW see our video explainer of Connected Articles) to reveal closely connected/linked articles. Here are the top results:

In total Connected Articles returns 100 results and even at the bottom many seem relevant:

Connected Articles goes beyond the Trip content and is great for helping minimise the chance of overlooking important articles. I need to work through these 100 to look for supplementary documents to enhance the review.

So, the above is a nice example of using Trip to highlight some highly relevant documents including guidelines from the European Association of Urology, some documents from NICE and Cochrane, as well as some other systematic reviews.

Our guideline scoring system is nearly a year old and it’s been well received by all types of Trip users. Over time we add new guideline producers and these scores are added to the system.

We are in the process of adding ten new guidelines producers and it’s really disappointing to see most of them scoring really low! As a reminder this is how we score guidelines (at the publisher level, not individual guideline level):

• Do they publish their methodology? No = 0, Yes = 1, Yes and mention AGREE (or similar) = 2
• Do they use any evidence grading e.g. GRADE? No = 0, Yes = 2
• Do they undertake a systematic evidence search? Unsure/No = 0, Yes = 2
• Are they clear about funding? No = 0, Yes = 1
• Do they mention how they handle conflict of interest? No = 0, Yes = 1

Of the ten new publishers we could find no information on their methodology, most didn’t use GRADE and we have no idea about the search. Funding is most frequently mentioned and their is no way of knowing how they deal with conflict of interest. A number highlight potential conflicts but not how this is managed – in which case the publisher gets a zero score for that element.

It does make me wonder how users of the guidelines feel? I think the answer is, and this is a worry, that many won’t care or consider the implications. How can you be confident that the guideline you’re using has been produced in an unbiased manner? In short the answer has to be faith, faith in the publishing organisation. This authority bias is problematic for many reasons.

In short, guideline publishers, please be more transparent with your processes, it shouldn’t be difficult…!

EDIT: Mock-up (mentioned below) is available to view here.

Trip adds loads of great evidence every month and this idea relates to better presenting this new evidence for users. We have a current way of doing this, which is to show it as a set of search results:

Above is the latest evidence for Primary Care!

We’ve been wanting to improve this for a long time but, with the advent of LLMs, the time might be right! So, instead of simply listing them as a set of results we could, for clinical areas (such as primary care, oncology, gastroenterology) create specialist ‘journals’ for each topic e.g. Trip Oncology Insights or Trip Clinical Evidence Review: Cardiology.

For each clinical area we would select 20-30 of the best/most important articles that month and summarise them to allow for easy reading (with link-outs to the full-text) with the option of seeing all the remaining articles via the search results method or possibly some visualisation like this to allow for easy identification of articles in your particular interest:

I’ve asked our designer to mock up what this might look like, I’ll share when we have that. EDIT: Mock-up is available to view here.

At Trip, we continuously update our database with thousands of new articles each month, prioritizing the inclusion of high-quality evidence such as guidelines and systematic reviews. Our ‘Latest’ feature is designed to present these recent additions in a user-centric manner, tailored according to individual profiles. This customization occurs in two primary ways:

• Clinical area: For users with specific clinical interests noted in their profiles, such as Gastroenterology, we curate and list the most recent, relevant evidence in that field. (Click here to explore the latest articles in Gastroenterology).
• Personalised Topics: Users have the option to highlight particular topics of interest within their profiles. Based on these selections, we actively search for and recommend new articles that align with these specified interests, such as research on bipolar.

To ensure our users remain informed of the latest evidence, we distribute a monthly email containing a direct link to these updated resources. This link is accessible at any time, allowing users the flexibility to explore the latest findings at their convenience. While this feature is frequently utilized and appreciated by our community, we believe there are opportunities for enhancement to further enrich user engagement and satisfaction.

At the top is a list of 5 ‘key’ documents and below that a list of recorded search terms and/or clinical areas of interest. If you click on a link it takes you to a page of search results – the results being a search for the topic for content added that month:

Recognizing the current approach as ‘sub-optimal’—a diplomatic term for inadequate—we are in the process of re-evaluating this feature. It’s evident that users have a strong desire to access the most recent evidence related to their specific areas of interest, and we acknowledge the need for improvement in meeting this demand. We highly value your feedback and encourage you to share your insights and suggestions either through the comments section or by directly emailing me at jon.brassey@tripdatabase.com. This initiative is part of our ongoing commitment to enhance user experience and optimize our services for better search engine visibility and user satisfaction.