Trip Database Blog

2023 was a great year for Trip.

We continue to have a impressive impact on care around the globe, helping support millions of decisions with evidence-based content. This is Trip’s core function, as such, it’s ‘business as usual’. However, we have continued to improve the site as much as we can and 2023 was full of significant improvements.

First quarter of 2023: A real focus on quality improvement including the way we handle updates to content such as systematic reviews and ongoing clinical trials. We also started work on a major de-duplication effort and ending the quarter by refining our aspirations on quality.

Second quarter of 2023: This is where we really started to explore the potential of large language models (LLMs) such as ChatGPT. We also introduced scoring systems for guidelines and RCTs and a new guideline category for Europe. Finally, our de-duplication efforts saw 143,218 duplicate articles removed from Trip.

Third quarter of 2023: Our user survey results were published which is always interesting for us. We started exploring automated clinical Q&A using LLMs (see here and here). We also started our overhaul of the advanced search and started exploring some improvements to search relevancy.

Fourth quarter of 2023: We finished the year off strongly with the release on the new advanced search and the wonderful connected articles. We continued to push ahead with our LLM work with two main projects occupying our time (as well as a bunch of ideas bubbling away in the background). The two projects being a fully automated evidence review system and a clinical Q&A system. We also announced the creation of the AI Innovation Circle to help us to manage the AI potential/hype. Finally, we announced the main focus for the start of 2024: moving the Trip infrastructure to ‘the cloud’ and, unsurprisingly AI!

All this remarkable progress is supported by having a stable business model that has allowed us to remain independent and grow (as the site improves our income increases; a virtuous cycle)! While subscriptions to Trip Pro is the main source of income our bespoke rapid reviews have boomed in 2023. These are produced for a small group of selected organisations who appreciate our speed, evidence-based approach and cost-effectiveness. If you want to know more please email us.

Oh yes, don’t forget, we are still a ridiculously small organisation.

Another great boost for us is our users, they are wonderful and frequently crucial in helping us with surveys, feedback on new features etc. So, a big thank you from Trip to you.

2023 has proved immense, 2024 will hopefully be even bigger.

We import extra systematic reviews from a number of sources and we use special filters to identify them. It has come to our attention that one of the filters was too sensitive and therefore brought it a number of false positives ie tagged as systematic reviews and are clearly not!

So, we have started to clean these up and we’re currently removing 42,956 of them. Due to the method employed we’re also removing some true positives and therefore, once these are all removed, we will reimport these true positives! Convoluted, but the best way of getting this right.

I recently wrote about our plans for the next few months at Trip, this is already out of date! In the post I mentioned our work on automated reviews – this is still ongoing. However, we’ve also started an additional project – on Q&A.

Both projects are about generating full-automated POCs (proof of concepts). We’ve written about the automated reviews in the previous post. However, the Q&A work is a new one and will be based on retrieval augmented generation (RAG). One criticism of using LLMs (such as ChatGPT) is that they hallucinate. By using RAG you can, to a great extent, overcome these fears. RAG forces the LLM to obtain the answer from a predefined corpus of knowledge (as opposed to the knowledge within the LLM). In our case, for the POC, it will be using Trip’s clinical guidelines, systematic reviews and RCTs. If all goes to plan we should have a wonderful, evidence-based, Q&A system – one that can grade the answer based on the evidence used (taking into account risk of bias in RCTs, guideline scores etc).

Two final details:

• For both topics we are concentrating on the topic of migraine – a topic which isn’t too big or too small!
• We hope to be testing both of these in January

We’re quietly confident of both approaches but I suspect things will not be perfect!! As I said in my previous post on the topic – happy days!

Using the same mechanism as Connected Articles users can see related content for individual results on Trip.

Under each result on Trip you will see a related articles link:

If you hover over this link you’ll see the following:

If you click on this it’ll return the related content:

Nice and simple and it can be very useful 🙂

Now that we’ve got Connected Articles and Advanced search out and live we are working on two main areas of work:

Migrating to the cloud

Currently Trip sits on a server somewhere in the UK. However, this has some drawbacks and so we’re going to move our presence to the cloud. Given the complexity of Trip this is not easy but we can build it separately from the current Trip and then switchover when we’re happy it’s stable.

As part of this process we will be upgrading the underlying search system and rolling out ‘fixes’ to issues such as relevancy.

Artificial Intelligence

This is going to be interesting and there are two main avenues here:

• AI Innovation Circle: Even though we’ve hardly advertised this we’ve already had a great selection of volunteers. This group will help guide Trip in harnessing some of the amazing power that is now readily available.
• Automated reviews. Trip has a reasonably long tradition of exploring automated evidence reviews (e.g. Developing a fully automated evidence synthesis tool for identifying, assessing and collating the evidence) and more recently we’ve been experimenting with ChatGPT/LLMs (see More experiments with LLMs/ChatGPT). We’re about to take the experimentation into a proper working system. We’ll initially concentrate on a single condition and use that to learn before, hopefully, rolling out a something altogether more ambitious.

Happy Days 🙂

Advanced search is designed to give you much greater control of the search allowing more focussed results.

NOTE: Advanced search is a Pro only feature. If you’re fortunate to have access here follows a quick guide as to how to use it.

Above the search box on the homepage press the ‘Advanced’ tab and this will take you to a screen that looks like this:

There are 4 main elements:

• Search box – this is where you add you term or terms
• Where to search – this allows you to select where you want the search terms to occur. There are two options (1) Document text and (2) Title Only. The former is the default and if you want to search in the text of a document you can leave it alone. Also, note, that the document text refers to the text we have in our index, often this is full-text, but sometimes it might simply be the abstract
• Search type – this allows more Boolen functions and there are three options (1) All of these words – the default and effectively an ‘AND’ search (2) Any of these words and – an ‘OR’ search (3) The exact phrase – effectively places quotation marks around the added terms.
• Search – once you’ve created your search you press the search button.

See this example:

I have added the terms brexpiprazole Rexulti and selected ‘Any of these words’. I have left the ‘Where to search’ untouched. This effectively looks for any document in Trip that contains the word brexpiprazole OR Rexulti. After pressing ‘Search’ I see this:

As you will see it has created a line at the top of the search box, representing the first search. This is now #1. I will now do another search:

This time the search (#2) was a search for documents which have schizophrenia in the title. You can build up multiple lines of searching. The next aspect of the search is to combine the lines:

Here, I have selected the tick-box next to #1 and #2 and it has automatically populated the search box with the search. It defaults to an AND search but if you click on the OR or NOT buttons these are inserted e.g.

Reverting back to an AND search the results look like this:

The combination is now #3

A few things to note:

• You can manually add Boolean phrases, so you don’t need to use drop-downs or the ‘Boolean’ buttons. In other words you can simply type brexpiprazole OR Rexulti. Similar for phrase searching you can simply type “prostate cancer”.
• On the right-hand side of each search there is a bin symbol – this allows you to delete a single line of search. At the top there is a ‘Clear All’ link, this removes all the searches.
• To see the results for the most recent search simply scroll down. If you want to see a previous set of results, simply click on the hyperlinked number of results on the right hand side e.g. the 64, 22608 or 297.
• We default to showing the most recent 10-12 results. If you have done more and need to see them all, click on the ‘Show all’ link at the bottom of the list of searches.

Any questions or need for further clarification? Drop us a line: advanced.search@tripdatabase.com

We’re edging ever closer. As you can see from the latest screengrab we’re close:

The actual search works well just a few design issues we need to work out. But I’d say we’re 99% there 🙂

Trip is on the cutting edge, leveraging the transformative power of Artificial Intelligence, with a spotlight on sophisticated large language models (LLMs) like ChatGPT. You might have seen some of our recent forays: LLMs and Clickstream, LLMs again…. and More experiments with LLMs/ChatGPT.

As the AI landscape evolves at lightning speed, we recognise the significant potential of AI to help us deliver for our users. This is where the AI Innovation Circle comes into play – an informal group designed to strategically steer our AI journey to enhance user experience on Trip.

What will the AI Innovation Circle do?

• Harness AI’s vast capabilities to significantly elevate the Trip experience.
• Provide valuable insights to refine existing concepts, making them even more user-centric.
• Identify untapped opportunities where AI can drive innovation.
• Foster a collaborative learning environment to fuel collective expertise.
• Ensure Trip’s development are problem-led, not technology-led.

Engagement is not anticipated to be arduous and is likely to be managed via the convenience of email and the dynamic spark of occasional Zoom or Google Meets sessions.

Ready to be at the forefront of AI-powered transformation? Reach out to jon.brassey@tripdatabase.com to shape the future of Trip with us!

NOTE: The above was written by ChatGPT (including the group name) after I gave it a rough draft. Not 100% my sort of wording, but in the spirit of the technology I’m going to go with the flow!

Connected articles launched at the weekend and we’ve explained how it works here. But this blog is more about why we’ve introduced it and the benefits.

Every time you search and click on an article the system starts to ‘understand’ your interests. This is important as it can be very difficult to convey, via a handful of search terms, what your intention is. In search, user intention is vitally important. Two users might both search for the same thing e.g. prostate cancer screening yet one is interested from the public health perspective while the other might be interested in the best test to use.

However, while search terms might hide the intention user’s clicks quickly ‘reveal’ their actual intention by clicking on document that they feel might answer the question they have. So, a public health search might click on articles that discuss the cost-benefit of screening at a population level. While someone else might click on articles comparing PSA to DRE.

The Secret Sauce: Co-Clicks, Semantics, and Citations

So, what makes Connected articles so clever? Three words: co-clicks, semantics, and citations.

• Co-clicks are like the search engine’s version of “people who bought this also bought…”. We have a huge and unique ‘library’ of co-click data that we can use to help build up connections between articles (see previous blogs Structure in Trip and Ok, I admit it, I’m stuck).
• Semantic similarity finds documents that are semantically similar to the ones you’ve already clicked on. You click on an article about cost-benefit of screening at a population level, it finds other similarly worded articles.
• Citation data is the final piece of the puzzle. Our system looks at the references used in the articles you clicked and also looks for any articles that have citated the articles you clicked on. Using citation data is starting to take off (e.g. Citation analysis) and more specifically in the evidence synthesis world (for instance Citation tracking for systematic literature searching: A scoping review and Citationchaser: A tool for transparent and efficient forward and backward citation chasing in systematic searching)

Our system takes the data available from the above sources and combines them using a special algorithm to ensure the most closely connected appears top.

Ok, so why should you care?

• You’ll find articles your keyword search might have missed. You might have searched for atrial fibrillation but a closely related – and useful – article is on arrythmias. Different clinical terms but closely connected.
• You’ll save time. Results more focussed on your interests will mean fewer articles to look at to obtain the answers you need.
• Safety! Whatever search you’ve done – superficial or in-depth – Connected articles is a really useful tool to ensure you’ve not missed really important articles!

An example of using Connected articles

Using a search for urinary tract infections we clicked on three articles on a similar topic (can you guess what that might be?) below are the top 6 results, but you can scroll down through the results and see many more:

And the topic, I’m sure you can see it’s concentrated the results down to cranberry juice and UTIs!

Convinced? Curious? Sceptical? Give Connected articles a try and tell us what you think. Find an unexpected article that delighted or surprised you? Share it with us! We want you to be delighted!