Trip Database Blog

We import extra systematic reviews from a number of sources and we use special filters to identify them. It has come to our attention that one of the filters was too sensitive and therefore brought it a number of false positives ie tagged as systematic reviews and are clearly not!

So, we have started to clean these up and we’re currently removing 42,956 of them. Due to the method employed we’re also removing some true positives and therefore, once these are all removed, we will reimport these true positives! Convoluted, but the best way of getting this right.

I recently wrote about our plans for the next few months at Trip, this is already out of date! In the post I mentioned our work on automated reviews – this is still ongoing. However, we’ve also started an additional project – on Q&A.

Both projects are about generating full-automated POCs (proof of concepts). We’ve written about the automated reviews in the previous post. However, the Q&A work is a new one and will be based on retrieval augmented generation (RAG). One criticism of using LLMs (such as ChatGPT) is that they hallucinate. By using RAG you can, to a great extent, overcome these fears. RAG forces the LLM to obtain the answer from a predefined corpus of knowledge (as opposed to the knowledge within the LLM). In our case, for the POC, it will be using Trip’s clinical guidelines, systematic reviews and RCTs. If all goes to plan we should have a wonderful, evidence-based, Q&A system – one that can grade the answer based on the evidence used (taking into account risk of bias in RCTs, guideline scores etc).

Two final details:

• For both topics we are concentrating on the topic of migraine – a topic which isn’t too big or too small!
• We hope to be testing both of these in January

We’re quietly confident of both approaches but I suspect things will not be perfect!! As I said in my previous post on the topic – happy days!