Trip Database Blog

Now the recoding of the site is in the past we’re looking forward to adding new features to the site and improving the quality of what is already there. This latter was brought into focus by a strong, but fair, email about issues with Trip. Perhaps there was an element of complacency on our part (subscribers are increasing, we had the rewrite under our belt and little negative feedback). Perhaps we took our eye off the ball. So, it was a great email to receive – removing the complacency and reminding us that quality is really important.

So, what have we done and/or are doing in the short-term:

• Mentioned in the last blog we’re working on a better de-duplication system. There are a small, but significant, number of duplicate articles which we will soon be removing [Work starting next week]
• Dual categories – another blog post highlighting some articles are classified as, say, a ‘systematic review’ and ‘primary research’. This has now been fixed! [Fixed]
• Mis-classified results – linked to the above, some articles were appearing in the wrong filter. So, a user might select a ‘guideline’ and be shown a ‘systematic review’! [Work starting next week]
• Covid-19 – I really dropped the ball here! At the start of Covid-19 we tried to keep up with things, which meant some shortcuts were taken. To speed up the synonym aspect we hard-wired certain terms. This meant we avoided the main synonym function and resulted in early articles having their title words being double-counted – so early documents (particularly those mentioning ‘coronavirus’) were put at the top of the results. In the early days this wasn’t an issue but became problematic as time rolled on. This has now been fixed! [Fixed]
• Ongoing systematic reviews and ongoing clinical trials. Our results are ordered by 3 main elements (1) Relevancy (2) Date and (3) publication score. So, we had a situation where, due to the sheer volume of ongoing trials and reviews (all with high relevancy to the search terms and often many in the last few years) they had started to dominate the results. So, what we have done is to significantly reduce the other element (publication score) with a view to pushing these results down. This has required a complete reindex of the site and this should hopefully be finished early next week [Ongoing].

So, lots has happened and lots more happening next week. Including a visit to the senders of the email. I suspect I’ll have another few things to think about after that!

But, if you’re reading this and think there are other problems with the site please, please, please reach out to me directly jon.brassey@tripdatabase.com and let me know. We’ll hopefully make the site better for you and your fellow users…

After a prolonged battle we have finally sorted out our email system (switching to AWS). A major problem was that, while over 150k have registered for Trip, many of these are over ten years old and therefore likely to have changed. So, we needed a very intensive effort to weed these defunct ones out. So, now that’s done we can look forward to what’s next. The two immediate bigger jobs we’ll tackle are:

De-duplication: This is part of our commitment to improving the quality of Trip (as opposed to new functionality). We’re aware of a small, but significant, number of duplicate links, for instance:

As you’ll see, results 11 and 13 are the same article. This is hopefully an easy fix as we need to make better use of DOIs in the de-duplication process.

Guidelines: As we know, not all guidelines are great and we’ve been talking about this project for over three years (see original posts Dec 2019 and Jan 2020). Well, we’re hoping to finally start work shortly on this. As well as assigning a quality score for guideline publishers we will also be creating a new category for European guidelines. We are also discussing removing the country filters for free users….!

After that, the next big piece of work is improving our advanced search – in consultation most people favour Ovid’s approach to advanced search. While we don’t use MeSH we can still learn lessons from their approach. But this is in the medium term, so watch this space!

Many of you will have seen the hype around ChatGPT and other Large Language Models (LLM). ChatGPT uses artificial intelligence (AI) to generate an answer to various prompts (such as questions) and the answer appears written by humans and in real-time. After a huge amount of initial hype the hype has died down a bit, in part due to a number of issues were raised, mainly related to quality.

Irrespective of the problems we’re keen to have a play with this technology and as such we’re exploring using one such LLM called BioMedLM, a model specifically trained on biomedical data (in theory making it more accurate). The idea being that a user can ask a natural-language question and get an AI generated answer. Applying the trained model to just Trip data (as opposed to all of PubMed) should be an interesting experiment. At the very least the answers are more likely to be based on higher quality content than PubMed alone.

We’re secured the funding to explore this and we’re just lining up the team to deliver. Being an optimist I’d like to think we can deliver on that and, if we do, we’ll be asking for volunteers to test it….!

In the image below you will see that the user has restricted the category to ‘Primary research’:

So, you can understand their confusion when the top result is classed as a ‘Systematic review’. This is not the first time it has been raised and, as it’s a recurring source of confusion, we need to act!

The issue arises because we treat ‘Primary research’ to mean ‘Journal articles’. So, in the example above the systematic review was published in a journal – so it gets two categories and we display the highest.

So, our plan is to make sure that any dual category article (primary research and systematic review) is treated only as a systematic review. This means a stricter interpretation of ‘Primary research’ (excluding systematic reviews, which are secondary research).

No timeline on this but it’s in planning. And thank you to Adam T for raising it with us.