Connected articles is an amazing tool to unearth closely connected articles to the articles a user has already clicked. It uses three types of connection data:
Clickstream/co-click data
Citations (forward and backward)
Semantic similarity
For every article opened by a user we look for connections from any of the three sources mentioned above. These are then algorithmically combined and presented to the user. The results will look like this:
We’ve been working on the advanced search for a while now and it has undergone one round of testing in an un-designed format. Based on feedback we’ve arrived at something like this:
The bottom half is where you build individual lines of queries and these appear at the top of the page. You then combine them using the AND, OR and NOT commands.
After the recent post, asking for volunteers, we have had a number of users testing the advanced search. We have finished analysing these and most of them relate to design and usability. This is good news as it means things will probably move more quickly (as no massive debugging operation is needed).
So, we will shortly pass the feedback on to our designer and after that we can deploy. Hopefully this will be by early November.
This is really interesting. We’re busy looking at ways of using the power of LLMs to improve the user experience on Trip. One recent set of experiments relates to automated evidence reviews. While the following was semi-automatic it it is entirely possible to fully automate this. The review was exploring Brexpiprazole in schizophrenia and the overall report was three pages long and this included asking the system to generate both a conclusion and a clinical bottom line:
Clinical bottom line: Brexpiprazole shows efficacy in treating schizophrenia with most randomized controlled trials reporting it as ‘effective’, especially at non-low doses. However, its effectiveness compared to certain other antipsychotics can vary, and further studies are needed to assess its cognitive benefits.
Conclusion: Brexpiprazole’s efficacy in treating schizophrenia has been studied in several contexts. A systematic review from 2022 has raised concerns about the efficacy of low-dose brexpiprazole as it was not found to be superior to placebo or standard-dose brexpiprazole. However, another systematic review from 2023 suggests that while brexpiprazole may be less effective than a specific dose of lurasidone (160 mg), it generally demonstrates comparable efficacy and safety to both lurasidone and cariprazine.
Guidelines have noted some promising indications of brexpiprazole’s efficacy on cognitive performance in the context of second-generation antipsychotic compounds. Still, they have highlighted the need for further studies to accurately evaluate the extent of such benefits.
Randomized controlled trials (RCTs) have provided mixed but mostly positive feedback. The majority of the RCTs mentioned rated the effectiveness of brexpiprazole as ‘effective’. Notably, these trials reported consistent improvements in PANSS scores, maintained improvements over time, and a favorable side-effect profile. Only one RCT gave a ‘neutral’ rating, mainly due to mixed findings and potential bias introduced by the known side effects of an active reference.
Taking into account the weight typically given to systematic reviews and guidelines, the evidence suggests that brexpiprazole has some efficacy in treating schizophrenia, especially when not considering the low-dose. However, its effectiveness compared to some other antipsychotic compounds may vary, and further studies are recommended to determine its full range of benefits, especially in cognitive performance.
This is amazing technology and we’ll continue to test the capabilities.
We have just pushed out the revamped Advanced Search onto our test site:
We’re testing it in-house but are keen to get some external users to test it. So, if you are interested please contact me via jon.brassey@tripdatabase.com.
As mentioned in the previous post we have been spending a lot of effort trying to improve our search and the current focus (possibly obsession) is removing low relevancy results from the search.
TLDR long documents might mention the search term only once, in say 50,000 words. In that situation it’s almost an incidental result – but it’s still a true hit as it contains the user’s search terms even though it’s irrelevant to the user’s intention. One approach we have tried is to create a pseudo-abstract of guidelines – typically long documents – to see how that fared (by removing terms not linked to the core themes of the guideline). And here’s an example search taken from our testing site:
This image shows a search for diabetes and metformin and it returns 19 UK guidelines and the top results all look good. However, one was Guidelines for the investigation of chronic diarrhoea in adults. This contains the word metformin 1 time and diabetes 8 times in a 21 page document. So, another example of a result that is a poor match! This next image is when we searched just the ChatGPT summary:
3 results, so removing 16 results, including Guidelines for the investigation of chronic diarrhoea in adults. So, that’s good. However, it also removed Diabetes (type 1 and type 2) in children and young people: diagnosis and management, from NICE. In this 91 page document it mentions metformin 28 times. It is entirely feasible that a user, searching for diabetes and metformin, might think the NICE document was relevant!
Bottom line: Using the ChatGPT summary, as we have, means the search is too specific. So, on to the next approach….
Relevancy is a key element of search. A user types a term and the intention it to retrieve a document related to the term(s) used. But relevancy is relative and in the eye of the beholder. If someone searches for measles and the document has measles in the title, then it’s clear it’s relevant. But there might be another document, about infectious diseases, which has a chapter on measles. The document is 10,000 words long and has 50 mentions of measles = 0.5%. So, that seems a reasonable matche.
But what about a 100,000 word document, entitled prostate cancer, which mentions measles once = 0.001%. The document is a true match – as in it mentions the search term – but the reality is it’s clearly not about measles. Another example from a recent presentation I gave:
It’s a contrived example, but helps illustrate the issue!
For most searches this isn’t really a big deal as most of the time the top results will always be relevant. If the search returns 50 pages of results the low relevancy results will appear towards the end of the search – say from page 40. Not many people go to that results page – so it’s not an issue.
However, it is an issue when you have few results – either a very specific search OR if you click on a filter (eg UK guidelines) – then if 75% of the results are relevant and 25% poor – you can see some fairly poor results even on the first page. True hits as they contain the search terms but not really relevant to the user’s intention!
So, we’re exploring multiple options to help for instance An alternative search button? But another approach is to summarise long documents into shorter ones – so removing very low frequency words. We’ve experimented with ChatGPT and that summarised too much, so the search went from too sensitive to too specific. So, another approach is to do text analysis to explore word frequency (how often a word appears in a document) and remove those terms that are rarely mentioned (perhaps remove those terms only mentioned 1-3 times (depending on the document length.
We took one NICE guideline and analysed the frequency of words across the document and it looks like this:
The Y-axis denotes the number of times words appear in the document. With more granularity:
So, we’re going to run some tests where we remove terms mentioned 1 time, 2 times or 3 times (so, three separate tests). These don’t remove many terms but will hopefully remove the terms that cause problematic sensitivity. In the search example above, removing terms that appear once, would remove the term measles, while removing terms that are mentioned twice will remove prostate cancer.
This issue has been frustrating me for years so hopefully we’re edging closer to solving it!
After a user asked the Q about the differences I gave my answer – in a nutshell it’s quality over quantity. But then I thought, why not ask ChatGPT? The answer is below and it’s pretty good (but not perfect), a few observations:
It focusses on our Q&A role but not our literature search
Full-text doesn’t make a great deal of sense and underplays our offerings
Anyway, the answer it provided is below:
The Trip Database and PubMed are both search engines for retrieving biomedical and clinical literature, but they have some key differences in terms of scope, focus, user interface, and types of content they index. Here are some of the differences:
Scope and Focus:
PubMed: Primarily focused on research articles in the biomedical field, PubMed is a free search engine that primarily indexes MEDLINE, a large database of biomedical and life sciences journal articles.
Trip Database: Trip (Turning Research Into Practice) is geared towards evidence-based medicine and clinical guidelines. It aims to provide clinicians, researchers, and others with high-quality clinical evidence for decision-making.
Types of Content:
PubMed: Contains more than 30 million citations for biomedical literature, including research articles, reviews, case reports, clinical trials, and more.
Trip Database: Contains a more limited set of documents but includes different types of high-quality evidence such as systematic reviews, clinical guidelines, and other “pre-appraised” evidence. It also provides links to primary research articles.
User Interface:
PubMed: Offers more advanced search features like MeSH (Medical Subject Headings) terms, filters for article types, and more. It’s more suited for academic research.
Trip Database: Designed with a simpler, more user-friendly interface aimed at clinicians who need quick answers to clinical questions.
Access to Full Text:
PubMed: Provides citations and abstracts, and sometimes links to full-text articles, but many articles are behind paywalls.
Trip Database: Attempts to link directly to free full-text articles when available and offers a PICO search function to answer clinical questions.
Grading of Evidence:
PubMed: Does not grade the quality or level of evidence.
Trip Database: Often grades the quality of evidence and links to the highest level of evidence available (e.g., systematic reviews and guidelines).
Target Audience:
PubMed: Aimed at researchers, clinicians, and academics.
Trip Database: More targeted towards practicing clinicians who need to make quick and evidence-based clinical decisions.
Both platforms have their own unique strengths and weaknesses, and the choice between the two may depend on your specific needs.
Our current advanced search has been static for years and since then other advanced search systems have evolved and people have become comfortable with these newer approaches. Bottom line: our existing advanced search is a bit clunky! So we have prioritised work on this and here is the first look:
As mentioned early days, but you can see the direction we’re taking. I’d like to think we can have the proper version released in the next 2 months.
Trip Database Blog 
Recent Comments