Trip Database Blog

The questions are still coming in and we’re approaching 300, so I thought we could have a quick look at the types of questions!

Conditions: below is a list of conditions that the questions related to, the fact that diabetes, with 4, is the highest shows the large range of conditions asked about.

Diabetes: 4
Heart failure: 3
Hypertension: 3
Pregnancy: 3
Tetanus: 3
VTE (Venous Thromboembolism): 3
ASCVD (Atherosclerotic Cardiovascular Disease): 3
Duchenne muscular dystrophy: 3
Perinatal mental health: 3
Surgical site infection: 2
Stroke: 2
Type 2 diabetes mellitus: 2
Infertility: 2
Ventilator associated pneumonia: 2
Obesity: 2
Frozen shoulder: 2
Alzheimer dementia: 2
Asthma: 2
Coronary revascularization: 2
Tuberculosis (TB): 2
Low back pain: 2
Bowel cancer: 2
Cirrhosis: 2
Acute myocarditis: 2
Binge eating disorder: 2
Insomnia: 2
Venous insufficiency: 2
Pain: 2
Dilated cardiomyopathy: 2

Question type:

• Treatment/Management 70-80
• Evidence/Studies/Literature Review 45-55
• Diagnosis/Assessment 20-25
• Guidelines/Recommendations/Best Practices 15-20
• Drug Information/Mechanism of Action/Reviews 15-20
• Etiology/Causes/Mechanisms 10-15
• Prognosis/Outcomes 10-15
• Public Health/Prevention 10-15
• Basic Science/Pathophysiology 5-10
• Patient Experience/Qualitative Aspects 5-10
• Ethical/Societal Considerations < 5
• Other/Unclear < 5

Broad versus narrow questions:

• Broad questions: These typically cover a wide range of aspects related to a condition, treatment, or topic. They might ask for general overviews, multiple options, or the fundamental principles.
• Narrow questions: These focus on a very specific aspect, such as a particular drug, a precise diagnostic criterion, a specific patient population, or a detailed mechanism.
  Approximate Count of Broad vs. Narrow Questions:

Broad Questions: Approximately 60 – 75 questions seem to have a broader scope. These often start with phrases like “What are,” “Explain,” “Discuss,” “What matters to patients in their patient experience,” or ask for lists or overviews of a topic. Example of a Broad Question: “What are the core concepts in the primary prevention of ASCVD ?”

Narrow Questions: Approximately 150 – 165 questions appear to be more narrowly focused. These often inquire about specific treatments (“What is the best treatment for…”), particular diagnostic methods (“How to diagnose…”), the role of a specific drug (“How do SGLT2 inhibitors affect…”), or very defined scenarios. Example of a Narrow Question: “What is the correct dose for Meropenem in patients with hemodiafiltration?”

Fascinating….

Why Usage Speaks Louder Than Words

In some ways, I didn’t even need to read the beta tester feedback. Why? Because the most compelling evidence was in the behavior itself: users kept coming back. That repeated engagement spoke volumes – it showed the system was delivering real value and gaining meaningful traction.

Positive Feedback Highlights

But we did ask for feedback and it was broadly very positive, the headlines:

• 70% were health professionals
• Most asked 3+ questions
• Accuracy was deemed high
• The answers were deemed relevant and trustworthy
• Speed – 70% said ‘very fast’ and 30% said ‘reassuringly paced’
• 100% of health professionals would recommend the system to their colleagues

Here are a few standout quotes:

• Thanks for the opportunity – I feel a product from the Trip family has particular value given your history in information architecture and providing credible, evidence-tracked, healthcare information support
• It is very impressive to see the speed and capacity to extract and summarise data from evidence resources
• Amazing system – would use very frequently in clinical practice!
• Please continue this excellent initiative
• Honestly, overall the database is intriguing. It has a resiliency and foundation that lends itself to be far more trustworthy and clinically focused than most other databases. I see it also as a great tool to teach med students about building blocks of clinical reasoning and research.

What’s Next: Immediate and Future Enhancements

As well as the good there were lots of constructive feedback which falls into a number of stages of the Q&A process, with some examples of the issues:

• Initial question processing – when a user submits the question we need to do some processing to better disambiguate questions, for instance one Q we received was simply liver elastography.
• Answer creation – we need to better handle the search process e.g. send additional meta data, make the search more sensitive if too few result etc.
• Answer design – the way we include references was problematic for many but also there was a wish for an overall strength/weakness of evidence statement to be included.
• Answer placement – we need to add the Q&As to the Trip search index and to have systems in place to deal with duplications

All the above are seen as being ‘immediate’ action points, by that I mean these will be done before we roll this out as an open beta on Trip. There are some medium-long term improvements we need to do:

• Add extra content types eg drug information resources.
• Use location information – if the user is from the USA then favour American guidelines.
• For each Q&A give additional prompts for follow-up questions. In other words if a user asks What are the pros and cons of prostate cancer screening? We might suggest follow-up questions such as What is the best screening tool for prostate cancer? or What are the different mortality rates at various cancer stages in prostate cancer?
• Multi-lingual – allow users to ask Qs in their own language and get the answer back in their own language (see Apoyando el uso del idioma español en Trip Database.).

In conclusion

The beta test has been energising and insightful. With such a strong foundation and clear areas to build on, we’re more confident than ever that we’re creating something genuinely valuable for clinical decision-making. The next phase? Opening up the beta and continuing to learn, refine, and improve – together with our users.

As we begin rolling out AI features in Trip (the first being primary care new evidence summary and the second Mind Maps, with Q&A to follow) we need to consider how to do so sustainably. This includes encouraging more users to upgrade to Pro subscriptions and reviewing the pricing structure of those subscriptions.

To encourage free users to upgrade to Pro, we plan to limit the filtering options available to them. This will help further highlight the distinction between Free and Pro tiers. Here’s what that could look like:

We’re likely to increase Pro pricing and are considering a grandfather clause – allowing existing institutional customers to retain current pricing, while the new rates apply only to new customers.

We’re taking a considered approach to these changes, so please share your thoughts – we’d love to hear your feedback.

Over 30 people signed up to test our automated Q&A system, though it’s unclear how many actively participated. That said, we received over 200 questions – which averages out to around 7 per tester. Realistically, some asked just one or two while others were clearly more enthusiastic, which is a great sign that we’re on the right track!

We’re now moving into the feedback phase and have asked testers to share their views across several key areas:

• User characteristics: Confirmation of professional status
• Usage frequency: Number of questions asked during the trial
• Perceived accuracy: Subjective judgement of how well answers reflected the evidence
• Clinical relevance: Relevance of responses to the clinical scenario posed
• Trustworthiness: Level of trust placed in the answer content
• Responsiveness: Perceived speed of system response
• Answer format: Feedback on the structure and style of the response (e.g. narrative vs. quantitative balance, referencing)
• Likelihood of recommendation: Willingness to recommend the tool to colleagues
• Improvement suggestions: Opportunities to improve usability, content quality, or design
• Overall impressions: General feedback on value, potential for routine use, and any concerns

We’re a mix of nervous and excited – but that’s the whole point of testing. We know it’s not perfect, and with thoughtful feedback, we’re confident we can make it significantly better.

Let’s call it nervously optimistic.

The volume and breadth of questions is amazing. The beta test, while challenging, has been a wonderful experience. Below are some example questions and, for brevity, I have simply add the clinical bottom line (NOTE: many of these will have been answered with a corpus lacking the majority of our systematic reviews so many are likely to be improved)

Treatment for Basedow Graves?
Clinical Bottom Line: The mainstays of treatment for Graves’ disease include antithyroid drugs, radioactive iodine, and surgical intervention, each with specific indications based on disease characteristics and patient preferences. Emerging therapies, such as monoclonal antibodies, offer promising future options, particularly for refractory cases or those with significant orbitopathy. A personalized approach, considering safety and patient-centric factors, is crucial for effective management.

What is the best treatment for binge eating disorders?
Clinical Bottom Line: For Binge Eating Disorder (BED), Cognitive Behavioral Therapy (CBT) remains the gold standard, supported by substantial evidence affirming its efficacy in reducing binge frequency and improving associated psychological outcomes. Pharmacotherapeutic options, such as lisdexamfetamine and naltrexone/bupropion, offer additional non-psychological avenues, particularly benefiting patients where BED coincides with obesity. Ultimately, a multidisciplinary approach tailored to individual needs—incorporating psychological, pharmacological, and possibly family-based interventions—is optimal for the management of BED.

Is type 2 diabetes mellitus a disease? What is the diseased organ if it is a disease?
Clinical Bottom Line: Type 2 diabetes mellitus is a complex metabolic disease involving multiple organs, primarily characterized by pancreatic β-cell dysfunction and insulin resistance across various tissues such as the liver, skeletal muscles, and adipose tissue, leading to systemic metabolic consequences.

Causes of septal panniculitis?
Clinical Bottom Line: Septal panniculitis encompasses a broad spectrum of diseases, chiefly characterized by etiologies involving inflammation of fat septa, either with or without vasculitis. Recognizing these underlying factors is imperative for targeted therapy and effective management.

What is the current evidence regarding nighttime dosing of antihypertensives? What is the rationale? What are the key trials regarding this? What are the current recommendations?
Clinical Bottom Line: While nighttime dosing of antihypertensive medications shows promise in certain contexts, overall evidence remains inconclusive. Current guidelines do not recommend routine switching to nighttime dosing, favoring individualized approaches based on patient needs and comprehensive BP management strategies.

In adult patients who have heart failure does providing early discharge teaching instead of current practice improve the 30-day readmission rate?
Clinical Bottom Line: Implementing early discharge education, particularly using interactive methods like the teach-back approach, significantly reduces 30-day readmission rates in heart failure patients compared to standard practice. Tailoring educational interventions to individual patient needs within a structured, multidisciplinary patient discharge plan is recommended to optimize outcomes and enhance self-management. Careful adoption of proven methods into practice will ensure better patient engagement, adherence, and overall health improvement.

Is niacinamide effective for primary or secondary prevention of skin cancer?
Clinical Bottom Line: Nicotinamide is beneficial for secondary prevention of non-melanoma skin cancers in high-risk patients, while its role in primary prevention warrants further investigation.

The Q&A beta testing has been going for just over a week. After 24 hours I described he trial as a rollercoaster, that is still the case and the trial is currently paused – hardly ideal.

The pause was triggered by informal feedback from a user who had asked about strength training for knee osteoarthritis. They felt the response was poor and lacked key references. Curious, I ran a quick search on Trip for strength training AND knee osteoarthritis and found plenty of systematic reviews. So why did the Q&A system miss them? It turned out the system was searching the free version of Trip, not the Pro version. And that’s a crucial difference—because the free version is missing nearly half a million systematic reviews, including those vital to answering the question properly.

It’s definitely a setback, with potential implications for many of the Q&As the system has previously answered. That said, this is exactly what beta testing is for – identifying issues so we can improve the final version – we’ve uncovered a major flaw and we’re already working on fixing it.

There are plenty of positives too. We’ve had well over 100 questions submitted (a sample are shown below), which suggests testers are coming back and engaging with the system – an encouraging sign that they like it. Many of these questions likely weren’t affected by the missing systematic reviews. Plus, a number of Q&As have been externally reviewed, and the quality of the answers remains strong.

Hopefully, the systematic review issue will be resolved today, allowing us to re-open the beta. Then, we’ll move into the user feedback stage next week.

Will the rollercoaster ever stop?

A sample of the questions we’ve answered:

🧠 Mental Health
– Do different age groups have different outcomes with rTMS for depression?

– Can GLP-1 drugs increase the risk of suicidality and self-harm in people with diabetes or obesity?

– Does magnesium supplementation help improve sleep?

❤️ Cardiology & Blood Health
– SGLT2 inhibitors: Do they reduce heart failure mortality?

– What’s the target blood pressure according to ESC 2024 guidelines?

– Low-dose aspirin for primary prevention: What do the latest guidelines say?

– Are there benefits and risks to long-term anticoagulation for VTE prevention?

🦴 Musculoskeletal & Rehab
– What’s the best evidence-based treatment for frozen shoulder?

– Strength training vs. other treatments in knee osteoarthritis

– Diagnosing a syndesmosis ankle injury

– When can kidney donors return to normal activity after surgery?

🧬 Endocrinology & Metabolism
– Is type 2 diabetes a disease — and what organ is affected?

– What cholesterol changes are caused by Actemra?

– Does endometriosis cause or result from infertility?

– Zoledronic acid: How it works and its role in treating osteoporosis

🧒 Paediatrics
– Best practices to prevent surgical site infections in post-op pediatric abdominal surgery

– Are there any unique Tdap booster recommendations for adults in contact with infants or who are pregnant?

🦠 Infectious Diseases
– Managing iatrogenic UTI caused by MDR Klebsiella

– Tetanus booster after Td: Is Tdap now recommended, and what’s the schedule?

🧘‍♂️ Complementary & Lifestyle Medicine
– Does magnesium help with sleep or muscle cramps?

– How does the patient experience shape care in primary care settings?

🧪 Pharmacology & Guidelines
– What are the effects of Actemra on cholesterol?

– How do different anticoagulation durations compare in terms of safety and efficacy?

🧠 Neurology & Imaging
– Predicting hospital stay with the Modified Rankin Scale

– What conditions (other than pneumothorax) show a lung point on ultrasound?

– How to conduct a peripheral neurological assessment

🧬 Genetics & Rare Conditions
– Are people with Ehlers-Danlos Syndrome at increased risk of periodontitis?

🩺 Kidney Health
– What’s the living kidney donor work-up process?

– How long is recovery and medication use for recipients of kidney transplants?

– What are the surgical risks of living kidney donation?

Wow, what a 24 hours!

It’s been intense – in a good way.

Nearly 30 people signed up to test the Q&A system (I was hoping for 10!), and we’ve already answered 20 questions, including:

• Do different age groups have different outcomes with rTMS therapy for depression?
• What are best practices for surgical site infection prevention for post operative pediatric abdominal surgery?
• How do SGLT2 inhibitors affect heart failure mortality?
• Hyperthyroidism causes dilated cardiomyopathy through which mechanism?
• Please list any studies that use the modified Rankin scale to screen for length of stay in hospital that have a p value less than .05

A really interesting and challenging range of questions!

The system did break for a couple of hours (of course!), but it’s now fixed. The issue was one of three we’ve had to deal with, to improve performance – so even the hiccups was part of progress. With over a week of testing still to go there is still time for a lot more excitement/feedback…!

The only downside? I won’t know how accurate or useful the answers were until feedback comes in—and that might take a little while. But I can’t wait to find out.

We’re excited (and a little nervous!) to share something we’ve been working hard on. After resolving many of the early teething issues, we’re now ready for real-world testing — and that’s where you come in.

Our new system is designed specifically for health professionals, allowing you to ask any medical question — from the straightforward to the complex — and receive fast, evidence-based answers. We’re now looking for users to really put it through its paces. If you’re interested in trying it out, please get in touch: jon.brassey@tripdatabase.com.

We’d love your feedback on accuracy, speed, format, use of evidence, and anything else that could help us improve.

Below is an example Q&A – What are the pros and cons of prostate cancer screening? – it took 25 seconds to generate, so it gives you an idea of the type of output to expect

Clinical Bottom Line:

Prostate cancer screening, particularly PSA testing, presents a balanced proposition: it can lower disease-specific mortality and catch cancers early, yet it carries significant risks, including overdiagnosis and overtreatment. Decisions should be individualized, factoring in the patient’s risk profile and preferences, while ensuring informed discussions. Ultimately, the harms and benefits need careful evaluation to make appropriate clinical choices.

Answer:

Pros and Cons of Prostate Cancer Screening

Prostate cancer screening, particularly through prostate-specific antigen (PSA) testing, has been a subject of extensive debate. The evidence from several studies and guidelines highlights the following benefits and drawbacks:

Pros

1. Reduction in Prostate Cancer-Specific Mortality: Screening can lead to a significant reduction in prostate cancer mortality. The European Randomized Study of Screening for Prostate Cancer (ERSPC) showed a 21% reduction after 13 years of follow-up (Canadian Urological Association, 2022).
2. Early Detection: PSA tests enable the early detection of both clinically significant and insignificant prostate cancers, allowing for earlier intervention that might be curative (National Cancer Institute).
3. Reduction in Metastatic Disease: Screening decreases the incidence of metastatic prostate cancer, potentially reducing its physical and psychological burden ([Ilic et al., 2018]).
4. Refinement with Risk Stratification: Implementing tests like the 4Kscore as a reflex can considerably reduce unnecessary MRI and biopsy procedures by predicting the aggression of detected cancers (GÖTEBORG-2 Trial).

Cons

1. Overdiagnosis and Overtreatment: A considerable proportion of screen-detected prostate cancers are clinically insignificant, leading to unnecessary treatments and associated side effects such as impotence and incontinence (IQWiG).
2. Harms of Treatment: Treatments may lead to lasting side effects, impacting quality of life significantly. Radical prostatectomy and radiation therapy are linked to higher rates of urinary incontinence and sexual dysfunction ([BI]).
3. Psychological Impact: The anxiety associated with false positives, biopsies, and even a diagnosis of indolent cancer can have significant psychological impacts (National Cancer Institute).
4. Uncertain Benefit on All-Cause Mortality: Despite some decrease in prostate cancer mortality, the overall mortality benefit remains unclear, with no significant impact on all-cause mortality (IQWiG).
5. False Positives and Biopsy Complications: The high false-positive rate with PSA testing leads to unnecessary biopsies, which have risks such as infection and significant patient distress (ACP, 2013).
6. Implementation in the Elderly: The benefit of continued PSA screening in older men is contentious, with a considerable risk of overdiagnosis without significant mortality reduction (Pubmed).

Considerations for Decision-Making

• Individual Preferences: It is essential to involve patients in shared decision-making. Their individual risks, values, and preferences should guide decisions about whether to pursue screening ([BI]).
• Complexity of Screening: Screening-related decisions are complex and involve balancing potential life extension against the risk of significant harm and reduced quality of life.

This morning, we released the fully automated Q&A system internally. We’ve already asked it a number of questions, and responses are coming back in 10–30 seconds, which is promising.

That said, it hasn’t all been plain sailing—maybe my expectations were a bit too high. Two main issues cropped up:

1. Format – The answers didn’t look great. They lacked flow and polish. Thankfully, this is a relatively easy fix.
2. Content – Moving from the web interface of the LLMs (e.g., ChatGPT) to their APIs (so we can access the data without visiting the site) and then ‘stitching’ all the steps together, introduced some problems. In previous tests, we didn’t have a single bad answer. This new system, though, delivered a few that just weren’t up to scratch. I don’t mean disastrously wrong – just not good enough.

We’re now working through these issues, refining the prompts (the instructions we give to the LLMs), and tightening things up.

It’s a bit disappointing not to get it right first time, but we don’t think the fixes will be too onerous. The next version should be much stronger—and hopefully ready for external testing soon.