Trip Database Blog

Estará disponible pronto — qué tan pronto depende de cómo vayan las pruebas (It’s coming soon – exactly how soon depends on the testing!)

Over the weekend we started to rollout the RCT score:

One thing you might spot is that there are two different types of scores. The newer score are the top two and the older Risk of Bias score at the bottom. This will be in place for a few weeks as we overwrite the old method of scoring with the new one!

Each RCT will see a scale like this:

And if you click on the question mark on the right-hand side you’ll see a pop-up explainer:

As per previous scores, the use of scores is not without criticism – but clearly we feel it’s worthwhile – here’s an old discuss on the topic. Also, a significant drawback is that it’s based on abstracts. But the rationale is not to do a full critical appraisal but to help highlight potential problems with the trial. The user is then free to do a full appraisal.

Because AskTrip is still new, we’re actively reviewing all the answers it generates. When we find responses that fall short—not due to a lack of evidence, but because of verifiable process issues—we’re logging them.

Two recent examples include:

• A user requested references for a specific question, but the system generated fictitious citations (a case of AI hallucination).
• A study involving non-human subjects was cited without clearly indicating that it wasn’t a human study.

By tracking these problematic cases, we hope to identify patterns and ultimately improve the service.

If you ever come across an answer you believe is problematic, please let us know by emailing quality@tripdatabase.com

While much of our attention is focused on AskTrip we are still working on other aspects of Trip and one thread is introducing a score for RCTs. We already have a risk of bias score but it has been increasingly difficult keeping this up to date. So, time to do our own in-house one. It will look like this:

It looks like our existing scores for guidelines and systematic reviews but will be powered by LLMs. And, as with the systematic review score, it will be based on the trial abstract.

Take this trial: Occupational therapy improves social participation of complex patients discharged from hospital: results of a powered randomized controlled trial. Using our prompts we were able to generate an overall score of ‘good’ (this is just our in-house terminology) and we generated these strengths and weaknesses:

Strengths

• Clearly identified as a randomized controlled trial (RCT).
• The aim follows the PICO format: Population (complex patients post-hospital discharge), Intervention (experimental occupational therapy plus standard rehab), Comparator (standard rehab alone), Outcome (social participation).
• Participants are described with inclusion criteria and demographics.
• Randomization is mentioned, indicating attempts to reduce selection bias.
• The intervention and comparator are clearly described, specifying the type and duration.
• Outcomes are reported with specific measures (Reintegration to Normal Living Index, COPM) and include statistical results (mean changes, confidence intervals, p-value).
• Study is registered on ClinicalTrials.gov, suggesting transparency and planning.

Limitations

• No mention of blinding, which could introduce performance or detection bias.
• Sample size is relatively small (92 participants), which may limit generalizability.
• No explicit mention of drop-out or loss-to-follow-up rates, which is important for assessing attrition bias.
• Harms or side effects are not mentioned, which is critical for evaluating the safety of the intervention.
• There is no information on funding or potential conflicts of interest, which is important for identifying bias.

When we roll this out, the above text will be available when a user clicks the question mark (far right on the image).

As per previous scores, the use of scores is not without criticism – but clearly we feel it’s worthwhile – here’s an old discuss on the topic.

This will be a gradual rollout starting with the most recent trials first.

At the top of this post I mentioned that this wasn’t directly linked to AskTrip, the reality is that it can be. In the medium term, we can factor in the quality of guidelines, systematic reviews and RCTs when answering a question. That would be rather special!

Maybe not lots of fun, but I enjoyed doing it! At AskTrip we automatically assign clinical categories and also the strength of evidence used to answer the question. Put these together and you can – fairly – easily see what evidence is used to answer the questions.

I restricted it to a handful of the clinical areas where we had lots of questions (50+) and I’ve plotted it using two graphs (not sure which is best)

What we can see is that the cardiology questions we received we able to be answered with the most robust evidence (rating of high) and that was 61% (surgery was the worst with 36%)

Surgery and oncology are tied – with 44% – of the questions being answered with lower quality evidence (rating of limited or moderate) with cariology the best 22%

I called it fun as there are all sorts of methodological issues with the analysis – so take it with a pinch of salt…

AskTrip launched just over two weeks old and we’ve already had over 600 questions – it’s been brilliant…. However, we’ve recognised two changes we’d like to make.

Spanish Language

We’re developing a Spanish-language version of the site, enabling users to ask questions in Spanish and receive answers in Spanish. To support this, we’ll duplicate the existing site and translate all content, including previously asked questions. If the launch proves successful, we plan to expand the platform to support additional languages. (see our earlier post Apoyando el uso del idioma español en Trip Database).

Limited Answers

We rate all answers based on the strength of the evidence used — High, Good, Moderate, or Limited (click here to understand our approach). Here’s the current breakdown:

636 total Q&As

• 300 – high (47%)
• 102 – good (16%)
• 139 – moderate (22%)
• 95 – limited (15%)

That means over a third of the questions have little supporting evidence. Interestingly, in the early days of manually answering clinical questions, clinicians often found it reassuring when no evidence was available – it confirmed that their uncertainty was valid.

Now, we’re exploring two ways to uncover more evidence:

• Broaden the search – if the original question is too specific, we suggest alternative, broader questions that are more likely to return relevant evidence. For example, if the original question is about a specific drug, we might suggest one about the entire drug class instead.
• Go beyond Trip – when manually answering questions in the past, if nothing was found in Trip, we extended the search to other databases and, occasionally, even to Google. While we’re not suggesting Google now, we could offer users the option to search beyond Trip, effectively broadening the net.

I’m genuinely excited about both the Spanish-language launch (as a Hispanophile, it’s a no-brainer!) and these new ways to broaden the search. With a bit of luck, both features will roll out this summer.

At AskTrip, we are committed to protecting your privacy and using data responsibly. The following explains what information we collect, how we use it, and how we safeguard it.

What We Collect & Why
When you use AskTrip—either by logging in or accessing the service via IP authentication—we log your user ID (or institution ID) and IP address. This applies both when submitting questions and when viewing answers.

We collect this data to support service delivery, session management, usage monitoring, abuse prevention, and auditing. Question content and associated metadata are retained only as long as necessary for operational reasons (currently 90 days), unless selected for internal quality assurance or audit purposes.

Processing by Large Language Models (LLMs)
AskTrip uses external large language models (LLMs) to generate optimized search terms based on user-submitted questions. These models receive only the text of the question—we do not send login details, IP addresses, or any other user-identifying information.

However, if a user includes personally identifiable information (PII), such as a patient’s name or date of birth, this content will be sent to the LLM before AskTrip’s redaction layer can act. While the LLM may label such data as sensitive, it cannot be withdrawn once submitted.

Internal Query Handling
After the LLM generates search terms, all further processing happens securely within AskTrip’s systems. We search our internal clinical database (Trip Database), extract key clinical findings, and return an evidence-based summary to the user. No user-identifiable information is involved in this step, and no external systems access your data.

Analytics & Reporting
We use system logs, including user IDs, institution IDs, and IP addresses, to generate analytics reports that help us improve performance and understand usage trends. These reports are anonymized and aggregated.

Only authorized Trip staff can access raw log data, and it is never shared with external parties or visible to other users.

Use of Data for AI Development
We currently do not use any user-submitted data or usage logs to train or fine-tune AI models.

If this policy changes in the future, we will update this statement in advance and offer users the ability to opt out of such use.

Identity & Integration
AskTrip operates as a standalone service. It does not currently integrate with external identity providers such as Microsoft 365 or Google Workspace. User authentication is managed directly by AskTrip or through IP-based access provided by your institution.

Secure Access to Information Sources
AskTrip accesses only licensed or publicly available content from the Trip Database. All queries to our content sources are performed in read-only mode. No external system can access your queries or the content returned by AskTrip.

Data Security & Compliance
All data transmission is encrypted using industry-standard protocols (e.g., HTTPS/TLS 1.2+). Our hosting infrastructure is provided by Amazon Web Services (AWS), which is certified under standards such as ISO/IEC 27001 and SOC 2.

While the hosting platform meets these standards, the AskTrip application itself is not yet ISO/IEC 27001 certified—but we are actively working toward this. We comply with the UK General Data Protection Regulation (UK GDPR) and other relevant data protection laws.

Responsible Use
AskTrip is not intended to process or store sensitive personal data. We strongly advise users not to submit queries containing patient-identifiable information, such as names, dates of birth, or NHS numbers.

Clinical Decision-Support Disclaimer
AskTrip is designed to support – not replace – clinical decision-making. The information it provides is intended to aid professionals but does not substitute for individual clinical judgment.

This disclaimer applies to all users, including those who view previously answered questions, not just the original question submitter.

Enterprise Deployment Options
For institutional partners, AskTrip offers tailored deployments, including private hosting, single-tenant environments, data segregation, and custom retention policies, to meet specific governance, compliance, and security requirements.

Since starting the automated Q&A work I’ve been struck by something I’m calling for now question trajectories, but it’s more knowledge trajectories or even uncertainty trajectories. It stems from the observation that a user asks a question, gets and an answer and then follows up with a related question. Their knowledge needs evolve in a short space of time. During the beta we had two such trajectories:

One

• Are EDOF Monofocal IOL considered as treatment of refractive error
• Do EDOF monofocal IOL’s have significant clinical benefit for patients without any prior refractive error when having cataract surgery
• Are EDOF IOL classed as an “accommodative” lens
• Is an EDOF lens mono or multifocal
• Is an EDOF lens monofocal?

Two

• What is the prognosis for paediatric C3 crescentic nephritis inba 2 year old with 73% sclerosed glomeruli
• Are crescentic lesions seen on the kidney biopsies of children with C3 glomerulonephritis
• What is the prognostic significance of crescentic lesions seen on initial diasgnostic biopsy in a child with C3 glomerulonephritis
• what is the long term prognosis of a renal biopsy with 73% sclerosed glomruli with successful immunotherapy treatment and now in remission but with persistent proteinuria

And now that AskTrip is live, we have our third:

• How common is hand, foot, and mouth disease in Canada?
• How long should children with hand, foot, and mouth disease stay home?
• Can adults get hand, foot, and mouth disease from children?
• Is hand foot and mouth disease more common in Canada than it was thirty years ago?
• Is hand-foot-and-mouth disease the same as trench mouth?
• Was hand, foot, and mouth disease a common disease among soldiers during WWII?

I asked ChatGPT to analyse the questions and suggest future, connected, plausible questions. Here are the top 7 suggestions:

• What are the first signs and symptoms of HFMD in children and adults?
• How long is a person with HFMD contagious, and when is it safe to return to school or daycare?
• Are there particular seasons or months when HFMD outbreaks spike in Canada?
• Which provinces or territories report the highest rates of HFMD?
• What home care measures relieve mouth pain or fever in kids with HFMD?
• How can I disinfect toys and household surfaces to prevent HFMD spread?
• Can pregnant women or immunocompromised adults suffer complications from HFMD?

This is possibly only interesting to me but with enough of these trajectories could we start to see patterns in thought/knowledge evolution? What other uses might we put these question trajectories to?

At Trip, we’ve spent years helping clinicians around the world get trusted, evidence-based answers to their clinical questions. Now, with the success of AskTrip – our automated clinical Q&A system – we’re opening the door to a new way of delivering answers: a fully white-labelled solution that lives on your own site or intranet.

We know that many healthcare organisations want to provide instant, intelligent support to their clinicians – but don’t have the technical resources, capacity, or time to build such a system from scratch. That’s exactly where we come in.

Your Platform, Your Content, Our Technology
We offer a fully tailored version of AskTrip that’s designed to seamlessly blend into your digital ecosystem. It will look and feel like your own product, aligned with your branding and tone of voice. Most importantly, you have control over the content that gets surfaced and prioritised.

By embedding your content alongside globally respected, evidence-based answers from Trip, you’re creating a hybrid knowledge hub that connects external best practice with local context and policies. That means faster answers, more confident decision-making, and improved alignment with your standards of care.

Why This Matters
Healthcare professionals often struggle to find trustworthy answers quickly, especially ones that are locally relevant. With a white-labelled AskTrip solution, you’re not just offering access to high-quality evidence. You’re offering answers that carry your authority, shaped by your clinical leadership and tailored to your services.

This isn’t just a search box. It’s a clinical decision support tool, built into your environment, promoting consistency, safety, and speed.

The Benefits at a Glance

• Integrated: Embedded directly into your website or intranet.
• Branded: Customised with your organisation’s identity.
• Contextual: Highlights your internal guidance and policies.
• Intelligent: Uses AI to provide accurate, relevant clinical answers.
• Efficient: Supports clinicians at the point of need—without searching through multiple systems.

A Growing Opportunity
We’re already in conversations with forward-thinking organisations who see the value in offering a smart, branded Q&A service to their staff. Whether you’re a hospital trust, a health system, a professional society, or an educational body, AskTrip offers a scalable, cost-effective way to put smart answers into clinicians’ hands, without the need to develop your own tech from scratch.

If that sounds interesting, we’d love to talk: jon.brassey@tripdatabase.com