Trip Database Blog

One of the most common criticisms of AI in clinical search is the black box problem.

You ask a question, you get an answer, and it is not always clear how the system got there.

For clinical evidence, that matters.

AskTrip takes a different approach. Alongside each answer, users can now view a transparency page, a step-by-step account of what happened between the original question and the response they received.

It is designed to show the working behind an AskTrip answer, not just the final response.

Why this matters

AskTrip is not a general chatbot. It is an evidence-based clinical Q&A system built around Trip Database.

Its job is not simply to produce fluent medical text. Its job is to help users move from a clinical question to relevant, source-linked evidence – while preserving the intent of the question and being clear about the strength and limits of the evidence found.

That means the route to the answer matters.

A confident answer can still be weak if the question was misunderstood. A citation-rich answer can still be misleading if the cited sources do not directly support the conclusion. And an answer can sound clinically useful while quietly drifting away from what the user actually asked.

The transparency page is our attempt to make those risks visible.

Nine steps, fully visible

Take a simple query:

What is the evidence for transurethral water-jet ablation (Aquablation) in benign prostatic hyperplasia?

The transparency page shows how AskTrip moves from that question to the final answer. It can be reached via the ‘Transparency’ button on each answer page:

1. The question is interpreted

First, AskTrip shows how it understood the question.

In this example, the system identifies the clinical intent as Outlook & Future Care. It also identifies the key elements of the question:

• population: patients with benign prostatic hyperplasia
• condition: benign prostatic hyperplasia
• intervention: transurethral water-jet ablation, also known as Aquablation
• phase: treatment
• outcome: evidence

Some of these are marked as explicit, because they come directly from the user’s question. Others are marked as implied, because they are necessary to make sense of the question.

This step is important because even a short clinical question carries assumptions. Here, the user is not asking about “surgical removal of the prostate” in general. They are asking about the evidence for a specific minimally invasive treatment for benign prostatic hyperplasia.

By showing the interpreted question, AskTrip makes it easier to check whether the system has preserved the user’s clinical intent.

2. Searches are constructed

AskTrip does not rely on a single search.

For the Aquablation question, the transparency page shows several different search routes:

• broad lexical searches
• focused lexical searches
• PubMed-style searches
• vector search using the original question
• similar previous questions

The broad searches include natural-language variants such as:

• transurethral water-jet ablation evidence benign prostatic hyperplasia
• Aquablation clinical trials benign prostatic hyperplasia
• transurethral water-jet ablation outcomes BPH
• Aquablation vs other treatments for BPH
• Aquablation long-term results benign prostatic hyperplasia

The focused searches are narrower, using terms such as Aquablation, water-jet ablation, benign prostatic hyperplasia, symptom improvement, quality of life, efficacy, effectiveness and recurrence.

The PubMed searches use MeSH and title/abstract terms, while the vector search sends the original question as-is to the semantic search system.

This matters because different search methods find different things. A lexical search may find exact terminology. A vector search may find conceptually similar material. PubMed-style searching may retrieve biomedical literature indexed in a more formal way.

The point is not to trust one search route. It is to gather candidate evidence from several routes, then filter and prioritise it.

3. Results are retrieved

In this example, AskTrip retrieved 292 total results:

• 107 from vector search
• 77 from broad lexical search
• 75 from focused lexical search
• 33 from similar questions (See ‘reference stripping’ in this blog post)

PubMed found a further 53 results, which were held in reserve and used only if more evidence was needed.

This makes the retrieval stage visible. The final answer is not based on a single hidden query. It is built from a wider set of candidate evidence gathered through several search methods.

4. Duplicates are removed

The same guideline, review or study may be found through more than one route.

AskTrip therefore removes duplicates before moving further through the pipeline. In this example, 292 retrieved results became 202 deduplicated articles.

This matters because duplication can distort the apparent volume of evidence. A document found by three routes is not three separate pieces of evidence. Deduplication helps keep the evidence base cleaner before relevance scoring begins.

5. Relevance is scored

The deduplicated documents are then scored for relevance.

In the Aquablation example, the transparency page shows the score breakdown:

• score 10: 11 documents
• score 9: 22 documents
• score 8: 13 documents
• score 7: 12 documents
• score 6: 7 documents
• score 5 and below: 137 documents

The standard inclusion threshold is 6, but guidelines with a relevance score of 5 can also be included because guidelines may still be clinically important even when their wording does not closely match the user’s question.

This step helps separate documents that merely mention BPH or surgical treatment from documents that are likely to answer the specific question about Aquablation.

6. Documents are prioritised

From the scored results, AskTrip prioritised 26 documents.

These included:

• 6 essential sources, such as guidelines and systematic reviews
• 14 desirable sources, such as RCTs, cohort studies and high-quality primary research
• 6 other sources, such as case reports, opinion or background material

The transparency page also shows where the prioritised documents came from:

• 23 from vector search
• 21 from focused lexical search
• 18 from broad lexical search
• 5 from similar questions

Those numbers overlap because a single document may be found by more than one route.

This stage is important because AskTrip is not simply counting results. It is trying to identify the documents most likely to support a useful, evidence-aware answer.

7. Evidence is extracted

The next stage is evidence extraction.

In this example, 10 documents were extracted and none were skipped.

The transparency page shows the documents that contributed evidence, including guidelines and systematic reviews. It also labels sources by type, quality and directness.

For this question, several extracted sources were marked as direct evidence. These included material on Aquablation for lower urinary tract symptoms caused by BPH, comparative outcomes against TURP, symptom improvement, safety, sexual function, ejaculatory preservation and reintervention rates.

One source, the AUA guideline amendment, was marked as indirect. That is useful because it shows that not all included sources are treated as equally direct. A document may be relevant and still not provide direct outcome data for the intervention question.

This is a key part of the transparency work. AskTrip is not just listing citations. It is trying to show what evidence was extracted, how it relates to the question, and whether it is direct or indirect.

8. Evidence quality is scored

AskTrip then provides an evidence confidence judgement.

For the Aquablation example, the answer confidence was High. The transparency page explained this using several dimensions:

• source quality: direct, high-quality evidence
• evidence base: 3/3
• answer quality: 3/3
• actionability: strong
• directness: direct
• consistency: consistent
• effect signal: clear
• sufficiency: adequate

The explanation notes that the evidence for Aquablation in BPH is direct and consistent for outcomes such as symptom reduction, safety and preservation of sexual function. It also notes that multiple systematic reviews and guidelines report improvements in lower urinary tract symptoms and quality of life, broadly comparable to established treatments such as TURP, with potential advantages around sexual function preservation.

These judgements are not meant to replace formal critical appraisal. They are practical signals to help users understand how much weight the answer deserves, and why.

9. The answer is generated and cited

Finally, the answer is generated from the evidence that has passed through the previous stages.

In this example, the final answer cited 8 documents, with a median publication year of 2025.

The cited documents included:

• NICE interventional procedures evidence review material
• systematic reviews on minimally invasive treatments for BPH
• Canadian guidance on male lower urinary tract symptoms and BPH
• systematic reviews on ejaculatory function and sexual outcomes
• systematic reviews on reintervention rates
• French clinical guideline material on surgical and interventional management of bladder outlet obstruction related to BPH

Users can see which documents were cited and click through to inspect them.

The important point is that the final answer is not presented in isolation. It is connected back to the question interpretation, the searches, the retrieved results, the deduplication, the scoring, the prioritisation, the extracted evidence and the evidence quality judgement.

It does not ask for blind trust

The transparency page is not there to make AskTrip look clever. It is there to make AskTrip inspectable.

It helps users see:

• whether the question was interpreted correctly
• which searches were run
• what evidence was retrieved
• how duplicates were removed
• how relevance was scored
• which documents were prioritised
• what evidence was extracted
• how evidence confidence was judged
• which sources were finally cited

This makes it easier to spot problems: a misread question, weak retrieval, over-reliance on indirect evidence, or an answer that sounds stronger than the evidence allows.

Transparency does not make an evidence system perfect. There will still be questions where the literature is incomplete, inconsistent, indirect or poorly applicable to the patient in front of the clinician.

But transparency changes the relationship between the user and the system.

Instead of asking clinicians, librarians and evidence specialists to trust a black box, AskTrip gives them a process they can inspect, question and challenge.

Clinical AI should not just provide answers.

It should show its working.

AskTrip turns one today. And we’ve released a huge new upgrade to the site

In its first year, AskTrip has answered more than 20,000 clinical questions – which tells us something important: health professionals want fast, transparent, evidence-based help with the questions they actually face.

To mark the anniversary, AskTrip Q&As are unrestricted for free users until the end of July. It is a good moment to try the new version.

Because a lot has changed.

Tackling EBM wallpaper and intent drift

Two problems kept appearing in user feedback of AskTrip answers.

The first we came to call EBM wallpaper: answers that contained familiar evidence-based language, but did not always get sharply enough to the heart of the question. Evidence-shaped, but not always as useful as it needed to be.

The second was intent drift: where the answer subtly moved away from what the user had actually asked – particularly when questions were complex, ambiguous or clinically messy.

We have substantially reworked the prompts and answer workflow to address both. The goal is to better understand the clinical intent, stay closer to the question, and produce answers that are more directly useful.

From one search to three

AskTrip is only as good as the evidence it finds.

Originally, it used a single lexical search. It now uses three complementary approaches:

• two lexical searches, operating on different assumptions and strategies
• a vector search, to surface conceptually relevant material that may not use the same wording as the question

This is particularly important for real-world clinical questions, which are often expressed in varied, imperfect or informal language.

Answer score

Is now ‘Answer Confidence’ The old answer score was one dimensional, how good was the evidence used. It’s fine but you could have an answer that used systematic reviews and guidelines and yet was still a poor answer. So, Answer Confidence reflects both the strength of the evidence and also how well the question was answered. It’s a big – positive – change, and the new graphic looks lovely!

More room for nuance

Around a third of users told us answers were too short.

We listened.

Answers are now longer and more detailed, not for length’s sake, but to give proper space to caveats, uncertainty, harms, benefits and practical implications. Clinical questions often deserve more than a paragraph.

Explore further

Clinical questions rarely stop at the first answer.

Explore further lets you interact with an answer directly, asking for clarification, more detail, a follow-up question, or a different angle.

It turns AskTrip from a one-shot Q&A into something closer to an evidence conversation.

Transparency

How we generate our answers should not be hidden. Openness helps users understand the process and that, in turn, improves trust

Export to PDF

Answers can now be exported as PDFs, making it straightforward to save, share or discuss them with colleagues.

New design/page layout

All of this is supported by an updated interface designed to accommodate longer answers, follow-up exploration and the wider feature set.

---

We are hugely grateful to the subscribers who have supported AskTrip in its first year. Their support has helped us improve the system, expand its features, and build a stronger evidence service. We are also indebted to the numerous testers of the new version of AskTrip – you’ve been brilliant.

Trip connects clinicians to a vast range of articles, guidelines and evidence reviews spread across thousands of external websites. That breadth is one of Trip’s great strengths, but it comes with a long-standing vulnerability.

Websites are redesigned, publishers migrate platforms and documents move. Links break. And when a link breaks, a clinician following the evidence hits a dead end.

This is a problem we have wanted to solve properly for a long time. We are now close to doing so.

Users can already report broken links using the option beneath each Trip search result, and many do. But this only catches the links that someone happens to encounter and takes the time to flag. We have never known what proportion of the broken links encountered by users are actually reported or the true scale of link rot across a database of Trip’s size.

The new system gives us a much more systematic approach.

It checks links automatically and at scale, identifying 404 errors, server failures, timeouts and unhelpful redirects. Because some failures are temporary, flagged links are checked again before being treated as genuinely broken.

Where a link remains unavailable, the system uses the article’s title, date, publication and other metadata to search for a replacement. This is where large language models have made a previously impractical task achievable at scale. Using the article’s title, date, publication and other metadata, the LLM first searches for and identifies a likely new location for the article, something it does remarkably well. It then assesses whether the proposed page is genuinely the same article, rather than relying on title similarity alone. Only when this initial recovery process fails do we move to broader searches through Google, Google Scholar and other sources, with the resulting candidates subjected to a further LLM-based validation check.

Potential replacements found through Google, Google Scholar and other search routes receive an additional LLM-based validation check. The proposed URL must also pass the link checker itself.

High-confidence matches can then be updated and reindexed automatically. Uncertain cases are placed in a review queue rather than being changed on the basis of weak evidence.

We are currently completing the final testing. Once the system is ready, we will run a substantial one-off check across Trip’s eligible records, something we have never previously been able to do. This will be followed by regular checks, so that broken links can be identified and repaired rather than silently accumulating.

For a resource that exists to connect people with the best available evidence, ensuring those connections actually work is exactly the kind of unglamorous – but vital – infrastructure that matters. We are pleased to be getting it right.

At 01:38 this morning, AskTrip answered its 20,000th question. Less than a year after launch in June 2025, that’s a milestone worth pausing on – and the timestamp itself is a useful reminder of something easy to forget when you’re looking at a UK clock. AskTrip’s users aren’t in one time zone, or even a handful: 01:38 in Bristol could just as easily have been mid-morning in Sydney or early evening in Auckland. The “around the clock” nature of AskTrip isn’t really about clinicians burning the midnight oil, it’s that, for a global tool, there is no midnight. Demand for evidence-based answers is continuous, because somewhere it’s always the working day.

The question itself was:

What are the differential diagnoses for achondroplasia in paediatric patients with chronic constipation?

On the surface, that reads like a perfectly reasonable specialist query. Look at it a little more closely, though, and it’s actually a nice example of something we’ve started referring to internally as a cross-cutting question, and it’s a useful one to write about, because milestones are as much an occasion for reflection as celebration.

Two questions wearing one coat

“Differential diagnoses for achondroplasia” is a well-defined clinical question in its own right. It points towards the small group of other skeletal dysplasias – hypochondroplasia, thanatophoric dysplasia, pseudoachondroplasia and the like – that can be mistaken for achondroplasia, particularly in early life before the radiological picture is fully established. There’s a solid, settled evidence base for that.

“Differential diagnosis of chronic constipation in children” is also a well-defined question, but a completely different one — it’s a gastroenterology question, classically framed around distinguishing functional constipation from Hirschsprung’s disease, with its own literature, its own red flags, and its own pathway.

The question as posed asks for both at once, but it’s not really clear which one the asker meant – or whether they meant a third thing entirely: what causes chronic constipation specifically in children who have achondroplasia (where spinal or foramen magnum stenosis affecting bowel innervation might be relevant). Three different clinical questions, three different evidence bases, one sentence.

Why this matters for AskTrip

A system built around finding and synthesising evidence for a focused clinical question runs into trouble with questions like this, because there’s essentially never a guideline or systematic review written for that exact intersection. The honest answer requires either picking the most likely intended framing and being explicit about that choice, or addressing more than one framing clearly enough that the asker can see which bit applies to them. Quietly blending the two – taking a bit of the achondroplasia differential and a bit of the constipation differential and presenting it as one coherent list – is the failure mode to watch for, because it can look authoritative while actually answering a question nobody asked.

This is exactly the kind of pattern that’s feeding into the question-type work behind AskTrip 2 – better recognising, at the routing stage, when a question is genuinely compound, and either decomposing it or being transparent about the interpretation that’s been answered, rather than letting the ambiguity disappear into a fluent-sounding response. We’re always striving to improve AskTrip, and examples like this one are genuinely useful – they help us focus on what matters, and we’ll keep working hard to get our approach to cross-cutting questions right.

The milestone bit

None of which takes away from the headline: 20,000 questions in well under a year, with usage continuing to accelerate, is a strong signal that there’s real demand here. But it’s the awkward, messy, occasionally malformed questions – like number 20,000 – that are the most valuable part of that growing dataset. They’re the ones that show us where the next round of improvement needs to go.

Here’s to the next 20,000 – and to learning as much from the tricky ones as the easy ones.

A recurring theme in our recent analyses has been simple but important: if we want to understand evidence-based practice, we should not only ask what evidence exists? We should also ask what clinicians and care professionals are really trying to find out.

AskTrip’s most-viewed Q&A pages offer another window into that question.

These pages may have been reached through AskTrip, the Trip Database, or Google. But that makes the signal more interesting, not less.

The original question tells us what one user wanted to know. Later views show that others recognised the same question as relevant to their own evidence need.

In that sense, page views provide a kind of triangulation. They connect what people ask, what search systems surface, and what users choose to read when looking for evidence.

And the pattern is striking.

The most-viewed AskTrip Q&As are not dominated by narrow treatment questions. They include rehabilitation plans, social inclusion, green spaces, service pathways, monitoring, post-operative care, safeguarding, public health and highly specific clinical scenarios.

This suggests that users are not simply looking for papers. They are looking for help with complex, practical, real-world decisions.

Not just medical lookup

We recently reviewed the top 50 most-viewed AskTrip Q&As. Familiar clinical topics were present: diabetes, COPD, CKD, sinusitis, hepatic steatosis, drug safety and diagnosis.

But sitting alongside these were questions about:

• scar massage in amputee patients
• rehabilitation after ACL injury
• post-intensive care syndrome recovery
• therapeutic gardens in ICU
• nature-based interventions for neurodivergent children
• adults with intellectual disabilities and social inclusion
• child-to-parent violence
• emergency department redirection services
• cultural sensitivity in dietary prescribing

This is not a list dominated by mainstream biomedical topics. It reflects the wider reality of health and care: rehabilitation, therapy, public health, social care, service delivery, inclusion and implementation.

That matters because these are often areas where usual evidence sources may be less satisfying.

Traditional medical evidence tools tend to be strongest for drugs, diagnostics, disease-specific guidelines and formal clinical interventions. They can be less helpful when the question is about recovery, participation, environment, wellbeing, service organisation or community-based care.

So the concentration of views around rehabilitation, green space, social care and disability inclusion may point to an unmet knowledge need. Users may be finding and viewing these answers because they provide something that is harder to get elsewhere: a clear, synthesised, practice-ready account of evidence for complex real-world questions.

This also echoes findings from our earlier analysis, “What clinicians are really trying to find out” That analysis used a different method, looking across the questions being asked rather than the Q&A pages being most viewed. But it pointed in a similar direction.

Many real-world evidence needs sit outside the relative “safety” of simple trial, guideline or drug-effectiveness questions. They involve complex populations, practical decisions, service contexts, rehabilitation, implementation, safety, applicability and judgement.

In that sense, the most-viewed Q&As are not an isolated finding. They reinforce a broader pattern: health and care professionals often come to evidence not just to ask “what works?”, but to understand how evidence applies when the question is messy, multidisciplinary or difficult to answer from usual sources.

Rehabilitation and therapy

One of the strongest themes was rehabilitation, therapy and functional recovery.

Questions covered ACL rehabilitation protocols, post-intensive care syndrome, physical therapy exercise parameters, scar massage in amputees, hypertrophic scar management, and occupational or physiotherapy approaches to specialist neuropsychiatric problems.

These are not abstract evidence questions. They are practical questions.

What does a rehabilitation programme look like?
How often should exercises be done?
What helps recovery, comfort, function or participation?

This is a different knowledge need from simply asking whether a drug works. It requires evidence to be translated into usable guidance.

It also illustrates a wider lesson from AskTrip’s question data: some questions are hard not because evidence is absent, but because the evidence is scattered, indirect, interdisciplinary or difficult to apply.

Green space, children and wellbeing

Another striking cluster involved nature, green space, play and child wellbeing.

Questions included the impact of green spaces on children’s mental health, socioeconomic access to outdoor play, nature-based occupational therapy for neurodivergent children, green care interventions for developmental disorders, and therapeutic gardens in ICU.

These questions sit at the boundary of healthcare, public health, education, environment and wellbeing.

They do not always fit neatly into conventional PICO formats. The intervention may be environmental. The outcomes may include participation, distress, development, wellbeing or quality of life. The evidence may come from different disciplines.

That does not make these questions less important. It makes the evidence need more complex.

Social care, disability and inclusion

A further theme involved adults with intellectual or cognitive disabilities, day-centre activities, art-based interventions, social inclusion, participation and quality of life.

Again, these are not marginal issues. They are central to health and care.

But they are not always well served by conventional clinical evidence pathways. They sit partly outside the traditional model of diagnosis, treatment and disease management.

For AskTrip, this is important. Evidence-based practice does not stop at the clinic door. Many important decisions are about helping people live well, participate, recover, connect and remain safe.

Guideline and safety questions remain central

The broader themes do not replace traditional clinical questions. They sit alongside them.

Many highly viewed Q&As were still about familiar clinical and guideline-focused topics: diabetes in older adults, diabetic foot management, COPD and asthma follow-up, CKD, antibiotic use, gabapentinoid risks and diagnosis of obstructive jaundice.

But even these questions often include real-world modifiers: age, comorbidity, monitoring, adverse effects, diagnostic uncertainty or applicability to a particular setting.

This is the crossover with our previous analyses. The important signal is not just the topic. It is the underlying uncertainty.

Users are often asking:

• Does this evidence apply here?
• Is this safe for this patient group?
• What should I do in practice?
• Is the evidence direct or indirect?
• Is this a research gap, or an application gap?
• Is the problem that evidence is missing, or that existing evidence is hard to use?

That is what clinicians and care professionals are really trying to find out.

Evidence demand, not just evidence supply

The most-viewed Q&As were not dominated by weak or speculative topics. Most had evidence ratings of Moderate or High, with only a small number rated Limited.

That is encouraging. It suggests that many practical, multidisciplinary questions can be answered using reasonably strong evidence.

But the value of AskTrip is not just in finding evidence. It is in making evidence usable.

For some questions, the key task is to identify direct evidence. For others, it is to say clearly that the evidence is indirect, incomplete or difficult to apply. For many, the challenge is translation: turning scattered evidence into a clear account of what it means for practice.

That is why these viewed Q&As are useful signals. They help reveal the demand side of evidence-based medicine: the questions, uncertainties and practical decisions that bring people to evidence in the first place.

Evidence for the real world

AskTrip’s most-viewed Q&As suggest that real-world evidence needs are broader and messier than conventional evidence systems often assume.

They cross boundaries between medicine, rehabilitation, psychology, public health, social care and service design.

They also suggest that some of the strongest unmet needs may sit outside the most mainstream medical topics – in areas where professionals still need evidence, but where usual sources may not provide clear, practice-ready answers.

Evidence-based medicine will always need to ask: what evidence exists?

But AskTrip’s question data keeps pointing us back to another question:

What are clinicians and care professionals really trying to find out?

That is the challenge – and the opportunity – for AskTrip: to make evidence usable not only for clinical decisions, but for the wider decisions that shape care, recovery, inclusion and wellbeing.