Every so often, I sit down and read a batch of questions that AskTrip refused to answer. It can be an uncomfortable exercise. These are real clinicians who came to the system with real queries – and we sent them away. We have guardrails for good reasons: to prevent problematic questions, including poorly formed queries, out-of-scope requests, and questions containing patient-identifiable information.

But the latest batch of around fifty rejected questions tells a clear story – and not quite the one I expected.

The guardrails are not mainly catching unsafe questions. They are catching unpolished ones.

The “vague” problem

The feedback most users see is some variant of “your question is vague.” Read enough of these and you notice the word is doing a lot of different work.

Here’s “vague”:

  • “immunotherapy in TNBC” — a topic, not a question, but the clinical content is perfectly clear.
  • “Oculogyric Crisis” — same. A clinician typed a topic and wanted to know about it.

And here’s also “vague”:

  • “if b12 if 186 due to metformin tehn what is the reccomeneded dose for oral replaceement?”
  • “systolic hypertension in an83 years old man whose diastoluc BP is 66-70what is 6the best treatment”
  • “En la bacteriemia por Listeria sin foco definido y siendo alergico a Penicilinas alternativas de tto al septrim…”

These last three are not vague. They are extraordinarily specific — naming the drug, the lab value, the age, the allergy, the alternatives being considered. They’re just typed badly, in capitals, or in Spanish without accents.

The giveaway is what happens next. When the system rejects a question and then suggests a rewrite that is essentially the same question with the typos fixed, it has shown that it understood the question all along.

We’re judging the wrong thing

The pattern across the batch is that the system is acting like an examiner of question quality rather than a recogniser of clinical intent. It’s asking “is this already phrased as a good clinical question?” when it should be asking “can we safely infer a useful clinical question from this?”

Real clinicians don’t type like exam candidates. They type like people typing into search boxes — fragments, shorthand, accidental capitals, missing accents. A junior doctor in a busy clinic does not stop to construct a PICO statement. They type “B12 186 metformin oral replacement” and they need an answer.

Spanish deserves better

The Spanish questions are particularly telling. We claim to support Spanish, yet several were rejected for being poorly formed. Look at what tripped them up: a missing accent, an unusual phrasing, all-caps. These are not signs of a bad clinical question. They’re signs of someone typing in Spanish. If we say we support a language, we need to support how people actually write in it.

There were also a few French and Italian questions in the batch – outside our supported languages. Two got no feedback at all; the guardrail just silently failed. The honest response there is “we currently support English and Spanish,” not a generic vagueness message.

A different model

I’d like AskTrip to move from binary accept-or-reject to three-way handling.

Accept directly for well-formed clinical questions.

Normalise and accept for questions that are clinically meaningful but messy. Show the user what we interpreted — “I’ve read your question as: …” — and answer. The clinician can correct us if we got it wrong.

Reject or clarify only when the question isn’t clinical, the language isn’t supported, or there’s no recoverable clinical intent. And when we reject, the reason should be the actual reason, not a generic “vague.”

The rewrite path has its own risk: if we silently rewrite and answer, we’ve made a clinical interpretation on the user’s behalf. That’s why showing the rewrite matters. The slight friction of confirming our interpretation is the cost of doing this safely.

The headline

Most of the questions we rejected this round were not unsafe and not out of scope. They were just unpolished. Reject less. Normalise more. And when we do reject, tell people the real reason.