We have just started testing our automated Q&A system mentioned here. In short you ask a free-text question and it looks over Trip’s content to generate the best answer. Our medical director’s first tests were very encouraging so we’ve thrown it open to our AI Innovation Circle for additional testing. The initial questions from the ‘Circle’ being:

  • Which decreases Migraines more: Magnesium, Zinc, Vitamin D or Omega-3?
  • Which decreases Cluster Headaches more: Magnesium, Zinc, Vitamin D or Omega-3?
  • Which combination of Magnesium, Zinc, Vitamin D or Omega-3 has been found to decrease headaches?
  • How can cluster headaches be quickly stopped?
  • What is the best treatment for a migraine?
  • What is the QUICKEST treatment for a migraine?
  • How does the combination of acetaminophen + caffiene + aspirin for migraine treatment compare to prescription treatments for migraine?
  • How do corticosteroids compare to other treatments for preventing migraine recurrence?

We have sent these Qs to our system and have generated two answers per question, one using ChatGPT-4 and the other version 3.5 (ChatGPT describes the difference between the two as “ChatGPT-4 is an advanced iteration of OpenAI’s language model that offers improved understanding and generation of text, enhanced contextual awareness, and greater nuance in responses compared to its predecessor, ChatGPT-3.5.“). Each response takes around 10-15 seconds to generate and, really importantly, these answers are referenced.

We’ve sent the answers back to the people who asked them and we’re awaiting feedback. We know that some of them are suboptimal but we also know how we can improve things. That, ultimately, is the purpose of this proof of concept project – can a Q&A system work based on Trip’s content? Based on our work to date we are absolutely confident that we can make a robust system. Once this project is finished being tested we will take the learning and combine it with the learning from the second project (fully automated evidence reviews) and decide if/how to operationalise one or both of them…