Thank you for the many hundreds who took part in this survey, it has been really helpful and will definitely guide our future engagement with AI.

Overall, 51.4% of responders were health professionals, 31.8% information specialists, 9.8% academics, leaving 7% ‘other’!

We asked 4 questions, the first 3 being:

  1. Automated Q&A system: Users can ask questions in free-text format. The system would generate answers using content exclusively from Trip, explicitly mentioning the strength of the evidence and including references. How desirable is this feature for you? Please rate it on a scale from 1 to 4, with 1 being not desirable and 4 being highly desirable.
  2. Semi-automated evidence review system: Users can select a review topic, and our system will find the best available evidence, extract relevant content, and present it in an evidence table. The information would be summarised and automatically updated. How desirable is this feature for you? Please rate it on a scale from 1 to 4, with 1 being not desirable at all and 4 being highly desirable
  3. Better results ordering: This system would allow users to perform their initial search and then they could provide additional context explaining the reason for their search. Based on this extra information, the search results would be re-ordered (using AI) to ensure the most relevant articles appear at the top. How desirable is this feature to you? Please rate it on a scale from 1 to 4, with 1 being not desirable at all and 4 being highly desirable.

Observations:

  • All ideas were popular – which is good and bad!
  • The questions could have been more discerning (linked to the above point). So, instead of asking about how desirable a feature we could have offset it with highlighting potential negative aspects of the approach!
  • There was little difference between the groups of responders

Our 4th question took a slightly different format:

Focus on highest quality evidence: Currently Trip generates results from all evidence types, from the highest quality secondary evidence, through to journal articles and eTextbooks. Trip’s specialism is the higher-quality evidence and it might be the main reason you visit the site. To what extent would you want to use Trip to only see results from the highest quality evidence?

Again, very positive responses (y-axis = percentage) with little difference between types of users.

Free text responses were fascinating! The main issues being:

  • Lots of concern about accuracy/hallucinations and having the ability to check responses
  • Control – can any AI be optional
  • Reproducibility
  • Transparency
  • Lots of very lovely comments about how people love Trip!
  • A number of very interesting ideas for new developments…!

We are delighted with the above as they are very closely aligned with our own thinking. We have been working with LLMs for many months and have a reasonable level of experience. We have also tested a few ideas out and shortly we will be meeting to discuss which elements we will be taking forward. Watch this space!