Trip Database Blog

A great explanation of the evidence pyramid (taken from this Walden University site):

We use this concept in Trip to help users navigate the results (helping indicate the likely reliability of the evidence):

The main place to see it is slightly to the right of the result.  Here you see the pyramid, a colour-banded representation of where the evidence lies in the pyramid and a phrase describing the content (e.g. systematic review, guideline etc). You’ll note the colour coding which can be found to the left of each result and it is carried through to the ‘Refine by’ area.

We’ve just had a large usability study of the site and this is what the report states:

• Users from research or information management backgrounds understood the pyramid icon, but users who were new to it wondered what it was.
• When they figured out what it was, or I talked them through it, they appreciated it and it appeared to add to their experience.
• Recommend adding a mouse-over explanation or some kind of ‘Introducing the pyramid of evidence’ box somewhere, as part of onboarding or in help.

Bottom line: it’s useful but only when you know what it means!  With no understanding of the concept it’s just confusing.

I need to go and sit on the ‘naughty step’ and contemplate why I fell in to the trap of assuming users know all this sort of stuff (oh yes, and to fix it).

From the last month, the top ten most viewed oncology articles on Trip:

1. Screening for breast cancer with mammography. Cochrane
2. Interventions for the treatment of oral and oropharyngeal cancers: targeted therapy and immunotherapy. Cochrane
3. A systematic review and economic evaluation of intraoperative tests [RD-100i one-step nucleic acid amplification (OSNA) system and Metasin test] for detecting sentinel lymph node metastases in breast cancer. NIHR HTA
4. Aspirin use for the primary prevention of cardiovascular disease and colorectal cancer: U.S. Preventive Services Task Force recommendation statement
5. Multiple myeloma. BMJ Best Practice
6. Guideline Summary: Oncology evidence-based nutrition practice guideline. Academy of Nutrition and Dietetics
7. Parental decision making about the HPV vaccine. Cancer Epidemiology & Biomarkers and Prevention
8. Immunotherapy for advanced renal cell cancer. Cochrane
9. Cancer and Rehabilitation (Treatment). eMedicine
10. Cervical cancer and HPV. NICE CKS

Two stand out articles in the list for me – 7 and 9.  All the other articles are ‘secondary evidence’.

In the case of 7 it’s a primary research article and it’s from 2010. So, as well as being primary research, it’s also relatively old. This is significant as it potentially links in with our efforts to gather uncertainties (to improve research procurement – both primary and secondary research).  We’re always looking to analyse click patterns to see if we can unearth uncertainties but finding the rights signals is problematic.  The fact that a significant number of users have an information need met by an oldish article from down the evidence pyramid suggests it’s a ‘hot topic’. Perhaps one for a systematic review or further research?

Number 9 is an eTextbook – the lowest level of evidence (in the Trip hierarchy).  Again, lots of interest and users are not finding their information need met by higher quality research.

I’m excited by this ‘discovery’ one to discuss with my research funding colleagues!