While we released the new front-end to the site last year we have been busy rewriting the back-end portion of the site. The back-end being the whole document grabbing, processing and indexing side of things – the brain of the site. I visualise it as a car – the body-work is the front-end and the rest is the back-end!
We have virtually finished the back-end work and are currently building a whole new index consisting of over 5 million documents. We’re processing around 275,000 documents a day. Once that’s in place it’ll hopefully be a quick test and then make things live.
The advantages of the new back-end are numerous, including:
- A higher level of automation than before
- Indexing is much quicker
- A lot of the work is cloud-based/distributed making it more resilient/flexible
- Modern code means easier to maintain and fix
Most users won’t notice much difference; possibly more up-to-date content
Once we’re happy we can then start to have fun and develop the site. Given the rewrite has taken over 2 years we’re very keen to start to improve the site and we’ve got lots of threads of work, for instance:
- Guideline and RCT ‘scoring’ system in the short-term and systematic reviews in the longer term
- Experiment with a 3rd layer of search (the site currently only has specific and sensitive)
– Title (specific)
– Body (sensitive)
- Experimenting with a machine learning/deep learning search algorithm – this is a project we’ve recently started and will have the initial results mid-summer
- Improve the Advanced search