The explosion of systematic reviews – have we gone too far?

Posted by Simon Knight on September 14, 2016

A fascinating recent paper from Professor John Ioannidis from Stanford University, published in the Milbank Quarterly, has examined the explosion in the publication of systematic reviews and meta-analyses over recent years. Publications between 1991 and 2014 increased by 2,728% for systematic reviews, and 2,635% for meta-analyses – far in excess of the rate of publication of new randomised controlled trials. Of concern, meta-analyses frequently overlap in scope, published close together and without any additional or novel information. Sometimes these studies have identical conclusions, sometimes differences in inclusion criteria lead to differing conclusions resulting in confusion to the reader. Futhermore, Ioannidis estimates that up to 20% of meta-analyses go unpublished, and 20% are flawed beyond repair. He estimates that only around 3% are of decent quality and are clinically useful.

A number of ways in which reviews can be improved are discussed, including standardised reporting, prospective registration of review protocols, improvements in trial design and consistency of outcome reporting and greater availability of individual-patient data. The problem of duplication could be addressed by the use of collaborative tools that allow for continuously updated prospective meta-analyses as new data are published (such as open collaborative meta-analysis). Use of network meta-analysis tools would even allow the addition of data regarding new treatments for a condition as they become available.

One area where I disagree slightly with Ioannidis is his assertion that a high proportion (17%) of reviews are non-informative and hence not useful. Often, a systematic review will conclude that there are a paucity of quality data regarding an intervention, and that further high-quality studies are required. I have written previously about the Cochrane approach to such topics which often results in empty reviews (due to the the lack of randomised evidence). Whilst this approach is undoubtedly non-informative, a well performed narrative systematic review will not only inform the reader of the lack of evidence (an important finding in itself), but also examine the existing (often flawed) evidence to help inform the design of future trials.

Despite the reservations expressed in his paper, Ioannidis still sees value in properly-performed systematic reviews and meta-analyses performed by experienced teams in the absence of commercial bias and other interests. The challenge is to maintain the reputation of these important studies in the face of increasing dilution from biased and flawed reviews.

September 2016 Transplant Trial Watch now available

Posted by Simon Knight on September 5, 2016

The September 2016 edition of the Transplant Trial Watch is now available. This month’s studies include include the perils of late CNI withdrawal, CNI to mTORi conversion in Liver Transplant recipients and computer-based home education.

August 2016 Transplant Trial Watch now available

Posted by Simon Knight on August 15, 2016

The August 2016 edition of the Transplant Trial Watch is now available.

This month’s edition includes systematic reviews of pre-emptive renal transplantation and management of ureteric strictures, along with trials of prostaglandin E1 in liver transplant recipients and C1 esterase inhibition in renal transplantation.

Peter Morris Retires as Director of the CET

Posted by Simon Knight on July 11, 2016

As many of you are now aware, Peter Morris has formally announced that he is retiring from his role as director of the Centre for Evidence in Transplantation. Peter established the CET in 2005 with the aim of improving the quality of evidence in solid organ transplantation and increasing awareness of evidence-based medicine amongst transplant professionals. His vision has been a great success, with initiatives such as the Transplant Library, Evidence in Transplantation course and a large number of published systematic reviews bearing testament to his hard work.

On the 7th July, Peter was joined for afternoon tea by many colleagues at the Clinical Effectiveness Unit at the Royal College of Surgeons of England. John Carr, the college photographer, presented Peter with gifts including a framed photograph from the start of his term as President of the College.

Sir Peter and team (1 of 2)

In the evening, we had a lovely dinner with Peter, his wife Joce, and many of the researchers, research fellows and transplant surgeons that have been involved with the CET and it’s work over the past 11 years.


The rest of the team here at the CET are obviously very sad to see Peter leave. His guidance and mentorship has had a large influence on us all and his vision for the CET represents an enormous advance in the field of Evidence-Based Transplant Surgery. Fortunately, he has agreed to stay on as an honorary advisor, whilst Liset Pengel and Simon Knight take over as co-directors.

July 2016 Transplant Trial Watch now available

Posted by Simon Knight on July 1, 2016

The July 2016 edition of the Transplant Trial Watch is now available.

This month, trials include long-term outcomes from BENEFIT-EXT, treatment of asymptomatic bacteriuria in renal transplant recipients and and in depth analysis of the VICTOR trial.

Would you like to get involved in Evidence Based Medicine?

Posted by Katriona O'Donoghue on June 15, 2016

We are looking for a (trainee) clinician to contribute to a systematic review involving the methodological quality appraisal of clinical practice guidelines in kidney transplantation. Specifically, you will evaluate and compare the quality of 20 clinical practice guidelines in kidney transplantation using the Appraisal of Guidelines for Research and Evaluation (AGREE) II Instrument. The estimated time commitment is around 40 -50 hours. You will learn skills to appraise clinical practice guidelines and you will receive a publication as a result.  For more information on the protocol, please visit Prospero here.

If you are interested in becoming involved, please e-mail Katriona O’Donoghue at [email protected]

June 2016 Transplant Trial Watch is now online

Posted by Simon Knight on June 3, 2016

The June 2016 Transplant Trial Watch is now available.

This month, the trial watch includes systematic reviews of quality of life reporting in renal transplant trials and extended criteria donors, as well as an analysis of wound complications with everolimus following cardiac transplantation.

How can we improve the Transplant Library?

Posted by Simon Knight on May 5, 2016

We are seeking feedback from users of the Transplant Library, to help us to improve the experience for our many users. We would be very grateful if current users would take the time to complete a short survey about the library.

The survey can be accessed here.

May 2016 Transplant Trial Watch now available

Posted by Simon Knight on April 29, 2016

The May 2016 Transplant Trial Watch is now available online and via our app for iPhone, iPad and Android.

This months new trials include long-term follow-up from the Spare the Nephron trial, methods for promoting organ donation and mobile medical apps for encouraging sun protection behaviour.

Facial Transplantation Moves Forward

Posted by Peter Morris on April 14, 2016

This is quite a remarkable achievement in that the patient, a volunteer fire-fighter aged 41, had suffered severe burns across his entire face including the loss of eyelids, ears, lips, most of his nose and the entire scalp during a rescue search in a burning home in 2010. He had had some 80 reconstructive procedures on his face since the injury and all this reconstruction had to be excised at the time of  the transplant. Dr. Rodriquez, a surgeon at NYU Langone, with a team of some 100 colleagues and assistants, found a suitable donor and successfully carried out the donor operation and the transplant in adjoining theaters.  However, what is unusual in this particular case, among the 37 cases of facial transplant carried out worldwide, was the use of 3-D modelling to enable a good fit with the patient’s skeleton as some of the skeleton of the patients face had to be replaced or augmented in addition to the soft tissues. He still had his eyesight intact and so they were able to transplant the face with eyelids and relevant muscles and both ears in April 2014. This was indeed a tour de force. At the time of the report there had been no episodes of acute rejection and the functional result is impressive.

Dr Rodriquez is a graduate of the Medical College of Virginia and received his plastic surgeon training at the Johns Hopkins before moving to NYU Langone. He had performed a face transplant earlier in Baltimore before his move to NYU. The cost of the procedure and all related care was estimated to be around $1 million dollars. This is definitely a further step forward in establishing facial transplantation as an acceptable procedure in the right centre, but it remains an experimental procedure.

Bulletin of the American College of Surgeons, April 2016.

Next Page »