Letter to Editor requesting retraction of Roman et al. Meta-Analysis

Download the letter

Robert T. Schooley, MD, Editor-in-Chief

Clinical Infectious Diseases
198 Madison Ave
New York

NY 10016   USA

Dear Dr Schooley

Open Letter

Statement of Concern and Request for Retraction

Re: Roman Y M, Burela P A, Paspuleti V, Piscoya A, Vidal J E and Hernandez A V

 “Ivermectin for the treatment of Covid-19: 
A systematic review and meta-analysis of randomized controlled trials”      

Clinical Infectious Diseases, ciab591 Accepted manuscript 28 June 2021

The above article, now an “Accepted Manuscript” in Clinical Infectious Diseases, does not meet the standards of accuracy and integrity that any learned journal should demand. In asserting Conclusions that are not defensible on the evidence presented, it makes no contribution to science or medicine. In its present form it should be retracted.

The title closely parallels that of Bryant et al.^[1] but asserts diametrically opposite conclusions. The clinical trials selected ( n=1173 participants, 10 studies ) are a minor subset of those available and analysed in Bryant et al. ( n=3406, 24 studies ). The article commits several methodological fallacies, but first it is essential to be working with correct data.

Roman et al. have in several instances mis-reported clinical trial data either published or in preprint. The most egregious such error was corrected after alerts on social media and in the Comments section^[2] of the manuscript’s preprint on medRciv; others however remain. 

It is inexplicable for the authors to have disregarded multiple public notices of substantive errors whilst on preprint. In failing to correct, the authors verge upon falsification of data. In the Journal’s statement of Publication Ethics this is deemed “unacceptable”. Most of the misreporting instances are conveniently collected on “PUBPEER”^[3] by various contributors.

1. Inversion of the treatment and control arms of Niaee^[4]. This error was alerted (including personal protest from Dr Niaee²) and corrected prior to publication^[5] and in Figure 2. In spite of a dramatic change in the point estimates of mortality Risk Ratio (erroneous 1.11 changed to 0.37) there was however no change whatever in the Abstract Conclusions: “IVM did not reduce all-cause mortality”. The published statement is now indefensible, even on the authors’ own highly selective choice of sources. Our concern here is not that the data were not corrected; it is that conclusions are no longer based on the data.
2. Niaee⁴ is further mis-reported in Figure 3 (duration of hospital stay) where source data patently indicate a reduction (albeit modest) in hospitalization. Yet the primary data are plotted as “favours control” when the reverse is correct. There appear to be unexplained discrepancies in treatment and control arm patient numbers between Figure 3 and the source. This error is noted in the Comments section of medRciv^2,5 and on PUBPEER³.
3. The study called “Karamat” (Dr Karamat Hussein Shah Bukhari^[6]) is mis-reported in the viral clearance meta-analysis (Figure 6) as “favours control” when the viral clearance data patently favour ivermectin. Moreover there is mis-reporting from the primary source, entering incremental clearances at Day 7 (20 vs 18 for ivermectin vs control) while ignoring the faster viral clearance at 72 hrs (17 vs 2). This error is noted in the Comments section of medRciv^2,5 and on PUBPEER³.
4. The study of Ahmed^[7] is utilised for Figure 3 (length of hospitalisation) but the data on viral clearance contained in Ahmed are ignored for Figure 6 (viral clearance). This is mis-reporting by neglect, but inconsistent with any systematic review to use some data but ignore others, the fallacy popularly known as “cherry picking”. 

 In Figure 5 (Serious Adverse Events) a single SAE is noted over three studies two of which had zero events in both arms. A quantitative meta-analysis and Forest Plot is not an appropriate analysis of the occurrence of just a single event.

All these errors make material differences. It is not sufficient to claim that the errors are minor and do not affect the conclusions. They do. Moreover, most were readily available to Reviewers exercising due diligence, simply by consulting the Comments in medRciv.

In addition to addressing these issues, it is essential to include other eligible trials and to re-analyse the data. Otherwise, the article has no value and the conclusions dangerously misleading. Bryant et al.¹ have previously shown these conclusions incorrect and the results different when eligible trials are added. Correctly analysed, the chosen mortality data show a difference between ivermectin and control which is not statistically significant, but with point estimates all favouring ivermectin. This alone suggests that with adequate power, further trials might strengthen conclusions. Roman et al. erroneously interpret the absence of significant evidence of a difference as demonstrating no difference. The point estimate of mortality Risk Ratio is in fact close that in Bryant et al.¹ The persistent headline Conclusion “IVM did not reduce all-cause mortality” is unsustainable. 

Roman et al is not the first review of the efficacy of Ivermectin in Covid-19. Other systematic reviews and meta-analyses are available in the public domain. The conclusions of this latest review should mirror those of previous reviews when additional trials are added. 

The authors quote (p 12), without apparent irony:

“in the context of a misinformation infodemic, the dissemination of these results caused confusion for patients, clinicians (in particular those without training in critical reading of the scientific literature) and decision-makers, who may manipulate the information with political interests”

This could well serve as a summary of this article. 

With mis-reporting of source data, highly selective study inclusion, “cherry picking” of data within included studies, and conclusions that do not follow from the evidence, this article amounts to disinformation.

Disinformation should not be associated with any learned journal of repute. Publication of this article as it stands does a grave disservice to Clinical Infectious Diseases and the good name of Oxford University Press.

 We respectfully request investigation, and retraction of the article as it stands.

Yours sincerely

Edmund J Fordham  MA  PhD(Cantab)  CPhys  CEng  FInstP  EurIng
EBMC Squared, Northgate House, Upper Borough Walls, Bath BA1 1RG

Theresa A Lawrie  MB  BCh  PhD 
EBMC Squared, Northgate House, Upper Borough Walls, Bath BA1 1RG

Andrew Bryant MSc
Population Health Sciences Institute, Newcastle University, Newcastle-upon-Tyne, NE2 4AX

Further signatories to this Open Letter:
Disclaimer: these indicate the personal opinions of the signatories and there is no intention to associate their institutions with them.

1. Dr. Darrell Hamm BTh BSc MD CCFP ASTMH, CANADA
2. Dr William Ralph, MICGP 
3. Dr Nyjon Karl Eccles, UK
4. Dr Liesel Marcela Holler Sotomayor, MD, MRCA, UK 
5. Dr Tony Hinton MB ChB, FRCS, Consultant Surgeon, UK
6. Dr. Ira Bernstein, Family physician, lecturer, CANADA
7. Prof. Hector Carvallo, Professor of Internal Medicine, ARGENTINA
8. Dr. Dan Macias Flores, CHILE
9. Dr. Francesco Anello MD, ITALY
10. Dr Christina Peers MBBS.DRCOG.DFSRH.FFSRH, UK
11. Dr. Stephen Malthouse, MD, CANADA
12. Prof. Howard C. Tenenbaum DDS, Dip. Perio., PhD, FRCD(C), CANADA/ISRAEL 
13. Prof. Andrea G Stramezzi MD DDS PhD, ITALY
14. Prof. Eleftherios Gkioulekas, PhD, USA
15. Dr. Barbara Powell, MD, CANADA
16. Dr. Ernesto de Bernardis, MD, Clinical Pharmacologist, ITALY
17. Dr. Roberta Lacerda Almeida de Miranda Dantas, Infectious Diseases Specialist, RQE, BRAZIL
18. Dr. Robert Banner MD, CANADA
19. Dr. Patrick Phillips, MD, CCFP, Family and ER Physician, CANADA
20. Agnes Pinnel, MSc., Former CEO Clinical Research, HUNGARY
21. Dr Sarah Hill, UK 
22. Paul E. Marik MD, FCCP, FCCM, Pulmonary and Critical Care Medicine, USA
23. Dr. Marc G. Wathelet, Ph.D., BELGIUM
24. Dr. Scott Mitchell MBChB MRCS, UK
25. Prof. Morimasa Yagisawa, PhD, Visiting Professor, Omura Satoshi Memorial Institute, JAPAN
26. Geoffrey Taylor, prof. (retd), AUSTRALIA.
27. Prof. Colleen Aldous, Professor of Medical Research, SOUTH AFRICA
28. , MBBS,MD, DNB, FANZCA, FCA, PG Dip Echo, Specialis Cardiac Anaesthesia and Perioperative Medicine, AUSTRALIA
29. Prof. Femi Babalola, MD, NIGERIA
30. Dr Haleema Sheikh, MD, UK
31. Prof. Thomas Borody, MD, AUSTRALIA
32. Dr. Veronica A Mcburnie, MD, UK 
33. Dr. John McCarthy BSc(Hons), PhD MBChB UK
34. Dr. David E. Scheim, PhD, US Public Health Service, USA
35. Ellen Guimarães, M.D, cardiologist, BRAZIL
36. Flavio A. Cadegiani, MD, MSc, PhD, PI of the AndroCoV trials, BRAZIL
37. Dr Manjul Medhi, Mbchb mrcp dtm+h, Consultant in Infectious Diseases and General Medicine, UK
38. Dr Rosamond Jones, retired consultant paediatrician, UK
39. Dr Jon Rogers MBChB,  UK
40. Prof. Matjaž Zwitter, MD, PhD. SLOVENIA 

^[1] (updated version as published) Bryant A, Lawrie T A et alia (2021) “Ivermectin for Prevention and Treatment of Covid-9 infection: a Systematic Review, Meta-Analysis and Trial Sequential Analysis to inform clinical guidelines”  Am. J. Therapeutics e-publish ahead of print https://journals.lww.com/americantherapeutics/Abstract/9000/Ivermectin_for_Prevention_and_Treatment_of.98040.aspx

^[2] Roman Y M et al., version 1: https://www.medrxiv.org/content/10.1101/2021.05.21.21257595v1

^[3]  Comments on Roman et al. in PUBPEER (“The online Journal club”) https://pubpeer.com/publications/955418F3D4D39742CFFA8C1B023AA3

^[4] Niaee M S et al. (2020) “Ivermectin as an adjunct treatment for hospitalized adult Covid-19 patients: A randomized multi-center clinical trial” Research Square https://doi.org/10.21203/rs.3.rs-109670/v1 . Now published as Niaee MS et 14 alia (2021) Asian Pacific Journal of Tropical Medicine 14 (6), 266-273.

^[5] Roman Y M et al. version 2: https://www.medrxiv.org/content/10.1101/2021.05.21.21257595v2

^[6] Bukhari K H S et al.  (2021) “Efficacy of ivermectin in Covid-19 patients with Mild to Moderate disease” medRxiv, https://www.medrxiv.org/content/10.1101/2021.02.02.21250840v1

^[7] Ahmed S et 14 alia (2021). A five-day course of ivermectin for the treatment of COVID-19 may reduce the duration of illness. International Journal of Infectious Diseases, 103, 214-216. doi: 10.1016/j.ijid.2020.11.191