JAC Advance Access originally published online on November 28, 2005
Journal of Antimicrobial Chemotherapy 2006 57(1):152-154; doi:10.1093/jac/dki431
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Correspondence |
The role of meta-analysis in the evaluation of antifungal prophylaxis: authors' response
1 Department of Internal Medicine I, University of Bonn, 53105 Bonn, Germany; 2 The Royal Free Hospital, London, UK
* Corresponding author. Tel: +49-228-287-5507; Fax: +49-228-287-5849; E-mail: glasmacher{at}uni-bonn.de
Keywords: antifungal prophylaxis , systematic reviews , meta-analysis , fluconazole , itraconazole , micafungin
Sir,
We thank Cornely et al.1 for their interest in our paper and we welcome the opportunity this gives us to discuss further the value of meta-analysis in medical decision making (particularly on antifungal prophylaxis), the assessment of heterogeneity in clinical trials, the value of the evidence-grading system of the Infectious Diseases Society of America (IDSA) and the assessment of fluconazole and micafungin in antifungal prophylaxis.
We agree completely with them that meta-analyses ‘are not a substitute for high quality individual trials’. We believe that it is self-evident that meta-analyses are based on randomized trials and can never replace them and that meta-analysis of only poor quality trials rarely provides reliable results. However, bias can be reduced and conclusions drawn using systematic review and meta-analysis where the power of the single trial is not large enough to deal with these two problems. Lack of power is the most important problem of trials in antifungal prophylaxis and has led to much confusion because conclusions have been based on less well defined but more frequent endpoints (e.g. the rate of suspected fungal infections; the rate of empirical antimycotic therapy) and the lack of evidence of an effect has been mistakenly interpreted as evidence of a lack of effect.
We assume that Cornely et al. would agree that clinical trials are performed for the benefit of patients. To deny patients the benefit of meta-analysis of these trials where it is possible may create a risk for them if an effective intervention is not adopted and an ineffective intervention is not refuted owing to the insufficient power of single trials. We refer readers to the well published example of streptokinase in myocardial infarction where more than 17 000 patients were unnecessarily treated in placebo arms of randomized clinical trials because a meta-analysis was not performed in time.2 In such a situation a meta-analysis of the already published randomized trials should lead to the recognition of a new standard. The confirmation of findings from a valid meta-analysis, as Cornely et al. call for in their letter, in a new large randomized trial would be harmful to patients who would not receive the new standard therapy.
Simply listing clinical trials, as Cornely et al.3 did in their review, is inadequate for evidence-based medical decision making and, far from being a ‘decisive strength’, is no match for the unbiased quantitative synthesis of the results of all these trials. Counting the trials with a significant outcome versus those with a non-significant outcome and basing the decision on the higher score (‘vote counting’) does not address the question of whether a trial of 600 patients with a significant effect is more likely to be relevant than three trials of 200 patients each with no significant effect.
Cornely et al. are quite right to emphasize the heterogeneity in patient population and risk factors for fungal infections but wrong to claim that a meta-analysis cannot overcome the lack of ‘statistical power to detect a statistically significant difference between placebo and antifungal prophylaxis’ for three reasons.
First, the idea that only studies which are the same in all respects and show no heterogeneity can be compared is self-contradictory. There is no need to combine these studies since they would obviously have the same findings within statistical error. The only studies which need to be aggregated are heterogeneous studies.
Second, statistical heterogeneity, which is the consequence of clinical and methodological diversity among studies, can be quantitatively examined.4 Our meta-analysis of the efficacy of itraconazole in neutropenic patients with haematological malignancies focussed very clearly on the assessment of heterogeneity.5 Several tables demonstrated that not only were the patient populations of these trials comparable but also that sensitivity analysis could not identify clinical or methodological factors that influenced the outcome. A statistical test for heterogeneity was performed for each outcome and there was no overall statistical heterogeneity (P values between 0.4 and 0.6) for the efficacy outcomes with the exception of the difference of subgroups defined by the bioavailable dose (which was one of the main results of our meta-analysis).5
Third, heterogeneity differs depending on the perspective of the reviewer. One may find more heterogeneity when, for example, all trials on antifungal prophylaxis are combined irrespective of the microbiological activity of the prophylactic drug, its bioavailability or dose than if a meta-analysis (or its relevant subgroup) is restricted to one drug or a homogenous group of drugs.
Cornely et al. have applied the evidence-grading system of the IDSA to support their recommendations. This grading system does not mention systematic reviews or meta-analysis6 and we cannot conceive of any reason why IDSA would wish to exclude meta-analyses from medical decision making in infectious diseases when several meta-analyses have made important contributions to the field.7 A recent editorial in this journal has made a clear point on this.8
The use of meta-analysis for the formulation of clinical recommendations has been a standard for some time9,10 and newer systems for guideline development which attempt to replace imperfect hierarchical classification of evidence clearly rely on quantitative synthesis of treatment effects.11,12
Cornely et al. kindly remind us of the publication from van Burik et al.13 At the time of writing our paper only the abstract of van Burik's study was available.6 Therefore, we are grateful to have the opportunity to add our comments on it here. In this large study of 882 stem cell recipients antifungal prophylaxis with micafungin was neither significantly superior to fluconazole in the prevention of fungal breakthrough infections overall (P = 0.486) nor for invasive Aspergillus infections (P = 0.071) or candidiasis (P = 0.436). Significant differences among the seven tested outcomes were seen in the rate of suspected fungal infections (based on the initiation of empirical antifungal therapy; P = 0.024), fungal colonization (P = 0.003) and treatment failure (P = 0.026). A P value correction for multiple testing was not performed. We do not think that micafungin applied as in this trial establishes a new standard for antifungal prophylaxis in stem cell transplantation as it was not performed against the most effective comparator, itraconazole,5,14–16 and, despite its size, could not demonstrate superiority in the prevention of proven invasive fungal infections.
In summary, quantitative synthesis (meta-analysis) is an integral part of systematic reviews. To avoid it where it would be appropriate and possible could result in incorrect and potentially harmful therapeutic choices. As John Ioannidis, one of the most active European researchers in this field said, ‘meta-analysis, the scientific method of critically appraising and quantitatively summarizing data from diverse studies, assumes a prominent role, next and in parallel to clinical trials’.7 Applied to the prophylaxis of invasive fungal infections in neutropenic patients our meta-analysis demonstrated that itraconazole solution in a sufficient dose is superior to no prophylaxis or fluconazole.5
Transparency declarations
A. Glasmacher: Gilead (speaker's honoraria), Janssen-Cilag/Ortho-Biotech (consultant, research support, speaker's honoraria), MSD Sharp & Dohme (consultant, research support, speaker's honoraria), Pfizer (research support, speaker's honoraria), Schering-Plough/Essex (consultant). A.G. Prentice: advisory boards and symposia from Janssen-Cilag, Johnson and Johnson, Ortho-Biotech, Pfizer, Gilead, Merck and Schering-Plough, speaking honoraria from Janssen-Cilag, Gilead, Merck and Pfizer, and consultancy for Ortho-Biotech/Janssen-Cilag.
References
1.
Cornely OA, Ullmann AJ, Karthous M. Comment on: Evidence-based review of antifungal prophylaxis in neutropenic patients with haematological malignancies. J Antimicrob Chemother 2006; 57: 151–152.
2.
Egger M, Smith GD. Meta-analysis: potentials and promise. Br Med J 1997; 315: 1371–4.
3.
Cornely OA, Ullmann AJ, Karthaus M. Evidence-based assessment of primary antifungal prophylaxis in patients with hematologic malignancies. Blood 2003; 101: 3365–72.
4. Deeks JJ, Altman DG, Bradburn MJ. Statistical methods for examining heterogeneity and combining results from several studies in meta-analysis. In: Egger M, Smith GD, Altman DG, eds. Systematic Reviews in Health Care: Meta-Analysis in Context. London: BMJ Publishing Group, 2001; 285–312.
5.
Glasmacher A, Prentice AG, Gorschluter M et al. Itraconazole prevents invasive fungal infections in neutropenic patients treated for haematological malignancies: evidence from a meta-analysis of 3,597 patients. J Clin Oncol 2003; 21: 4615–26.
6. Kish MA. Guide to development of practice guidelines. Clin Infect Dis 2001; 32: 851–4.[CrossRef][ISI][Medline]
7. Ioannidis JP, Lau J. State of the evidence: current status and prospects of meta-analysis in infectious diseases. Clin Infect Dis 1999; 29: 1178–85.[CrossRef][ISI][Medline]
8.
Leibovici L, Reeves D. Systematic reviews and meta-analyses in the Journal of Antimicrobial Chemotherapy. J Antimicrob Chemother 2005; 56: 803–4.
9.
Cook DJ, Guyatt GH, Laupacis A et al. Clinical recommendations using levels of evidence for antithrombotic agents. Chest 1995; 108: 227S–30S.
10. Atkins D, Eccles M, Flottorp S et al. Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group. BMC Health Serv Res 2004; 4: 38.[CrossRef][Medline]
11. Atkins D, Briss PA, Eccles M et al. Systems for grading the quality of evidence and the strength of recommendations II: pilot study of a new system. BMC Health Serv Res 2005; 5: 25.[CrossRef][Medline]
12.
Atkins D, Best D, Briss PA et al. Grading quality of evidence and strength of recommendations. Br Med J 2004; 328: 1490.
13. van Burik JA, Ratanatharathorn V, Stepan DE et al. Micafungin versus fluconazole for prophylaxis against invasive fungal infections during neutropenia in patients undergoing hematopoietic stem cell transplantation. Clin Infect Dis 2004; 39: 1407–16.[CrossRef][ISI][Medline]
14.
Winston DJ, Maziarz RT, Chandrasekar PH et al. Intravenous and oral itraconazole versus intravenous and oral fluconazole for long-term antifungal prophylaxis in allogeneic hematopoietic stem-cell transplant recipients. A multicenter, randomized trial. Ann Intern Med 2003; 138: 705–13.
15.
Marr KA, Crippa F, Leisenring W et al. Itraconazole versus fluconazole for prevention of fungal infections in patients receiving allogeneic stem cell transplants. Blood 2004; 103: 1527–33.
16.
Winston DJ, Emmanouilides C, Bartoni K et al. Elimination of Aspergillus infection in allogeneic stem cell transplant recipients with long-term itraconazole prophylaxis: prevention is better than treatment. Blood 2004; 104: 1581.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||