This article appears in the following Journal of Antimicrobial Chemotherapy issue: The British Society for Antimicrobial Chemotherapy Resistance Surveillance Project 1999/2000-2006/7 [View the issue table of contents]
Articles |
Analysis, power and design of antimicrobial resistance surveillance studies, taking account of inter-centre variation and turnover
1 Department of Medical Microbiology, Southmead Hospital, Southmead Road, Bristol BS10 5NB, UK 2 Centre for Biostatistics and Genetic Epidemiology, Department of Health Sciences, University of Leicester, 2nd Floor Adrian Building, University Road, Leicester LE1 7RH, UK
* Corresponding author. Tel: +44-117-959-4080; Fax: +44-117-959-3154; E-mail: rreynolds{at}bsac.org.uk
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionFundingTransparency declarationsReferences |
|---|
Objectives: Logistic regression is commonly used to analyse resistance surveillance studies, but variation between collecting centres undermines its assumption that isolates are independent. We studied the impact of this problem and the ability of alternative methods to overcome it. We also investigated different study designs and estimated the statistical power of the BSAC Resistance Surveillance Programmes.
Methods: We simulated datasets with various combinations of study design, inter-centre variation, annual centre turnover, initial resistance level and odds ratio, and analysed 1000 repetitions of each for trends in resistance by five variants of logistic regression.
Results: Traditional analysis by unadjusted logistic regression was invalid because it gave very high type 1 (false-positive) error rates, up to 49%, in the presence of high levels of inter-centre variation and turnover. Of the other methods investigated, logistic regression with random effects for centre performed best: it had appropriate error rates for all study designs assessed and generally had higher power than fixed-effects or cluster-robust approaches. A ‘Diffuse’ study with more centres contributing fewer isolates was less susceptible to the ill-effects of inter-centre variation than a study of equal overall size with fewer centres contributing more, and had slightly higher power.
Conclusions: Unadjusted logistic regression, ignoring inter-centre variation, is unsuitable for the analysis of trends in typical resistance surveillance studies, often leads to erroneous conclusions and should be avoided. Random effects logistic regression is an appropriate, widely applicable alternative, available in most standard statistical software. Collecting isolates from a larger number of centres has both statistical and scientific advantages.
Keywords: trend , random effects model , simulation , type 1 error , statistical analysis
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionFundingTransparency declarationsReferences |
|---|
Antimicrobial resistance surveillance studies, despite their apparent simplicity, pose some challenges for valid analysis and hence valid interpretation. Problems arise particularly because contributing centres may vary in their underlying resistance rates and may not participate throughout the duration of a study. Knowing the power of a study (the probability of its detecting an effect when that effect exists) is an essential element of valid interpretation, and simulation is a good approach to estimating it for non-standard analysis methods.1 Higher power, or greater robustness to analytical weaknesses, may be achievable with the same total number of isolates by improving the study design. We used simulation to investigate several methods of analysis and several study designs in order to assess their validity and power.
The assumption underlying most methods of statistical analysis is that all observations are independent and unrelated to each other. This assumption does not hold true in resistance surveillance studies. The isolates (observations) are generally collected from a moderately small number of clinical laboratories, and isolates from a single collecting centre are generally more similar to each other than expected under the assumption of independence. This can be described as clustering or intra-centre correlation (more generally, intra-class correlation).2,3 Conversely, isolates from different centres are generally more different than expected, which can be described as inter-centre variation.
It is not at all surprising that isolates collected from one centre should be more similar to each other than to isolates from other centres, even after accounting for factors such as patients' age and trends over time. The most obvious reason for this is that bacteria are infectious, meaning that multiple isolates of the same strain are likely to be collected by a single laboratory. Another reason is that there may be differences between the centres that affect resistance, but are either unobserved or not included in the statistical model, for example, differences in the general health of their populations, antibiotic policies and usage, case mix and so on. In effect, each centre has its own underlying resistance rate, and the probability of an individual isolate being resistant depends both on that underlying rate and on observed factors such as patients' age.
A further complication is that there is usually some turnover among collecting centres: over time, some centres withdraw from a study and must be replaced, so not all the centres contribute throughout the study period. This has the potential to produce the spurious appearance of resistance trends in the data. For example, if a centre with a low underlying resistance rate drops out and is replaced by a centre with a relatively high underlying resistance rate, this can give the appearance of a trend of increasing resistance over time.
The power of a surveillance study to detect trends in resistance or associations between resistance and other factors is information essential for a proper interpretation of results. A finding that a trend or association is ‘not statistically significant’ is meaningless, unless the power is known. A study with low power will be unlikely to identify even quite strong effects as statistically significant. Only if the power is high can a lack of statistical significance be validly interpreted as evidence of the absence of association.
Power is determined largely by the size of a study, but the design of the study also plays a part. The design could have other effects too, such as reducing the adverse impact of inter-centre variation and centre turnover.
In this study, we used simulated datasets to investigate the power and validity of different methods of analysis for trends in resistance, the effects of inter-centre variation and turnover and the influence of study design. We also assessed the power of the BSAC Resistance Surveillance Programmes to detect resistance trends in key species,4 given the number of isolates actually collected per centre per year so far.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionFundingTransparency declarationsReferences |
|---|
Simulation study
The simulation study focused on the detection of trends in resistance.
Simulation parameters and process
Datasets were simulated to represent results from surveillance studies of 5 years duration with three designs and various levels of inter-centre variation, centre turnover, and initial resistance rate, and with or without a trend of increasing resistance over time (Tables 1 and 2). The parameters were chosen with reference to available data to reflect typical resistance surveillance studies such as the BSAC Programmes.4 The three study designs were labelled ‘Small’, ‘Large’ and ‘Diffuse’. The Small study collected 1000 isolates over 5 years, with 20 centres each year contributing 10 isolates each. The Large and Diffuse studies both collected 5000 isolates over 5 years: the Large study included 20 centres each year contributing 50 isolates each and the Diffuse study had 100 centres each year contributing 10 isolates each.
|
|
The simulation process is shown in Figure 1. Each simulated dataset was analysed by several methods, giving a P value for the significance of trend in resistance over time, and the simulations were repeated 1000 times. The percentage of replicates giving a significant result (P < 0.05) is the power to detect that trend (at P < 0.05) or, if the simulation parameters implied no trend, the type 1 error rate. Type 1 error rates much above, or much below, 5% show that the analysis method is potentially misleading.
|
Methods of analysis in simulation study
Five variants of logistic regression were compared, as described below.5
Unadjusted logistic regression. Logistic regression is the customary form of analysis for binary outcomes such as resistant/not resistant. When used to model time trends in resistance, the probability of resistance in any particular isolate is treated as comprising a baseline probability at time 0, modified by an effect of time, i.e. by a trend in the odds of resistance. Both the baseline and the effect of time are assumed to be the same for all isolates, which are assumed to be independent of each other, and the collecting centre has no influence on the probability of resistance.
Cluster-robust standard errors. This approach to centre clustering does not model the effect of the collecting centre directly, and the estimate of the odds ratio for trend will be the same as in unadjusted logistic regression.6 However, the variance of the estimate is adjusted to reflect the grouping of the isolates into centres, and this alters the standard error, confidence interval and significance of the estimate.
Fixed-effects methods: dummy variables and conditional logistic regression. These methods also envisage a baseline probability of resistance, but then model each centre as having its own individual effect on resistance, the result being that each centre has its own underlying resistance rate. Meanwhile, the effect of time is the same in all isolates and all centres. There are no assumptions about the distribution of centre effects; the method using ‘dummy’ or ‘indicator’ variables for centres estimates the individual centre effects directly, whereas the conditional method avoids this estimation.
Random effects logistic regression.
A random effects model also treats each centre as having its own effect on the probability of resistance, but it differs from a fixed-effects model because it assumes a particular form for the distribution of centre effects. Specifically, the log-odds ratios for centres are assumed to have a normal distribution with zero mean. The standard deviation of this distribution, commonly denoted as 
u, describes the extent of inter-centre variation and is referred to as the inter-centre variation parameter throughout this paper.
Determination of power in BSAC Resistance Surveillance Programmes
A similar approach using 1000 replicate simulations was applied to assess power in two representative datasets collected in the BSAC surveillance studies,4 using the actual numbers of isolates and their distributions over time and among centres. The datasets chosen were respiratory Streptococcus pneumoniae (n = 5810 over eight seasons)7 and bacteraemia Escherichia coli (n = 1480 over six seasons).8
For the detection of time trends, actual isolate collection times were used. For the effect of binary covariates, these were simulated with isolates having a 10%, 20% or 50% probability of being ‘positive’ for the covariate; this would mimic 10% of the bacteraemia isolates being from community versus other sources, 20% being in surgical versus other specialties or 50% being male versus female, for example. Baseline resistance rates, at time 0 or in the baseline covariate group, were simulated as 5%, 10%, 25% or 50%, and inter-centre variation was simulated with a parameter of u = 0, 0.5 or 1. The odds ratios simulated for E. coli were 1.5 and 2 for binary covariates and 1.5 and 2 per 5 years for trend; for S. pneumoniae, odds ratios were 1.25 and 1.5 for binary covariates, and 1.25 and 1.5 per 5 years for trends.
Each simulated dataset was analysed by logistic regression with random effects for a centre, as this was the method used for the analyses of the surveillance datasets reported elsewhere in this Supplement.
Simulation and analysis used Stata versions 9.2 and 10.0 [StataCorp LP, College Station, TX, USA]. Sample commands in version 9.2 syntax are shown in Table 3.
|
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionFundingTransparency declarationsReferences |
|---|
Simulation study
The total number of combinations of study design, inter-centre variation, centre turnover, baseline resistance rate and odds ratio was 360 (Table 1). In the Small study, with a 1% baseline resistance and odds ratio for trend of 1, the simulated datasets very occasionally contained no resistant isolates at all and could not be analysed; this was a very minor problem as it affected only 4 of the 360 combinations and no more than 4 of 1000 repeated simulations in any case.
The results for logistic regression using dummy variables for centre are not presented in detail as this method had no practical advantages, and some theoretical disadvantages, compared with conditional logistic regression (conditional fixed-effect models).
Unadjusted logistic regression sometimes gave very high type 1 (false-positive) error rates, as shown for the Large study in Figure 2 (in other words, it often and misleadingly identified non-existent trends as significant). The error rate was close to its design level of 5% if there was no inter-centre variation, or no annual centre turnover, but increased with increasing levels of inter-centre variation, centre turnover and resistance (up to 50%). In the worst case (inter-centre variation 1, 20% turnover, 50% resistance), this analysis incorrectly identified a significant trend in resistance in 49.2% of the simulations, in which no trend existed, far above the acceptable level of 5%.
|
In the same Large study, the worst-case error rate for analysis using cluster-robust standard errors was 10.1%, with values more typically between 5% and 8%. Conditional fixed-effects and random effects models produced worst-case error rates in these simulations of 6.5% and 6.6%, with values generally close to the design level of 5%, as would be expected for valid analyses.
Fixed-effects models require that each centre includes both resistant and non-resistant isolates, and observations from centres that do not meet this condition are simply dropped from the analysis. This problem was widespread, affecting 313/360 tested combinations in some repetitions and 106/360 combinations in all repetitions. It was, predictably, more often an issue when the resistance level was low, and inter-centre variation and centre turnover were high, all conditions that increase the chances of one or more centres contributing no resistant isolates. The Diffuse and Small studies, with fewer isolates per centre than the Large study, were more severely affected, with up to 36% of the isolates dropped from analysis when the resistance level was 5% and up to 19% dropped at a 10% resistance rate (Table 4). No isolates were dropped from analysis in unadjusted, cluster-robust or random effects logistic regression.
|
The power of random effects logistic regression to detect a simulated trend, with the odds of resistance doubling over 5 years, is shown in Figure 3 for the Small study (n = 1000). Power, i.e. the probability of detecting the simulated trend, was, naturally, greatly reduced at lower initial levels of resistance. It was also reduced slightly by higher levels of inter-centre variation and, in the presence of inter-centre variation, by higher levels of centre turnover. The lowest power achieved by random effects analysis for the Large and Diffuse studies (both with n = 5000), at initial resistance rates of 1%, 5%, 10%, 25% and 50%, was 32%, 80%, 93%, >99% and >99%, respectively, compared with 9%, 27%, 40%, 60% and 65% for the Small study. Both cluster-robust and conditional fixed-effects analyses had slightly lower power than random effects on average (Figure 4), and the differences were sometimes substantial, particularly for the Large study with cluster-robust analysis. The power of unadjusted logistic regression is not shown, as consideration of its type 1 error rate has already shown that it is an invalid form of analysis for these surveys.
|
|
Figure 5 illustrates the difference in power for the three study designs, analysed by random effects logistic regression, with an initial resistance rate of 5% as an example. The Diffuse design had a slight advantage over the Large design in the presence of inter-centre variation and centre turnover, although both studies collected equal numbers of isolates in total, whereas the Small study naturally had much lower power.
|
The Diffuse design also had an advantage in limiting the impact of inappropriate analysis by unadjusted logistic regression: the worst-case false-positive error rate of 49.2% for the Large study was reduced to 16.1% for the Diffuse design; however, this was still high compared with the intended rate of 5%. The worst-case error rate for the cluster-robust analysis was similarly reduced from 10.1% in the Large study to 7.3% in the Diffuse study, with most values in the Diffuse study close to the acceptable level of 5%.
Power in BSAC Resistance Surveillance Programmes
The power to detect trends and the effect of binary covariates (Tables 5 and 6) was higher in the S. pneumoniae dataset than in the E. coli dataset, as expected, given that it was considerably larger (5810 versus 1480 isolates) and somewhat longer (8 versus 6 years). Within each dataset, power was higher when there was less inter-centre variation, when the baseline resistance was higher (up to 50%) and, for categorical covariates, when the proportion of covariate-positive isolates was higher (up to 50%). The power to detect the effect of a binary covariate was substantially affected by its distribution; power was commonly 25–40 percentage points lower when 10% rather than 50% of the isolates were positive for the covariate. The effect of baseline resistance was even more marked, power being commonly 40–50 percentage points lower for baseline resistance of 5% when compared with 50%. The impact of inter-centre variation on power was relatively modest, amounting to at most 15 percentage points among the various combinations simulated here.
|
|
The power to detect a trend continuing throughout the 6 or 8 years of the study with an odds ratio of 1.5 per 5 years was 23% to 72% in the E. coli dataset and 88% to 100% in the S. pneumoniae dataset, assuming a moderate level of inter-centre variation. The E. coli dataset could detect a stronger trend (odds ratio of 2 per 5 years) with 64% to 99% power, and the S. pneumoniae dataset could detect a weaker trend (odds ratio 1.25 per 5 years) with 37% to 92% power. (The ranges are for different baseline levels of resistance, from 5% to 50%, and the odds ratios 1.25, 1.5 and 2 represent increases in the odds of resistance of 25%, 50% and 100% over 5 years, respectively.)
The power to detect the effect of a binary covariate with odds ratio 1.5 varied from 25% to 97% in the E. coli dataset and from 62% to 100% in the S. pneumoniae dataset, depending on the baseline resistance level and the proportion of isolates positive for the covariate, and again assuming a moderate level of inter-centre variation. For larger effects (odds ratio 2), the E. coli dataset achieved 56% to 100% power, whereas the S. pneumoniae dataset had 24% to 99% power to detect a smaller effect (odds ratio 1.25).
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionFundingTransparency declarationsReferences |
|---|
Warnings about the inappropriateness of statistical methods that assume independence of observations have been given before,5,9,10 and it is not surprising that unadjusted logistic regression can be shown to be invalid for antimicrobial resistance studies. The scale of the false-positive error rate we found in testing for trend was, however, startling, and emphasizes that the issue is not just a theoretical nicety but a matter of serious practical importance. The worst-case error rate of 49% occurred in the simulated ‘Large’ study that had a 20% centre turnover and high inter-centre variation (parameter 1), but error rates were still remarkably and unacceptably high at more modest and demonstrably realistic levels of turnover and variation, e.g. 19% type 1 error for simulations with 10% annual centre turnover and moderate inter-centre variation (parameter 0.5), when the initial resistance rate was 50%. This simulation study shows clearly that unadjusted logistic regression should not be used for analysing resistance surveillance studies; similar logic would apply to other methods that make the same assumption of independence, such as
2 tests, including the 2 test for trend, and Fisher's exact test, all of which have been used as standard in the past and are still very commonly reported. Random effects logistic regression avoided the problems of the other analysis methods tested here. It gave a type 1 error rate close to the design level, it did not require isolates to be dropped from analysis and it was useable with all the study designs simulated. Its power was similar to, or greater than, that of analysis using robust standard errors or conditional fixed effects. The only obvious weakness of the random effects approach is that it depends on the assumption that the centre effects are normally distributed (on the log odds scale). The simulated datasets met this assumption exactly, but this is unlikely to be the case in reality. A skewed distribution seems more likely, with a minority of centres having particularly high resistance rates, perhaps as a result of outbreak clones. The impact of such a distribution on the performance of the random effects analysis remains to be investigated, but we expect that mis-specification of the exact form of the inter-centre variation will detract relatively little,11 especially in comparison with the very obvious errors entailed in ignoring the variation altogether.
Conditional logistic regression, which uses fixed centre effects, has the advantage that it does not require any assumption about the distribution of centre effects. It gave the correct false-positive error rate, but its power was somewhat lower than that of random effects analysis. Its major weakness is that it drops isolates from analysis in cases where a centre does not provide both resistant and susceptible isolates. For the studies aiming to collect 10 isolates per centre per year, even over 5 years, this was a major problem, with up to 36% of the isolates lost from analysis when the resistance rate was 5%, for example. Conditional logistic regression could be a good choice for those few studies large enough to ensure that every centre contributes both susceptible and resistant isolates, but will often be unsuitable otherwise.
Fixed-effects models estimated by including dummy or indicator variables for centre drop isolates from analysis in exactly the same way as conditional logistic regression. In addition, they are known, both on theoretical grounds and from simulation studies, to give biased estimates when the number of observations in each centre is moderate or small.12 This bias remains if a study is expanded by increasing the number of centres and is only reduced by increasing the number of observations per centre. The one situation in which analysis using centre dummies might have a place, therefore, is in a study with very large numbers of isolates per centre, where there is an interest in actually measuring the individual centres' effects, rather than simply accounting for them in order to estimate the effect of predictors such as time and specialty correctly.
The use of cluster-robust standard errors also avoids making assumptions about the form of inter-centre variation and does not have the problem of isolates being dropped. It reduced the false-positive error rate compared with unadjusted logistic regression, although not always to the planned 5% level, especially in the Small study. It also had power lower than the random effects analysis, particularly for the Large study. It is most suited to studies with a large number of centres and performed best in the Diffuse study design; in this design, it also had power similar to that of random effects. Logistic regression with cluster-robust standard errors might be a good choice for a Diffuse study design, if it was important to avoid assumptions about the distribution of centre effects.
The comparison of the Large and Diffuse study designs was instructive. The Large design concentrated its collection on 20 centres, each contributing 50 isolates per year, whereas the Diffuse design cast its net more widely, asking 100 centres to collect 10 isolates each per year. Despite having the same total number of isolates, the Diffuse design had some statistical advantages. It had higher power when there was inter-centre variation and centre turnover, and it allowed a choice between random effects and cluster-robust analysis without risking excessive type 1 error rates. The Diffuse study would also have scientific advantages, allowing for better geographical coverage and a more complete representation of the variety of hospitals, laboratories and populations making up the whole study area. The practicality of recruiting such a large number of contributing centres may remain a stumbling block, however.
We have restricted our study to matters of inter-centre variation and centre turnover, in which results of ongoing surveys give a clear indication of their existence and scale. We have not considered additional complexities such as serial correlation within centres (each isolate tending to be similar to the preceding one), further levels of clustering, or confounding between centre-level effects and predictors. Neither have we considered centre-level predictors in detail; these are factors that affect all isolates in a single centre such as hospital-level antimicrobial consumption or organizational ‘safety culture’, and they are of particular interest because they are, at least to some extent, modifiable.
No method of analysis will give good results with a study that includes both few centres and few isolates per centre, and such surveillance would, in any case, have hopelessly low power. If a study has few centres, but large numbers of isolates per centre, a fixed-effects analysis, usually conditional logistic regression, will be most appropriate; however, there will be doubts over whether the results can be generalized beyond the few contributing centres, and it will not be possible to include centre-level predictors. If a study has a large number of centres each contributing only a few isolates, the cluster-robust or random effects models perform well and would allow the inclusion of centre-level predictors; the results should be more representative than those of a study with few centres. For very large studies indeed, with many centres each contributing many isolates, more complex multilevel methods could be considered. Surveillance studies more typically have a moderate number of both centres and isolates per centre, and this was reflected in our simulations. Of the analytical methods tested on these simulated datasets, logistic regression with random centre effects gave the best combination of validity (assessed by the type 1 error rate), power and inclusion of all available isolates in the analysis. It was therefore chosen for the routine analysis of data from the BSAC Resistance Surveillance Programmes, as presented in other papers of this Supplement.7,8,13–16
In order to interpret findings of ‘no significant effect’ in the BSAC programmes, we undertook a ‘semi-simulation’ power study, making use of the actual patterns of centre turnover and isolate collection in two representative datasets, and investigating the effect of other parameters. Power depended critically on baseline resistance, covariate balance and odds ratio and somewhat on inter-centre variation, none of which is under investigator control. Where power is under 
80%, a non-significant result gives little information and cannot be taken as good evidence of a lack of effect. Therefore, non-significant results for associations between particular antimicrobial resistances and potentially predictive factors should be read alongside the estimates of power in Tables 5 and 6. For other species collected in the Respiratory Resistance Surveillance Programme, compared with S. pneumoniae (n = 5810), we can expect power to be higher for Haemophilus influenzae (n = 7371) and lower for Moraxella catarrhalis (n = 3369 total, 2529 with MIC results). Power for several organism groups in the Bacteraemia Resistance Surveillance Programme should be similar to that for E. coli (n = 1480), as they were collected in similar numbers, e.g. Staphylococcus aureus, Enterococcus and Klebsiella, but power will be lower for others where fewer isolates were collected, e.g. 
- and non-haemolytic streptococci (n = 1015).
Our results show that ignoring inter-centre variation when analysing typical resistance surveillance studies for trends in time leads to errors of interpretation in an unacceptably high proportion of cases. Traditional statistical methods that ignore the relatedness of isolates within centres, such as unadjusted logistic regression, 2 tests and Fisher's exact test, should no longer be used with surveillance data. Improvements in computing technology and statistical software now make several more appropriate methods readily available, each having its own strengths and weaknesses. Of the methods investigated here, we commend random effects models as offering a good combination of validity and power for analysing resistance surveillance studies.
|
Funding |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionFundingTransparency declarationsReferences |
|---|
The BSAC Resistance Surveillance Programmes up to 2006 (bacteraemia) and 2006/07 (respiratory) have received financial support from Abbott, AstraZeneca, Aventis, Basilea, Bayer, Cubist, GeneSoft, GlaxoSmithKline, Johnson & Johnson, Merck Sharp & Dohme, Novartis, Pfizer, Theravance, Wyeth or their predecessors. The BSAC funds the work of the Resistance Surveillance Coordinator (R. R.) and Resistance Surveillance Working Party.
|
Transparency declarations |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionFundingTransparency declarationsReferences |
|---|
This article is part of a Supplement sponsored by the British Society for Antimicrobial Chemotherapy.
All authors have no conflicts of interest to declare.
|
Acknowledgements |
|---|
We are grateful to all who have contributed to the success of the BSAC Resistance Surveillance Project, in particular to the many laboratories that have collected isolates and all who have played a part in testing them [see page ii10 (Acknowledgements)]. Additional information on the isolates collected in the Project is available on the BSAC surveillance web site (www.bsacsurv.org, or through a link on the BSAC homepage www.bsac.org.uk). See page ii12 (Publications) for a full list of previous publications from the Project, some of which may include parts of the information presented here.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionFundingTransparency declarationsReferences |
|---|
1 Feiveson AH. Power by simulation. Stata J (2002) 2:107–24.
2 Goldstein H. Multilevel Statistical Models (2003) 3rd edn. London: Arnold, Hodder Headline Group. Kendall's Library of Statistics Vol. 3.
3
Kerry SM, Bland JM. The intracluster correlation coefficient in cluster randomisation. BMJ (1998) 316:1455.
4
Reynolds R, Hope R, Williams L, et al. Survey, laboratory, and statistical methods for the BSAC Resistance Surveillance Programmes. J Antimicrob Chemother (2008) 62(Suppl 2):ii15–28.
5 Agresti A. Categorical Data Analysis (2002) 2nd edn. Hoboken: John Wiley & Sons. Wiley Series in Probability and Statistics.
6 Williams RL. A note on robust variance estimation for cluster-correlated data. Biometrics (2000) 56:645–6.[CrossRef][Web of Science][Medline]
7
Farrell D, Felmingham D, Shackcloth J, et al. Non-susceptibility trends and serotype distributions among Streptococcus pneumoniae from community-acquired respiratory tract infections and from bacteraemias in the UK and Ireland, 1999 to 2007. J Antimicrob Chemother (2008) 62(Suppl 2):ii87–95.
8
Livermore DM, Hope R, Brick G, et al. Non-susceptibility trends among Enterobacteriaceae from bacteraemias in the UK and Ireland, 2001 to 2006. J Antimicrob Chemother (2008) 62(Suppl 2):ii41–54.
9
Localio AR, Berlin JA, Ten Have TR, et al. Adjustments for center in multicenter studies: an overview. Ann Intern Med (2001) 135:112–23.
10 Moerbeek M, van Breukelen GJ, Berger MP. A comparison between traditional methods and multilevel regression for the analysis of multicenter intervention studies. J Clin Epidemiol (2003) 56:341–50.[CrossRef][Web of Science][Medline]
11
Neuhaus JM, Hauck WW, Kalbfleisch JD. The effects of mixture distribution misspecification when fitting mixed-effects logistic models. Biometrika (1992) 79:755–62.
12 Greene W. The behaviour of the maximum likelihood estimator of limited dependent variable models in the presence of fixed effects. Econometrics J (2004) 7:98–119.[CrossRef]
13
Brown DFJ, Hope R, Livermore D, et al. Non-susceptibility trends among enterococci and non-pneumococcal streptococci from bacteraemias in the UK and Ireland, 2001 to 2006. J Antimicrob Chemother (2008) 62(Suppl 2):ii75–85.
14
Hope R, Livermore DM, Brick G, et al. Non-susceptibility trends among staphylococci from bacteraemias in the UK and Ireland, 2001 to 2006. J Antimicrob Chemother (2008) 62(Suppl 2):ii65–74.
15
Livermore DM, Hope R, Brick G, et al. Non-susceptibility trends among Pseudomonas aeruginosa and other non-fermentative Gram-negative bacteria from bacteraemias in the UK and Ireland, 2001 to 2006. J Antimicrob Chemother (2008) 62(Suppl 2):ii55–63.
16
Morrissey I, Maher K, Williams L, et al. Non-susceptibility trends among Haemophilus influenzae and Moraxella catarrhalis from community-acquired respiratory tract infections in the UK and Ireland, 1999 to 2007. J Antimicrob Chemother (2008) 62(Suppl 2):ii97–103.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  R. Reynolds Antimicrobial resistance in the UK and Ireland J. Antimicrob. Chemother., September 1, 2009; 64(suppl_1): i19 - i23. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. Reynolds, R. Hope, L. Williams, and on behalf of the BSAC Working Parties on Resistanc Survey, laboratory and statistical methods for the BSAC Resistance Surveillance Programmes J. Antimicrob. Chemother., November 1, 2008; 62(suppl_2): ii15 - ii28. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. M. Livermore, R. Hope, G. Brick, M. Lillie, R. Reynolds, and on behalf of the BSAC Working Parties on Resistanc Non-susceptibility trends among Enterobacteriaceae from bacteraemias in the UK and Ireland, 2001-06 J. Antimicrob. Chemother., November 1, 2008; 62(suppl_2): ii41 - ii54. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. F. J. Brown, R. Hope, D. M. Livermore, G. Brick, K. Broughton, R. C. George, R. Reynolds, and on behalf of the BSAC Working Parties on Resistanc Non-susceptibility trends among enterococci and non-pneumococcal streptococci from bacteraemias in the UK and Ireland, 2001-06 J. Antimicrob. Chemother., November 1, 2008; 62(suppl_2): ii75 - ii85. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. J. Farrell, D. Felmingham, J. Shackcloth, L. Williams, K. Maher, R. Hope, D. M. Livermore, R. C. George, G. Brick, S. Martin, et al. Non-susceptibility trends and serotype distributions among Streptococcus pneumoniae from community-acquired respiratory tract infections and from bacteraemias in the UK and Ireland, 1999 to 2007 J. Antimicrob. Chemother., November 1, 2008; 62(suppl_2): ii87 - ii95. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
I. Morrissey, K. Maher, L. Williams, J. Shackcloth, D. Felmingham, R. Reynolds, and on behalf of the BSAC Working Parties on Resistanc Non-susceptibility trends among Haemophilus influenzae and Moraxella catarrhalis from community-acquired respiratory tract infections in the UK and Ireland, 1999-2007 J. Antimicrob. Chemother., November 1, 2008; 62(suppl_2): ii97 - ii103. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||