JAC Advance Access originally published online on July 19, 2006
Journal of Antimicrobial Chemotherapy 2006 58(3):594-600; doi:10.1093/jac/dkl272
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Optimal sampling strategies for early pharmacodynamic measures in tuberculosis
1 Wellcome Trust Centre for Clinical Tropical Medicine, University of Liverpool Merseyside, UK 2 Department of Pharmacology and Therapeutics, University of Liverpool Merseyside, UK 3 School of Pharmacy and Pharmaceutical Sciences, University of Manchester Lancashire, UK
*Corresponding author. Tel: +44-151-7944221; Fax: +44-151-7944222; E-mail: gdavies{at}doctors.org.uk
Received 16 January 2006; returned 16 March 2006; revised 5 May 2006; accepted 1 June 2006
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionTransparency declarationsAppendixReferences |
|---|
Objectives: To evaluate whether methodological optimization of serial sputum colony counting (SSCC) studies, a potentially important component in the drug development process for tuberculosis, could significantly improve their power.
Methods: Simulations were carried out using a model derived from a large SSCC dataset. Variance inflation factors (VIFs) were calculated for model parameters, focusing on the elimination rate constant likely to reflect ‘sterilizing’ activity and sampling schemes were optimized relative to a scheme of daily sampling during the initial phase of therapy. Corresponding sample sizes required for SSCC studies using different schemes were also computed.
Results: Published sampling schemes lacked efficiency with respect to the ‘sterilizing’ phase. Pragmatic optimized schemes yielding greatest precision were achieved using eleven sampling points around a skeleton of 0, 2, 7, 14 and 56 days. The standard error of the ‘sterilizing’ rate constant was reduced more than 4-fold, and sample size for realistic treatment effects was effectively halved. Even schemes with a restricted duration of sampling to avoid high proportions of missing data and those with fewer sampling points still achieved significant gains in precision. Sensitivity analysis suggested that such schemes should continue to perform well over the immediately foreseeable range of improvements in therapy.
Conclusions: Methodological improvements in the design of SSCC studies could make them a powerful tool in Phase II development of anti-tuberculosis agents.
Keywords: tuberculosis , clinical trials , Phase II , optimal design , sample size
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionTransparency declarationsAppendixReferences |
|---|
New anti-tuberculosis agents with enhanced sterilizing activity are urgently needed. However, the potency of modern ‘short course’ therapy makes assessment of new regimens difficult. Large numbers of patients are required in Phase III development, even for equivalence trials, and assessing the contribution of individual drugs within combination regimens efficiently is hardly feasible. Surrogate endpoints, measured in the early phase of therapy and reflecting sterilizing activity and treatment efficacy, could significantly accelerate the development of new drugs. They could assist in the confirmation and assessment of absolute activity in humans, selection of dose size and optimization of companion drugs. However, the quality of information obtainable from such measures will need to be evaluated and maximized.1
Serial sputum colony counting (SSCC) is among the most promising of these surrogate pharmacodynamic measures.2 Using standard bacteriological techniques developed for studies of ‘early bactericidal activity’ (EBA),3 the clearance of bacilli from sputum of patients with pulmonary tuberculosis (TB) is characterized by an initially rapid fall, followed by a slower decline, probably more closely related to the ‘sterilizing’ phase of treatment.2 In the largest published SSCC dataset using such techniques under a representative modern combination regimen,4 we have shown that, by comparison with conventional statistical approaches, non-linear mixed effects (NLME) analysis offers insights into the choice of model for data of this kind, removes bias in estimates of key parameters and improves their precision.5 NLME appears capable of clearly differentiating measures of ‘early bactericidal’ activity from ‘sterilizing’ activity. This is of particular importance since it is believed that it is only enhancement of the latter that holds the promise of shorter treatment.
Previously conducted studies using SSCC methodology were generally based on arbitrarily selected sampling schemes principally focused on the measurement of EBA and not ‘sterilizing’ activity.6–11 Using the results of our previous analysis,5 we have conducted a simulation exercise to see whether sampling schemes with greater efficiency for the ‘sterilizing’ phase can be constructed and what implications this would have for sample size required for SSCC studies.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionTransparency declarationsAppendixReferences |
|---|
Model for simulations
Our previous analysis supported a structural model for the SSCC data (Figure 1), described by the following function:
 |
= {1, 2, 3, 4 } is the vector of fixed effect parameters and t is the vector of sampling times. The form of the function, a biexponential decline, suggests that at least two subpopulations of bacilli are present in the sputum, though not necessarily contemporaneously. The parameter 2 thus reflects the rate of clearance of a rapidly eliminated subpopulation of bacilli, similar to ‘early bactericidal’ activity, while 4 represents clearance of a more slowly eliminated subpopulation, which may correspond more closely to ‘sterilizing’ activity. 1 and 3 can be interpreted as the sizes of these two putative subpopulations prior to treatment. Parameter values for the simulations were fixed at 1 = 15.769 ln cfu, 2 = 0.178 ln (day–1), 3 = 12.222 ln cfu and 4 = –1.526 ln (day–1), the final estimates in our analysis for the regimen comprising the drugs streptomycin, isoniazid, rifampicin and pyrazinamide (SHRZ). This regimen is very similar to the current standard and any immediately foreseeable ultra-short course regimen would be likely to include at least two of these drugs. An exponentiated parameterization was used to ensure positivity of the rate constants and to avoid constraints on optimization.12 The regression function was simulated in conjunction with the variance function selected in our previous analysis, equivalent to the following expression for the variance of the response
 |
was set to –0.285.13 This reflects a modest increase in variance at low log10 colony counts, despite the logarithmic transformation, partly due to accumulating censored or missing data and probably partly due to the properties of the bacteriological assay method itself.
|
Optimization of sampling schemes
The aim of the study was to define sampling schemes for SSCC studies with the highest possible parameter precision and statistical power with regard to estimating the parameter 4 (as a possible measure of ‘sterilizing’ activity), while maintaining operational feasibility. We utilized two search strategies: (i) a nested series of schemes based on sequential elimination of sampling points and (ii) a non-nested series of schemes employed in previous studies or considered feasible for future ones. In both cases, the performance of sampling schemes was compared against a theoretical ‘saturated’ scheme of daily sampling for 57 consecutive days (the current duration of the initial four-drug phase of therapy), using the ratio of the variance inflation factor (VIF) calculated for each of the parameters under each scheme. The variance of any parameter estimate can be expressed generally as
 |
2, a constant, which is the focus of statistical and scientific interest) and variability attributable to the design of the sampling scheme itself (VIF), sometimes referred to as the ‘design effect’ (see Appendix for further details).14 Optimization of sampling schemes may aim to achieve the lowest possible VIF for a parameter of interest or for all of the parameters simultaneously (‘globally’ optimized). We performed a simple sensitivity analysis of one such globally optimized scheme, identified using the methods above, for the parameter 4, repeating the simulations over a range of values whilst holding all the other parameters constant. Sample size calculations
Sample size calculations were performed in order to evaluate the impact of gains in precision conferred by optimized sampling schemes. These were computed using an approximate method based on linearization of an NLME model for the data (see Appendix for further details),15,16 for both the original sampling scheme and a globally optimized scheme over a realistic range of parameter differences [0.05–0.30 ln(day–1)] and inter-individual variability in the model parameters (10–30% coefficient of variation) using the same variance function for intra-individual variability.
Simulations of different sampling strategies and calculation of sample size were carried out using purpose-written functions in R (available on request from the corresponding author).17
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionTransparency declarationsAppendixReferences |
|---|
Properties of different sampling schemes
Sampling points were sequentially deleted from the ‘saturated’ scheme (daily sampling for 57 days) until the five-point scheme used in the original study4 was reached. Large reductions in the ratio of the VIF compared with the ‘saturated’ scheme were observed for 3 and 4 as the number of points increased from 5 to 11 with a steady decline thereafter (data not shown). Since complete intensive sampling involving more than 11 sampling points appeared impractical, further simulations focused on schemes using up to 11 points, attempting to minimize the VIF ratios for some or all of the parameters (see Table 1).
|
Most of the gain in precision of
4 was achieved by replacing the last point in the original scheme with day 56. This set (0, 2, 7, 14, 56) appeared a minimum skeleton for any larger scheme but given the vulnerability of the day 56 point to missing data, schemes with an earlier final point were also evaluated. Overall, precision on 4 was closely related to duration of the scheme (schemes 1–6) which had much greater influence than the total number of points (e.g. incorporating day 56 alone in a five-point scheme decreased the VIF ratio 10-fold while doubling the number of points thereafter led to a further 2-fold reduction). Schemes restricted to the first 6 weeks with less than 10 sampling points still achieved reductions of up to 5-fold in the VIF ratio with respect to 4 but sampling in the first 4 weeks alone considerably restricted its precision.
Adequate precision on 2 required at least one sample between days 0 and 7. Days 2 and 4 appeared equivalent individually but gave significant gains on 2 when used together (schemes 5, 6, 11 and 12). Schemes containing more than six points further increased precision on 2, by also including days 1 and/or 3. Most gains in the rate constant parameters 2 and 4 were achieved using a six- or seven-point scheme. Increasing the number of points to nine or more principally improved precision of the intercept parameters 3 and, to a lesser extent, 1.
Optimized sampling schemes
Most published schemes were relatively inefficient, even with respect to 2 (schemes 25–27). However, globally optimized schemes could be obtained that were superior to all these schemes with respect to all of the parameters (schemes 28 and 29). Compared with the original scheme (scheme 26), the VIF ratio could be reduced by a factor of 2 with respect to 2 and by a factor of 20 with respect to 4. Since precision on a parameter is related to the square root of the VIF this corresponds to an expected decrease in the standard error on 4 by a factor of 4.4. This could only be achieved by sampling on day 56 but the standard error was still reduced by a factor of 2.3 if no data were available after day 42.
Sensitivity analysis of the performance of a representative globally optimized scheme (scheme 29) was carried out by varying only the value of 4, since the intercepts 1 and 3 depend only on the study population and 2 is thought to be determined by the action of isoniazid alone. The scheme's superiority with respect to 4 appeared to be maintained over a range between –2.0 and –0.5 ln(day–1) (Figure 2). Since in our previous analysis the difference between SHRZ and SHT (i.e. streptomycin, isoniazid and thiacetazone) corresponded to a change in 4 from approximately –1.75 to –1.5 ln(day–1),5 it seems unlikely that any immediately foreseeable regimen will exceed a value for 4 of –0.5 ln(day–1). Over this range the VIF on 4 decreased up to a further factor of four, without affecting the precision of the other parameters.
|
Reduction in sample size achieved by optimized sampling schemes
Using a range of changes in 4 compared to an SHRZ control regimen, we computed the sample size required to detect such an effect for both the original (Table 2) and a representative globally optimized scheme (Table 3), over a range of inter-individual variability (covering the CV% of 0.20–0.25 observed in existing SSCC studies). The optimized scheme reduced sample size 1.5- to 4-fold depending on the level of inter-individual variability. The change observed in our previous analysis [0.30 ln(day–1)] appeared just detectable using the original design at the actual sample size of 50 per arm and smaller differences may be observed in future trials. Under optimized schemes, however, differences as small as 0.15 ln(day–1) could be detected with adequate power at a sample size of 100 per arm and possibly fewer if modest improvements in patient selection and laboratory methods or measurements of relevant covariates reduced the inter- and/or intra-subject variability.
|
|
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionTransparency declarationsAppendixReferences |
|---|
The use of SSCC as a surrogate pharmacodynamic measure capable of giving a reliable early indication of the efficacy of a regimen has evolved slowly over the last two decades but is now recognized as a potentially critical tool for increasing the pace of drug development in TB. There have been problems with the reproducibility of laboratory methods7 and controversy over the most suitable form of analysis for the data18,19 but since there is little experience with any other credible method, the question arises whether SSCC can be optimized to improve the precision of its comparisons.
We have previously demonstrated, using the largest published SSCC dataset employing a representative modern combination regimen, that NLME analysis is an appropriate and informative approach.5 Specifically, it supports a biexponential model for the data, suggesting that at least two subpopulations of bacilli are present in the sputum and estimates the rate constant for the slowly eliminated subpopulation (4) that is most likely to relate to ‘sterilizing’ activity observed in clinical trials. As importantly, this approach greatly increased the precision of parameter estimates. However, the number and spacing of the sampling points in the original dataset appeared to cause some estimation problems and we wished to know whether optimizing the sampling scheme might offer further improvements in precision.
Though the approach described here was non-exhaustive, we have evaluated most of the sampling schemes that appear operationally feasible. Simply doubling the number of sampling points and placing them appropriately leads not only to a globally superior strategy but to a potential improvement in precision on the critical ‘sterilizing’ parameter 4 of an order of magnitude. This is reflected in a reduction of the required sample size to detect realistic differences in 4 by approximately 2-fold with respect to the sampling scheme used in the original study. The computed sample sizes presented here for an SSCC superiority study are less than a quarter of that required for a definitive Phase III equivalence trial using relapse as an endpoint.
The optimization procedure used here was based on a single study and its usefulness depends on the correctness of the model and the repeatability of the experimental situation. Many key parameters, particularly the likely size of any treatment effect, remain uncertain at this stage. The robustness of the strategies proposed will require further evaluation as new data emerge but the NLME approach can in principle be adapted to structural models different from that used here and to changes in the variance structure. Clearly each new study design should be tailored using all the available information and no single scheme should be considered a portmanteau solution. Our simple sensitivity analysis suggests that globally optimized schemes should continue to perform well with respect to 4 in the short term but as regimens improve, schemes may need to be re-optimized to reflect new values of the relevant parameters.
Our analysis also only indirectly addresses many other important issues including losses to follow-up, decreasing availability of sputum with time and logistical capacity to collect and simultaneously process large numbers of samples. The simulations are based on data extending only to day 28 when 70% of subjects still had detectable colonies. By day 56, less than 15% would usually remain culture positive. The variance function used in the simulations takes some account of this but might not be an accurate model for the error structure beyond the range of the training set and near the limit of detection of the response. In the simulations, day 56 was the most influential sampling point with respect to 4 but predictions for the simulated function fall below log10 zero just after 56 days, hence we did not explore later sampling points. However, in the only SSCC study to have extended sampling beyond 56 days,10 only 6% of subjects had detectable counts at 60 days and none at 90 days, suggesting that such an approach might pose an absolute restriction on sample size in order to have sufficient data at these later time points, negating any potential efficiency gains.
The total number of sampling points in a scheme will depend on the precision required in parameters other than 4. While this is the parameter of interest in relation to ‘sterilizing’ activity, there are at least three reasons to prefer more intensive globally optimized schemes: correlation of the parameter estimates for 3 and 4, simultaneous treatment effects of a new regimen on both 2 and 4 (both situations where estimates of one parameter could affect or degrade the other) and some degree of protection against mis-specification of the model at the design stage. The marginal cost of adding additional sampling points versus recruiting additional patients in most studies would tend to favour the former but there are limits to this approach since increasing the number of points beyond 11 per patient leads to much smaller gains in precision and power.
Eleven-point schemes may appear impractical in some study settings if envisaged as complete intensive sampling. When carried out as essentially weekly sampling with an extra sampling point at day 2 (scheme 28), however, this is perhaps simpler than at first sight and similar schemes (scheme 29) could also be implemented using a design of, for instance, one baseline measurement followed by two balanced blocks of five points. Such deliberately incomplete schemes pose difficulties for other methods of analysis but can be accommodated by an NLME approach.13 The sample size for such a design would be increased by 20–40% (see Table 3) but this may be offset by improved operational feasibility. Furthermore, as Table 1 shows, even complete schemes with fewer than 11 sampling points could still achieve reasonable power.
SSCC is the most promising candidate for a useful early surrogate pharmacodynamic measure in the treatment of TB and NLME analysis of such data can yield large gains in precision. The additional methodological improvements we have described here suggest that SSCC may have yet to fulfil its potential as a tool for drug development.
|
Transparency declarations |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionTransparency declarationsAppendixReferences |
|---|
None to declare.
|
Appendix |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionTransparency declarationsAppendixReferences |
|---|
Variance inflation factors
The variance inflation factor (VIF) associated with an experimental design varies with the condition number of the design matrix X (in this case simply a matrix formed from the sampling times) and is determined by the collinearity of its component columns.20 Simulations using design matrices corresponding to the different sampling schemes were used to compute the matrix D of partial derivatives of the simulated function with respect to each of the parameters at each sampling point
 |
 |
) is a diagonal matrix of weights derived from the variance function.21 Sample size calculations
Sample size was calculated using an approximation of the population Fisher Information matrix of the model computed by the linearization method described by Kang et al.15 and Retout et al.16 The full non-linear mixed effects model for individual i may be formulated as follows:
 |
and ti are as specified above and 
i is the vector of random effects for the fixed effect parameters . Both and 
are assumed to be normally distributed with expected values of zero and covariance matrices 
and 
, respectively. A first-order Taylor expansion of the full model around the expected value of the random effects may be written
 |
 |
 |
is
 |
expressed as
 |
, the Wald test statistic
 |
2 distribution with h degrees of freedom. The asymptotic power of the test is then
 |
is the type I error rate, ß is the type II error rate and 
is the critical value of the central 2 distribution with h degrees of freedom. |
Acknowledgements |
|---|
Professor Davide Verotta, Dr Dongwoo Kang and Dr Sylvie Retout for providing the S-Plus scripts used in their original publications.15,16 Kayode Ogungbenro for advice on incomplete designs. Professor Dennis Mitchison for comments on earlier drafts of this paper. Dr Davies is supported by a Wellcome Trust Training Fellowship in Clinical Tropical Medicine (grant no. GR067910MA).
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionTransparency declarationsAppendixReferences |
|---|
1 Global Alliance for TB Drug Development. Scientific Blueprint for Tuberculosis Drug Development GATBDD 2001.
2
Jindani A, Dore CJ, Mitchison DA. (2003) Bactericidal and sterilizing activities of antituberculosis drugs during the first 14 days. Am J Respir Crit Care Med 167:1348–54.
3 Mitchison DA, Allen BW, Carrol L, et al. (1971) A selective oleic acid albumin agar medium for tubercle bacilli. J Med Microbiol 5:165–75.
4 Brindle RJ, Nunn PP, Githui W, et al. (1993) Quantitative bacillary response to treatment in HIV-associated pulmonary tuberculosis. Am Rev Respir Dis 147:958–61.[ISI][Medline]
5 Davies GR, Brindle R, Khoo SH, et al. (2006) Improved precision of early pharmacodynamic measures in tuberculosis using non-linear mixed effects analysis. Antimicrob Agents Chemother in press.
6 Jindani A, Aber VR, Edwards EA, et al. (1980) The early bactericidal activity of drugs in patients with pulmonary tuberculosis. Am Rev Respir Dis 121:939–49.[ISI][Medline]
7
Sirgel FA, Donald PR, Odhiambo J, et al. (2000) A multicentre study of the early bactericidal activity of anti-tuberculosis drugs. J Antimicrob Chemother 45:859–70.
8 Brindle R, Odhiambo J, Mitchison D. (2001) Serial counts of Mycobacterium tuberculosis in sputum as surrogate markers of the sterilising activity of rifampicin and pyrazinamide in treating pulmonary tuberculosis. BMC Pulm Med 1:2–9.[CrossRef][Medline]
9 Kennedy N, Fox R, Kisyombe GM. (1993) Early bactericidal and sterilizing activities of ciprofloxacin in pulmonary tuberculosis. Am Rev Resp Dis 148:1547–51.[ISI][Medline]
10 Joloba ML, Johnson JL, Namale A, et al. (2000) Quantitative sputum bacillary load during rifampin-containing short course chemotherapy in human immunodeficiency virus-infected and non-infected adults with pulmonary tuberculosis. Int J Tuberc Lung Dis 4:528–36.[ISI][Medline]
11
Gosling RD, Uiso LO, Sam NE, et al. (2003) The bactericidal activity of moxifloxacin in patients with pulmonary tuberculosis. Am J Respir Crit Care Med 168:1342–5.
12 Davidian M and Giltinan DM. (1995) Non–Linear Models for Repeated Measurement Data (Chapman Hall, London).
13 Pinheiro JC and Bates DM. (2000) Mixed-Effects Models in S and S-Plus (Springer Verlag, New York).
14 Hsieh FY, Lavori PW, Cohen HJ, et al. (2003) An overview of variance inflation factors for sample-size calculation. Eval Health Prof 26:239–57.[Abstract]
15 Kang D, Schwartz JB, Verotta D. (2004) A sample size computation method for non-linear mixed effects model with applications to pharmacokinetics models. Stat Med 23:2551–66.[CrossRef][ISI][Medline]
16 Retout S, Mentre F, Bruno R. (2002) Fisher information matrix for non-linear mixed-effects models: evaluation and application for optimal design of enoxaparin population pharmacokinetics. Stat Med 21:2623–39.[CrossRef][ISI][Medline]
17 R version 2.01, R Development Core Team. (2004) R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, available at http://www.R-project.org.
18
Gillespie SH, Gosling RD, Charalambous BM. (2002) A reiterative method for calculating the early bactericidal activity of antituberculosis drugs. Am J Respir Crit Care Med 166:31–5.
19
Dore CJ and Nunn AJ. (2003) Bactericidal activity of antituberculous drugs. Am J Respir Crit Care Med 167:663.
20 Marquardt DW. (1970) Generalized inverses, ridge regression, biased linear estimation and nonlinear estimation. Technometrics 12:591–612.[CrossRef][ISI]
21 Seber CAF and Wild CJ. (2003) Nonlinear Regression (Wiley & sons, New Jersey).
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||