Abstract
Chronic kidney disease is currently on the rise and not only leads to ESRD necessitating dialysis or transplantation but also increases cardiovascular disease risk. Measurement of the GFR, the gold standard for assessing kidney function, is expensive and cumbersome. Several prediction formulas that are based on serum creatinine are currently used to estimate the GFR, but none has been validated in a large cohort of individuals with diabetes. The performance of two commonly used formulas, the abbreviated Modification of Diet in Renal Disease (MDRD) study formula for the GFR and the Cockcroft-Gault estimate of creatinine clearance, were examined against GFR measured by the renal clearance of iothalamate in 1286 individuals with type 1 diabetes from the Diabetes Control and Complications Trial (DCCT). The performance of these formulas was assessed by computing bias, precision, and accuracy. The DCCT participants had normal serum creatinine, unlike the MDRD patients, and somewhat lower creatinine excretion than subjects in the original cohort Cockcroft Gault, which led to biased and highly variable estimates of GFR when these formulas were applied to the DCCT subjects. The MDRD substantially underestimated iothalamate GFR, whereas the Cockcroft Gault formula underestimated it when it was <120 ml/min per 1.73 m2 and overestimated it when iothalamate GFR was >130 ml/min per 1.73 m2. Overall, only one third of the formula’s estimates were within ±10% of iothalamate GFR. By underestimating GFR, these formulas were likely to flag early declines in kidney function. Refitting the MDRD formula to the DCCT data gave a more accurate and unbiased prediction of GFR from serum creatinine; percentage of estimate within 10% of measured GFR increased to 56%. A substantial variability in the estimates, however, remained.
Using serum creatinine to estimate true renal function has well-recognized inaccuracies and limitations (1). The difficulty and expense of direct measurement of the GFR, the gold standard in assessing renal function, have prompted development of prediction formulas for individuals with a variety of kidney diseases (2,3). In response to the increasing recognition that an alarming number of individuals have elevated serum creatinine, a series of publications examining the validity of the different GFR prediction formulas have appeared (4–10). The National Kidney Foundation Kidney Disease Outcomes Quality Initiative (K/DOQI) guidelines recommend estimating GFR in patients who are at risk for kidney disease using the Modification of Diet in Renal Disease (MDRD) study formulas (11). More recently, the National Kidney Disease Education Program has advocated that clinical laboratories provide an MDRD estimate of the patient’s GFR next to the serum creatinine value for any estimated GFR values that are <60 ml/min per 1.73 m2.
Estimating GFR in individuals who have diabetes and a normal serum creatinine has been problematic in the absence of a model that is based on clinical data in these patients. Both the Cockcroft-Gault creatinine clearance (CG Clcr) estimate and the MDRD formulas were developed in nondiabetic individuals (2,3).
Because careful validation of the CG Clcr and MDRD formulas in normoalbuminuric patients who have diabetes and preserved kidney function has not been carried out and recognizing that detection of early declines in GFR in individuals who have diabetes and do not uncommonly exhibit hyperfiltration is of potential importance, we used the Diabetes Control and Complications Trial (DCCT) public database to compare measured GFR with that calculated from CG Clcr and an MDRD prediction equation.
Materials and Methods
This was a retrospective analysis based on publicly released data from the DCCT (12). The DCCT was a randomized, controlled, clinical trial that studied the effects of intensive insulin therapy on the development and progression of microvascular complications of type 1 diabetes (13). At entry, subjects were 13 to 39 yr of age, had type 1 diabetes for 1 to 15 yr, had a serum creatinine of <1.2 mg/dl, and were normotensive by the standards used when the trial was initiated (<140/90 mmHg). The primary and secondary prevention cohorts were defined by absence or presence, respectively, of diabetic retinopathy. Data for the primary and secondary prevention cohorts were analyzed separately but reported in combination, because there were no statistically significant differences in the outcomes of interest. GFR was measured three times during the DCCT—at study entry, year 3, and closeout—but was not measured on all participants. Only the analysis of the closeout measurements is presented; results from the earlier measurements were similar. We present results from 1286 DCCT participants, comprising all available closeout GFR measurements after omitting nine subjects with <2 yr study participation at closeout, seven subjects with GFR <60 ml/min per 1.73 m2, and one subject with a GFR >350 ml/min per 1.73 m2.
Laboratory Methods
Measurement of GFR was implemented after the DCCT was already in progress and was determined from the renal clearance of 125I iothalamate (iGFR) using four consecutive timed urine collections and five serum samples bracketing these urine collections. The coefficient of variation (CV) among the four clearance periods was 11.7% (14).
All serum creatinine measurements for DCCT were performed at the University of Minnesota Laboratories using a Beckman CXR rate method using the Jaffe reaction. The CV for the measurement was 2%.
Serum Creatinine Calibration
Large differences in calibration of the serum creatinine assay across laboratories and, by extension, the prediction models that depend on them influence accuracy and bias of these formulas (15,16). Therefore, we compared the mean serum creatinine in the DCCT cohort with those of the “calibrated” Third National Health and Nutrition Examination Survey and found them to be virtually identical (Table 1). More important, in 2004, we sent to the MDRD laboratory 24 samples for creatinine measurement (range, 0.6 to 2.2 mg/dl) that were obtained from current clinical care subjects and compared them with the University of Minnesota laboratories, where the DCCT creatinines were performed. All but three determinations were identical (Figure 1). The mean difference (MDRD laboratory − University of Minnesota laboratory) was 0.0125 mg/dl, with SD 0.03 mg/dl, and the Pearson correlation coefficient was 0.9965.
Serum creatinine (SCr) measurements (mg/dl) from divided specimens done at the Cleveland Clinic laboratories, where the Modification of Diet in Renal Disease (MDRD) assays were performed, and at the Fairview-University of Minnesota Laboratories, where the Diabetes Control and Complications Trial (DCCT) performed its creatinine assay. Measurements of multiple specimens at 0.6, 0.7, 0.8, and 0.9 mg/dl were separated slightly so that points would not coincide.
Mean, median, and 95th percentile value for serum creatinine in NHANES III, “calibrated” NHANES III samples, and DCCT cohorta
Statistical Analyses
We used the measurements of the iGFR adjusted for body surface area (BSA), serum creatinine (SCr), and body surface area (BSA) as given in the DCCT data set: Variables GFR_ADJ99, BCVAL13, and BSA, respectively, from closeout data set CBL10; age and weight were computed at closeout. GFR was estimated using the formulas of Cockcroft-Gault and abbreviated MDRD estimated GFR (2,17) given below:
The Cockcroft-Gault formula predicts creatinine clearance:
We adjusted the creatinine clearance estimate for body surface area by multiplying by (1.73/BSA):
We used the abbreviated MDRD study equation (11):
This MDRD formula was used because measurements required for other extended MDRD equations (3,17), such as serum urea nitrogen, were not available in the DCCT data.
We assessed the performance of the CG Clcr and MDRD-GFR in several ways:
-
Bias: the average prediction error = Σ (estimated GFR − iGFR)/n, where n is the number of GFR studies performed.
-
Precision: the value of R2 from the linear regression of iGFR on estimated GFR.
-
Accuracy: mean of absolute errors as percent of iGFR = (1/n) − [100% × |iGFR − estimated GFR|/iGFR]
-
Relative accuracy: the percentage of estimates falling within 10, 30, and 50% of the measured iGFR.
Results are expressed as mean ± SD, unless indicated otherwise. Analyses and graphs were completed using statistical software R and SAS (18,19). Smooth estimates of the mean in the figures were computed using the lowest function in R, which does not assume a linear form and permits a visual check of linearity.
Results
Characteristics of the 1286 DCCT participants with closeout iGFR used in this analysis are summarized in Table 2. The distribution of iGFR values followed a bell curve with a mean of 122 ± 23 ml/min per 1.73 m2 and a few outlying high values as large as 300 ml/min per 1.73 m2. Several of the graphs have been cropped to focus on the bulk of the data and omit these outliers.
Characteristics of the DCCT cohort at closeouta
Cockcroft-Gault Estimate
Figure 2 displays the CG Clcr estimates on the horizontal axis (the known values, in practice) against their corresponding iGFR. This plot shows clearly two aspects of the relation between the CG Clcr estimates and iGFR. First, high variability: At each value of CG Clcr between 90 and 150, observed iGFR range over nearly 100 units. The converse is also true: For a fixed iGFR value between 90 and 150, the range of CG Clcr is nearly 100 units. The smooth estimate of mean iGFR as a function of CG Clcr suggests that the relation is linear for these data, but CG Clcr explains only 10% of the variability in iGFR (R2 from linear regression). Second, it is also evident from the smooth estimate of mean iGFR that lower values of CG Clcr (CG Clcr <120, to the left of the gray strip) were more likely to underestimate iGFR, whereas higher values (CG Clcr >130, to the right of the gray strip) were more likely to overestimate iGFR. Mutual cancellation of these errors is the reason that the overall CG Clcr mean of 116 ± 21 ml/min per 1.73 m2 is so close to the overall mean iGFR.
Measured 125I iothalamate GFR (iGFR; ml/min per 1.73 m2) versus Cockcroft-Gault creatinine clearance (CG Clcr) estimates for 1286 participants in the DCCT. The diagonal gray line is the line of identity, and the solid horizontal curve is a smooth estimate of mean iGFR at each value of CG Clcr. The gray strip includes all iGFR corresponding to CG Clcr between 120 and 130 units.
These trends are quantified in Table 3, where each row summarizes the CG Clcr estimates within a 10-unit-wide vertical strip of Figure 2 by listing the percentage of CG Clcr estimates within intervals around iGFR. The gray strip in Figure 2 includes CG Clcr between 120 and 130 ml/min per 1.73 m2, and the corresponding row of Table 3, labeled 120 to 130, states that 42% of CG Clcr estimates were within 10 units of the iGFR, that a total of 31% were below by 10 units or more, and a that total of 27% were above by 10 units or more. Rows of Table 3 above this, corresponding to CG Clcr <120, show that CG Clcr underestimates iGFR for a majority of individuals, whereas the rows below show that CG Clcr >130 overestimates iGFR for a majority of individuals. Overall, CG Clcr was within ±10 units for only 33% of the DCCT measured iGFR.
Performance of CG-GFR in predicting iGFR from the DCCT closeout measurementsa
Table 3 also shows the 90% tolerance interval (TI) for each 10-unit range of CG Clcr estimates. The 90% TI is the range of the middle 90% of observed iGFR and extends from the 5th to the 95th percentile of the observed iGFR corresponding to estimates within the given 10-unit interval. The 90% TI are extremely wide and almost identical for all strata of CG Clcr.
This high variability in the CG Clcr estimates is possibly explained if one recalls that Cockcroft and Gault developed their formula in two stages (2). They first performed linear regression to estimate creatinine excretion/kg body wt as (28 − 0.2 × age) in a sample of 249 men aged 18 to 92 yr. This linear function then was substituted into the formula for creatinine clearance:
Cockcroft and Gault reported their data as means and SE by decade (2), and, as shown in Figure 3, there was a fairly strong negative linear association between age and creatinine excretion for men. This figure also shows creatinine excretion from the DCCT with a smooth estimate of mean creatinine by age, separately for men and women. Mean creatinine excretion in the DCCT men who were older than 30 matched the decade averages in the Cockcroft and Gault sample, but DCCT men who were under 30 had mean creatinine excretion of 20.6 ± 5.7 mg/kg per 24 h, significantly less than the mean of 23.6 ± 5.0 for men in the Cockcroft and Gault sample (t test, P = 0.008). DCCT men all were <50 yr of age and showed a very weak association between creatinine excretion and age. Thus, the age variable in the CG Clcr estimate had limited usefulness for the DCCT participants.
Creatinine excretion (mg/kg per 24 h) versus age (years) at closeout for 1286 participants in the DCCT. Men are indicated by open circles, females by gray circles, and a small amount of horizontal noise has been added to each age to spread out overlapping points. The solid horizontal curve is a smooth estimate of mean creatinine excretion for DCCT men at each age; the dashed horizontal curve is a smooth estimate of mean creatinine excretion for DCCT women at each age. The filled squares and vertical error bars are means and SD of creatinine excretion (mg/kg per 24 h) by age decade for 249 men, reported by Cockcroft and Gault (2).
Abbreviated MDRD Estimate
MDRD-GFR estimates are plotted against their corresponding iGFR values in Figure 4. Like Figure 2, this plot shows high variability: For each value of MDRD-GFR between 75 and 140, iGFR measurements range 80 to 100 units. The smooth estimate of mean iGFR suggests that the relation is linear for these data, but MDRD-GFR explains only 12% of the variability in iGFR (R2 from linear regression). Second, the smooth estimate of mean iGFR shows that MDRD-GFR throughout the range 60 to 130 systematically underestimated iGFR in these data. The MDRD-GFR estimates had mean 110 ± 19 ml/min per 1.73 m2, well below mean iGFR. This large bias downward is quantified in Table 4, which uses the same format as Table 3 to list the percentage of MDRD-GFR estimates within intervals around iGFR. Overall, only 22% of MDRD-GFR estimates were within ±10 units of measured iGFR, whereas 71% underestimated iGFR by >10 units.
Measured iGFR (ml/min per 1.73 m2) versus MDRD-GFR estimates for 1286 participants in the DCCT. The diagonal gray line is the line of identity, and the solid horizontal curve is a smooth estimate of mean iGFR at each value of MDRD-GFR. The gray strip includes all iGFR corresponding to MDRD-GFR between 120 and 130 units.
Performance of the abbreviated MDRD formula in predicting iGFR from the DCCT closeout measurementsa
The MDRD estimate was developed directly by selecting predictors in linear regression with log(GFR) as the response (3). However, the MDRD sample on which the estimate was derived had much lower values of both GFR and the main predictor, reciprocal SCr, than the DCCT participants, as shown in Figure 5. In the MDRD sample, mean GFR was 40 ± 21 ml/min compared with 123 ± 22 ml/min in the DCCT; mean SCr was 2.3 ± 1.2 mg/dl in the MDRD sample and 0.85 ± 0.2 mg/dl in the DCCT sample. Figure 5 also shows that at each observed value of 1/SCr, variability of iGFR was much higher in the DCCT data than in the MDRD sample. The smooth estimate of the mean iGFR as a function of 1/SCr has a smaller slope than the MDRD data. More critical, even over the range of 1/SCr common to both MDRD and DCCT samples, the relation between 1/SCr and iGFR appears different in the two samples.
Measured iGFR (ml/min per 1.73 m2) versus 100 × reciprocal serum creatinine (100 dl/mg). Open circles are DCCT observations, and a small amount of horizontal noise has been added to the reciprocal serum creatinine values to spread out overlapping data. The diagonal gray line is the line of identity, and the solid horizontal curve is a smooth estimate of mean iGFR at each value of reciprocal serum creatinine for DCCT participants. The gray area approximates the MDRD data reported by Levey et al., with axes reversed (3). The boxplot at the bottom shows the minimum, 25th percentile, median, 75th percentile, and maximum of 100 × reciprocal serum creatinine for the DCCT participants, corresponding to a range of serum creatinine from 0.4 to 1.5 mg/dl. The bar at the bottom shows the approximate range of the MDRD data, corresponding to a range from 0.8 to 8.3 mg/dl.
To correct for this different relation, we took the predictors in the MDRD-GFR (SCr, age, gender) and refitted the same linear regression equation on a randomly selected subset of the DCCT data (the training subset, n = 815). The number of observations from black participants was too small to estimate an adjustment factor reliably, so these observations were omitted from the training and the test subsets. The refitted equation was:
The MDRD-GFR* expression uses the same variables as the original MDRD-GFR estimate, but the coefficients have been found from the DCCT data (the training subset). This is a substantial modification of the MDRD-GFR formula because the coefficients for SCr and age on the log scale were reduced by factors of almost 3 and 2, respectively.
Figure 6 shows the refitted MDRD-GFR* estimates applied to the remaining DCCT observations not used in its fitting (the test subset, n = 456). Although the high variability of iGFR at each value of MDRD-GFR* persists here (R2 is only 13%), the smooth estimate of iGFR coincides with the line of identity, indicative that the estimate is unbiased for these data. Table 5 repeats the format of Tables 3 and 4 to confirm these two characteristics: Refitted MDRD-GFR* had roughly symmetric 20 to 30% under- and overestimates throughout its range. Overall, 56% of the MDRD-GFR* estimates on the test subset were within ±10 units of the measured iGFR, more than a twofold improvement over the original MDRD equation.
Measured iGFR (ml/min per 1.73 m2) versus the refitted MDRD-GFR* estimates for n = 456 participants in the DCCT (the test subset), at the same scale as Figure 4. The gray diagonal line is the line of identity, and the solid horizontal curve is a smooth estimate of mean iGFR at each value of MDRD-GFR*. The gray strip includes all iGFR corresponding to MDRD-GFR* between 120 and 130 units.
Performance of the refitted MDRD-GFR* in predicting iGFR on the test sample from the DCCT closeout data (n = 456)a
Comparison of Estimates
The bias, precision, accuracy, and relative accuracy of the prediction equations are listed in Table 6. MDRD-GFR had larger bias but greater accuracy than CG Clcr. The refitted MDRD-GFR* was able to remove the bias while increasing accuracy and shows a marked improvement over both MDRD-GFR and CG Clcr. Table 6 also shows these comparisons stratified by estimated GFR and gender.
Bias, precision, and accuracy of the CG Clcr, MDRD-GFR, and refitted MDRD-GFR* as estimates of iGFRa
Multiple determinations of iGFR at closeout in the DCCT allowed us to calculate within-participant CV. We repeated the analyses of CG Clcr and MDRD-GFR restricted to iGFR measurements with CV ≤10% (n = 211). Neither the bias nor the accuracy of these formulas was improved (Table 6). The performance of the three prediction model is also depicted in Figure 7 using Bland Altman graphs.
Measured iGFR (ml/min per 1.73 m2) versus differences: (iGFR − CG Clcr) in the top panel, (iGFR − MDRD-GFR) in the middle panel, both calculated from 1286 participants in the DCCT; and (iGFR − MDRD-GFR*) calculated for the test subset (n = 456) in the bottom panel. The dashed lines indicate ±2 SD of the differences in each panel. The percentage of observations above and below the dashed lines is given in each panel.
Discussion
The bias, precision, and accuracy varied widely depending on the level of renal function. The Cockcroft-Gault estimate was less biased and more accurate at a GFR of 60 to 120 ml/min per 1.73 m2 than the abbreviated MDRD formula estimate. The latter, however, was more accurate at GFR >120 ml/min per 1.73 m2.
In the range of GFR observed in the DCCT population and analyzed for this validation effort (60 to 300 ml/min per 1.73 m2), the MDRD and the CG Clcr formulas generally underestimated renal function at levels of GFR that were <120 ml/min per 1.73 m2 and overestimated renal function at levels of GFR >120 ml/min per 1.73 m2. In other words, individuals with levels of renal function between 60 and 120 ml/min per 1.73 m2 are more likely to have a measured GFR that is higher than that predicted by the equations. Therefore, the formulas are less likely to miss individuals with early decrements in renal function. The overestimation at levels >120 ml/min would cause one to label an individual as a “hyperfilterer” when in reality he or she is not. Hyperfiltration is a putative risk factor for the initiation and progression of diabetic kidney disease, and its detection is very critical if one is to target this high-risk group of individuals who exhibit this phenomenon (20).
Recent evaluations of the CG Clcr and MDRD formulas in nondiabetic patients with relatively normal GFR have reported similarly modest agreement between these models and the gold standard (6–10). Many possible explanations have been offered, but largely differences between the patient populations studied and the population from which these models were derived have been incriminated.
The CG Clcr formula was validated in 249 patients who ranged in age from 18 to 92 yr with a creatinine ranging between 0.99 and 1.78 mg/dl in a predominantly male population (96%) with no information about disease status. Because the CG formula was designed to predict 24-h creatinine clearance and not GFR, it is not surprising that it performs poorly when used to estimate GFR. The MDRD formula, however, was developed from 1628 individual with a mean age of 50.6 ± 12.7 yr. It included patients with serum creatinine between 1.2 and 7 mg/dl and purposefully excluded patients with a GFR >70 to 80 ml/min per 1.73 m2 and those with diabetes. Therefore, its limited utility in the DCCT subjects is not surprising.
Published validations of the CG and MDRD formula in individuals with early diabetes have produced variable results (4,5,21–23). We know of one previous publication that evaluated the extended version of the MDRD formula in a similar population of individuals. Vervoort et al. (4) assessed the validity of the MDRD prediction equation by using inulin as the gold standard in a cross-sectional study involving 46 individuals who had type 1 diabetes and were normoalbuminuric, normotensive, and without retinopathy. In their analysis, the bias of the MDRD formula in predicting inulin GFR was 18.8 ml/min per 1.73 m2. In regard to accuracy, 50 and 90% of individuals studied had a GFR that was within 24 and 32% of the inulin-predicted GFR, respectively. The CG formula performed somewhat better than the MDRD with a bias of 15.1 ml/min per 1.73 m2. The authors note the high percentage of white patients in their sample as a limitation. A major limitation engendered by the small sample size, however, was that their evaluation could not determine the accuracy of the equations at a wide range of renal function. Because data from the Vervoort et al. study and the data presented here found that the CG performed better in patients with diabetes, clinicians may want to consider using this equation instead of the MDRD when evaluating patients who diabetes and GFR between 70 and 120 ml/min per 1.73 m2.
The DCCT data offer the largest collection of iGFR measurements to date for evaluating the performances of the CG Clcr and MDRD-GFR estimates of GFR in patients who have type 1 diabetes and normal serum creatinine. Despite the size of the data, comparisons with the data samples used earlier to develop the CG Clcr and MDRD-GFR estimates reveal several critical gaps and differences. First, the DCCT sample of iGFR contained no individuals who were older than 50. Despite the lack of appreciable differences in daily creatinine excretion rate for those who were older than 30 yr, individuals who were younger than 30 differed significantly from the sample of Cockcroft and Gault. Second, the DCCT sample as a group had lower values of serum creatinine than the MDRD sample, and the relationship between iGFR and 1/SCr did not seem to continue linearly from the MDRD cohort to that of DCCT. The refitted MDRD-GFR* offers a marked improvement over the two other formulas as a general formula to estimate GFR in patients with diabetes but should be used with caution considering the above mentioned caveats.
Obesity does not seem to explain the poor performance of these equations, because similar results were obtained by analyzing only those with body mass index <30 kg/m2 (data not shown). This is of critical importance in this cohort because the strict control group was heavier at closeout (13). Moreover, data obtained at study entry and at 3 yr were obtained before the differences in weights emerged and were similar to the closeout (data not shown).
Although direct calibration of the serum creatinine was not performed, we believe that, because the models underestimated renal function at relatively lower levels of GFR and overestimated renal function at relatively higher levels in our analysis, it is unlikely for a systematic variation in measurements to explain simultaneously both underestimation and overestimation of the model. One cannot rule out, of course, that a nonconstant calibration bias may exist. Moreover, the indirect calibration that was performed reassures against the possibility of variation in the serum creatinine as a culprit for the poor performance of these formulas. Whether the application of the more detailed, six- and seven-variable MDRD equations would have produced better results is unclear but is also unlikely considering that the simplified MDRD formula correlates well with the other MDRD prediction equations. Moreover, restricting the comparison to iGFR with CV ≤10% eliminates the gold standard as a culprit for the poor performance of these formulas. It is of note that previous validation efforts of creatinine-based formulas have ignored the possibility of the gold standard’s being at fault.Our results clearly show that the CG estimate of GFR predicts more accurately renal function in patients with diabetes in the normal GFR range (60 to 120 ml/min per 1.73 m2). Clinicians, therefore, should use the BSA-adjusted CG estimate in this range of GFR. The abbreviated MDRD formula, however, proved superior at GFR range that is >120 ml/min per 1.73 m2. Both prediction models, however, remain better tools than the serum creatinine alone in assessing renal function. One should consider the formula that we propose from this analysis in individuals in whom a closer estimate of GFR is needed. Validating the proposed formula in other populations is required.
Footnotes
Published online ahead of print. Publication date available at www.jasn.org.
- © 2005 American Society of Nephrology