A Clinical Score to Predict Acute Renal Failure after Cardiac Surgery

Materials and Methods

Results

The observed overall frequency of ARF requiring dialysis in the test data set was 1.7% (n = 269). Tables 2 and 3 show the distribution of the risk factors used to develop the scoring model according to the occurrence of ARF. By univariate comparison, patients with ARF were more likely to have preoperative risk factors such higher serum creatinine level, presence of diabetes, chronic obstructive pulmonary disease, previous cardiac surgery, markers of severe cardiovascular disease, and female gender, as well as intraoperative risk factors such as longer CPB time.

Table 1 shows those variables that were selected for the final scoring model. Age, weight, race, peripheral vascular disease, and cerebrovascular disease were excluded for the final scoring model on the basis of the results of bootstrap analysis. Table 1 also shows the logistic estimates, corresponding score points, and associated AUC for all of the risk factors. The resulting ARF scores (Table 4) range between a minimum of 0 to a maximum of 17 points (patients can belong to only one category of type of surgery or level of creatinine).

Table 5 shows the number of patients at each ARF score level and the corresponding frequency of ARF in both the test and the validation data sets. The frequency of ARF at each score level in the validation data set fell within the 95% CI of the ARF frequency at the corresponding score level in the test data set. This demonstrates the validity of the predictive model at all levels of the ARF score. Figure 1 illustrates frequency of ARF and the 95% CI at each score level in the two data sets.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Frequency of acute renal failure (ARF) that requires dialysis corresponding to ARF score in test and validation data sets.

There were fewer patients at higher score levels; no patients had a score of >13. We arbitrarily formed four risk categories of increasing severity (scores 0 to 2, 3 to 5, 6 to 8, and 9 to 13) to increase the number of patients in each risk category and to enhance the clinical utility of the score. The frequency of ARF across the four categories of severity in the test data set ranged between 0.5 and 22.1% (Figure 2). The frequency of ARF in each risk category of the validation data set also fell within the 95% CI of the corresponding ARF frequency in the test data set, as shown in Table 4.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2.

Frequency of ARF across risk categories in test and validation sets.

Figure 3 demonstrates the ROC curves generated to test and compare the diagnostic accuracy of the scoring model in the two data sets. The AUC for the score in the test data set was 0.81 (95% CI 0.78 to 0.83), whereas it was 0.82 (95% CI 0.80 to 0.85) in the validation data set; the area under the ROC curves was not significantly different (P = 0.39). We also calculated the AUC for the ROC curves for the score by excluding the total number of deaths in the overall cohort of patients. This allowed us to avoid the bias in prediction that may be introduced by differential survival among nondialyzed patients. The AUC for the test and validation sets were 0.82 (95% CI 0.79 to 0.86) and 0.84 (95% CI 0.80 to 0.87), respectively, and were statistically similar (P = 0.65).

• Download figure
• Open in new tab
• Download powerpoint

Figure 3.

Receiver operating characteristic curve analysis for the test and validation sets.

Discussion

ARF is one of the most serious complications after cardiac surgery; the occurrence of ARF strikingly magnifies the risk for postoperative death (5,8,9). There has been little improvement in our ability to treat renal injury and to demonstrate a survival benefit among this subgroup of patients (16). Preoperative renal risk stratification provides an opportunity to develop strategies of early diagnosis and intervention. The present study provides a clinical score to predict ARF after open-heart surgery and validates that score in a comparable population.

Few studies have addressed this issue. Chertow et al. (5) were among the first to develop a risk algorithm to predict postoperative ARF. This analysis involved a large multicenter cohort of patients (n = 43,642) who underwent cardiac surgery from the Veterans Administration health system. The population was predominantly men (99% men). The risk algorithm was subsequently assessed by Fortescue et al. (12) in a smaller cohort of patients (n = 8797). Although it included a larger proportion of women, this analysis did not include high-risk cardiac surgical procedures such as valve surgery. Both of these analyses excluded patients with severe preoperative renal dysfunction (creatinine >3 mg/dl). We also have analyzed a similar nonparametric recursive partitioning model in our cohort of patients (17). Although these were among the few analyses aimed at risk stratification involving large patient cohorts, there was a limitation in the statistical model that was used. It did not allow testing of the effect of various risk factors by order of their magnitude of association with ARF.

The present analysis includes all of the major risk factors of ARF. The estimates of prediction were subsequently converted into a clinically applicable score. The area under the ROC curve for the ARF score was 0.81 in the test data set and 0.82 in the validation data set. The ROC curve traditionally has been used as a method to describe the intrinsic accuracy of a diagnostic test as well as to compare various diagnostic tests; it can be interpreted as the probability that a randomly selected patient with a disease has a test result indicating greater suspicion of the disease than that of a randomly chosen patient without the disease (15). In other words, when applied to evaluate the accuracy of the ARF score in two randomly selected patients—one with and one without ARF—the probability is 0.82 that the patient with ARF will have a higher score than the patient without ARF. Thus, the present study enhances the accuracy of prediction by accounting for the interaction of all major risk factors by order of their degree of association with postoperative ARF.

In an earlier study, we identified the independent predictors of ARF after open-heart surgery. In addition to confirming the traditional risk factors in a large cohort of patients (n = 24,660), our earlier study identified female gender as an independent risk factor of postoperative ARF (8). The present analysis supplements earlier observations by developing a clinical ARF score that is validated on a randomly selected cohort of patients that was well represented by differences in gender and race. As indicated in Table 4, the frequency of ARF corresponding with each level of the score in the validation data set fell within the 95% CI of the ARF frequency in the test data set. Said in another way, at all levels of risk, the score was valid in predicting ARF when applied to a randomly selected cohort of patients.

A number of studies address the risk for ARF after open-heart surgery. When defined in its most severe form, as requiring dialysis, the reported frequency of postoperative ARF is usually <5% (5,6,18,19). This low event rate translates into two major limitations related to clinical research. First, it limits the ability of most of the epidemiologic studies to identify independent predictors of ARF, as a result of inadequate power (i.e., number of patients). Second, it impedes the development of clinical studies that pertain to early diagnosis and intervention in ARF by necessitating the enrollment of a large number of patients. This underscores the importance of risk stratification in clinical settings where renal injury can be anticipated.

The present study involves a large cohort of patients, sufficient to generate and validate a score that incorporates multiple independent risk factors, despite the low event rate of ARF. Furthermore, we arbitrarily divided the score into four risk categories (Table 4). This allowed optimizing of the number of patients who are at a moderate to high risk for developing ARF and improvement in the clinical utility of the score. The validity of the score was maintained even after the score levels were condensed into four risk categories. The frequency of ARF ranged between 0.4 and 22% across the risk categories. It should be noted that the overall frequency of ARF was 1.8%, so the risk categories allow identification of subgroups of patients who have lower- as well as higher-than-average risk for developing ARF. This can be a valuable tool used to randomize patients in clinical trials of ARF.

A weakness of the present study is that the data are derived from a single center. The model needs to be tested prospectively at multiple centers to substantiate its broad applicability. However, unlike other studies, it does include the single largest cohort of patients that is well represented by differences in gender, race, and all types of cardiac surgical procedures. Inherent to the observational study design is the limitation that it establishes association and not causality. Thus, it would be incorrect to justify a change in clinical decision making regarding open-heart surgery as a result of the risk for ARF on the basis of any such analyses. Nevertheless, it can improve individual patient care by allowing us to identify accurately patients who have a greater likelihood of developing ARF. It should be noted that the proposed scoring model predicts severe form of ARF defined by requirement of dialysis. Although less severe degrees of renal dysfunction (defined arbitrarily) may portend a risk for worse outcomes, the clinically relevant threshold of renal dysfunction after cardiac surgery remains unclear. Our rationale to choose this definition was based on clinical relevance and, more important, because ARF that requires dialysis after cardiac surgery has been unequivocally associated with mortality.

In conclusion, we provide a clinical score validated in our population of patients that predicts ARF after open-heart surgery. The score enhances the accuracy of prediction by accounting for the effect of all major risk factors of ARF. In addition, the score identifies patients who have a lower- as well as a higher-than-average risk for ARF. This increases the clinical utility of the score in improving both individual patient care and by providing a vital tool in planning future clinical trials of early diagnosis and intervention in ARF.