Leveraging the Capabilities of the FDA’s Sentinel System To Improve Kidney Care

FDA’s Sentinel and Its Capabilities

Sentinel was developed by the FDA in partnership with numerous health plans and scientific collaborators. Over time, Sentinel has created a large, national, multi-institutional, distributed data network of electronic health information and developed a sophisticated infrastructure that permits rapid and secure analyses, while preserving patient privacy.^3–⇓5 The partnership between the Sentinel Operations Center and the large network of data and academic partners enables access to rich electronic healthcare data and scientific and technical expertise. Sentinel and its capabilities enable rapid generation of high-quality, real-world evidence (Figure 1).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

An illustration of Sentinel's data assets and enhanced capabilities to generate real-world evidence using its privacy-preserving distributed network and infrastructure.

Sentinel’s national data network, the Sentinel Distributed Database, includes routinely collected electronic health data from 17 data partners^6,7 across the United States, totaling about 700 million person-years of observation. These data include administrative claims from several large national insurers and integrated delivery networks, components of electronic health records (EHRs) from a selection of integrated delivery networks, and inpatient EHRs from a network of 178 US hospitals. The Sentinel Distributed Database is a distributed network in that each data partner retains its data behind institutional firewalls and maintains both physical and operational control over the use of data—there is no central repository of data.^8,9 Each data partner regularly transforms their data into the Sentinel Common Data Model format, meaning that, despite the varied sources, the data are organized the same way for Sentinel analyses. Robust, routine processes are in place to ensure the data pass quality checks and are curated as expected.

The population in the Sentinel Distributed Database includes individuals covered by commercial plans including Medicare Advantage, beneficiaries of Medicare Fee-for-Service, and some Medicaid recipients. Longitudinal information on reimbursed medical and outpatient pharmacy services is captured for the covered population.¹⁰ Data partners refresh their datasets quarterly or annually, usually with an intentional 6- to 9-month lag to ensure completeness. Accelerated access to data in near real-time is possible when there are specific needs. Nearreal-time data have lags of as little as days for EHRs, and 1–2 months for claims data. Of note, accelerated access to data in near real-time typically requires trade-offs between completeness and timeliness, and incurs additional costs. Additionally, Sentinel routinely works with detailed inpatient data, including laboratory results from the nation’s largest hospital network that covers approximately 5% of inpatient care in the country.¹¹ The large size of the Sentinel Distributed Database⁷ and its inclusion of diverse populations enable evaluation of rare events¹² and generation of evidence, often with great precision. In addition, several data partners contribute laboratory result data that are typically unavailable in administrative claims databases, including results for tests like hemoglobin A1C, creatine kinase, and creatinine.¹³ Notably, approximately 47 million individuals in the Sentinel Distributed Database have at least one creatinine measurement. Although potential selection biases and other caveats apply to the use of this information, these measurements provide a foundation for assessment of renal function. Sentinel has also linked records for nearly 5 million mother-baby pairs, allowing routine evaluation of outcomes in neonates and infants after antenatal exposure to medical products.

To conduct analyses, scientists at the Sentinel Operations Center first work with scientific staff from the FDA to design studies and then create analytic computer programs, which the Operations Center distributes to individual data partners. Data partners opt in to run these programs on their data and return summary-level information to the Sentinel Operations Center, where it is aggregated across sites for reporting to the FDA and, ultimately, the public. In comparative analyses, Sentinel uses propensity score–based methods combined with risk set–based, data-sharing approaches to aggregate the summary-level data provided by the data partners to estimate overall effect estimates.^14,15 These analytic and data-sharing approaches produce results identical to those obtained from pooled patient-level data analysis. Patient-level analytic datasets can also be created and returned to the Sentinel Operations Center, when necessary. Shared patient-level datasets are either fully deidentified—for instance, through date shifting or grouping of ages and geographic locations—or they contain the minimum necessary information required to address the question at hand. These exchanges are facilitated by a software platform (PopMedNet) that ensures secure transmission of information in compliance with operational and governance processes of all participating organizations.¹⁶

Sentinel has developed a library of flexible “tools” (analytic computer programs, typically in the SAS programming language) to answer frequent questions from the FDA in a rapid manner, such as utilization patterns of medical products among specific cohorts of patients, occurrence of health outcomes of interest, and the evaluation of medication safety. Efforts are underway to determine the potential role of Sentinel in effectiveness research.¹⁷ These tools can be used to perform new-user cohort studies with propensity-score or multiple-factor matching, and self-controlled risk-interval analyses.¹⁸ Studies have shown that these tools are capable of reproducing well-established product-outcome associations, and, in general, produce findings comparable to those derived using de novo programming.^19,20 The library of tools also includes signal identification programs²¹ that use both cohort and self-controlled designs and perform analysis with tree-based scan statistics,^22,23 information component temporal pattern discovery,²⁴ or sequence symmetry analysis.²⁵

Enhanced Capabilities of Sentinel

In addition to routine querying using administrative data and the analytic tools described above, Sentinel data partners can obtain full-text medical records when necessary. When a chart review is required, direct identifiers in the charts are redacted and relevant information is abstracted before being used for expert adjudication. When requested, 70%–90% of records are obtainable for review. Reviews of full-text records have provided essential information for validating algorithms to identify important diseases¹² or to confirm case status for specific investigations.²⁶ When necessary, data partners can also link to external data sources, including birth or vaccine registries.^27–⇓29

Sentinel can also support the assessment and optimization of care delivery by facilitating the conduct of embedded randomized trials. The FDA created the FDA-Catalyst program to enable direct contact with patients and providers in Sentinel through collaboration with participating data partners. This program enhances the capabilities of Sentinel to support new research initiatives, moving beyond observational assessments of safety and effectiveness to interventional studies embedded in real-world delivery systems. The IMplementation of a randomized controlled trial to imProve treatment with oral AntiCoagulanTs in patients with Atrial Fibrillation (IMPACT-AFib) study is the first randomized, pragmatic, clinical trial through the FDA-Catalyst program. This large trial aims to evaluate the effect of patient- and provider-level education on the initiation of oral anticoagulants among patients with atrial fibrillation who may benefit from their use.^30,31 In this trial, the Sentinel infrastructure is used to identify eligible patients and providers and to conduct the analysis, assessing initiation of therapy and stroke and bleeding outcomes.

Sentinel can also facilitate effective collection of information directly from patients that is not routinely collected in administrative claims or EHRs. Augmenting patient-reported information with existing health data can bring the patient perspective into real-world evidence and expand the capacity of comparative effectiveness and safety studies. Sentinel has developed FDA MyStudies—a patient-facing, mobile-device application available in the public domain—to serve this purpose.^32–⇓34 In compliance with federal laws (Federal Information Security Management Act) and the Code of Federal Regulations (Title 21 CFR Part 11), the application facilitates the direct input of data by patients (e.g., outcomes, over-the-counter medication use, and other patient-experience data) and secure linkage of this self-reported data³⁵ to their information in the Sentinel Distributed Database, supporting prospective studies, registries, and randomized trials.

Limitations of Sentinel

Although Sentinel and its capabilities enable rapid generation of high-quality, real-world evidence, there are some limitations. The majority of the data in the Sentinel Distributed Database are administrative claims, which usually lack information on potentially important confounding factors such as body mass index, smoking, alcohol consumption, or other lifestyle-related factors. The Sentinel Distributed Database also does not have information on genetic markers of disease severity, and has varying access of laboratory test results and race information across data partners. Linking detailed inpatient hospital data, including exposure to specific medications, with claims data remains challenging. Individuals covered by commercial health plans also tend to have a shorter follow-up time than those covered by Medicare or Medicaid, making long-term treatment evaluations difficult. Moreover, Sentinel analyses have unknown generalizability to populations without health insurance. Lastly, Sentinel has limited information on patients with ESKD, for whom information on dialysis timing, modality, and outcomes may be incomplete. Important variability exists in missing laboratory results for serum creatinine across data partners.^36,37

How the FDA Has Used Sentinel

The FDA routinely uses Sentinel’s >10 billion pharmacy-dispensing and medical encounters to evaluate the utilization, safety, and effectiveness of marketed medical products. Evidence generated by Sentinel has informed revision of product labels or issuance of drug-safety communications. For example, during a safety assessment prompted by international evidence that suggested a higher risk of birth defects associated with etanercept use, the FDA used Sentinel to study the utilization of TNF-α inhibitors in pregnant women in the United States.³⁸ In 19,681 pregnancies where the expectant mother had at least one chronic inflammatory condition between 2004 and 2015, etanercept was found to be the most frequently used agent. This finding was one source of evidence considered by the FDA for the Pregnancy and Lactation Labeling Rule Conversion Safety Labeling Change for etanercept.³⁹

Findings generated from Sentinel analyses are also routinely discussed in advisory committee meetings at the FDA to inform regulatory decision making. In preparation for an advisory committee meeting focused on the development of opioid-sparing and opioid-replacement drugs for acute pain in November 2018, the FDA conducted an analysis within Sentinel to evaluate the duration of opioid analgesic use after surgeries.^2,40 The analysis included 820,767 patients initiating opioids immediately after one of 11 common surgical procedures. These data, along with other non-Sentinel studies, indicated that for certain indications, such as less-invasive procedures, opioid exposure could be reduced. The analysis also indicated variation in opioid analgesic needs by patient and procedure. These findings were discussed in the advisory committee meeting to inform guidance.^2,41–⇓43

The FDA is required to consider whether using the FDA Adverse Event Reporting System and Sentinel are sufficient to answer the question of interest before requiring manufacturers to conduct postmarketing studies that are often resource intensive.⁴⁴ Sentinel’s Active Risk Identification and Analysis (ARIA) System can efficiently conduct many safety analyses. From 2016 through 2019, the FDA conducted 394 analyses in ARIA.⁴⁵ As an example, Sentinel’s ARIA system was recently deemed sufficient to study the risk of renal cell carcinoma following treatment with canagliflozin.⁴⁶

Experience with Sentinel in Nephrology

Sentinel has been used to conduct several analyses of direct relevance to nephrology, assessing patient populations (e.g., acute or CKD), outcomes (e.g., kidney stones, rhabdomyolysis, and acute kidney failure), or exposures (e.g., nephrotoxic medications). Table 1 highlights select examples. Sentinel investigators developed algorithms to identify health outcomes, such as AKI⁴⁷ and rhabdomyolysis,⁴⁸ using administrative data. The system has also been used to perform many studies of conditions closely related to kidney disease, including hypertension, diabetes, or autoimmune diseases.²

These studies have informed a variety of research and regulatory decisions at the FDA. An example is a study in patients with gout. The Cardiovascular Safety of Febuxostat and Allopurinol in Patients with Gout and Cardiovascular Morbidities (CARES) trial reported increased cardiovascular mortality with febuxostat, a urate-lowering medication.⁴⁹ Prompted by these findings, the FDA used Sentinel to determine the generalizability of the CARES findings by evaluating real-world use of febuxostat. Compared with patients in the CARES trial, those initiating febuxostat in routine care were older, more likely to be female, with a lower burden of preexisting cardiovascular disease or CKD, and less severe gout.⁵⁰ This assessment contributed to a determination that there is a patient population with gout in which the risk-benefit profile of febuxostat is likely to be favorable for the treatment of hyperuricemia.²

Sentinel has also conducted several comparative analyses of nonkidney safety events following treatment with medications that interface with the kidney. In one study, Sentinel assessed the cardiovascular safety of drugs targeting the renin-angiotensin-aldosterone system (RAAS).⁵¹ Using an active-comparator, new-user study design and propensity score stratification to adjust for confounding, the study evaluated the risk of angioedema after initiation of treatment with angiotensin-converting enzyme inhibitors, angiotensin receptor blockers, the direct renin inhibitor aliskiren, or β-blockers. The study found a three-fold increased risk of angioedema following treatment with angiotensin-converting enzyme inhibitors or aliskiren relative to β-blockers, but not with angiotensin receptor blockers. Differential excess risks found in this study contributed to the understanding of comparative safety of these drugs, and demonstrated that not all drugs targeting RAAS have similar adverse event profiles.

Potential Applications of Sentinel’s Infrastructure and Capabilities in Nephrology

There are many opportunities to use Sentinel’s capabilities to address nephrology-related topics other than those directed by the FDA (Table 2). In this section, we discuss a selection of research themes that can be potentially addressed using its infrastructure. The Sentinel Distributed Database includes over a million patients with CKD.⁵² Existing analytic tools in Sentinel can be used to examine the longitudinal healthcare experience of patients with CKD and evaluate the safety of medical products used in this population. Furthermore, Sentinel can assess the burden of comorbidities and polypharmacy in the CKD population.

Sentinel’s safety signal identification and evaluation capabilities can be used to monitor kidney- or nonkidney-related adverse events of newly approved medications at a population level, including drug-associated nephrotoxicity manifesting as AKI. Although drug-associated nephrotoxicity is common, affecting 14%–26% patients worldwide,⁵³ it is often not well established at the time of market entry. Although many medications have a type-A nephrotoxicity pattern (dose related) and an acute onset (<7 days), medications with a type-B pattern (idiosyncratic) or a subacute or chronic onset may be difficult to detect in preapproval clinical trials.⁵⁴ Therefore, it is necessary to monitor and quantify risks postmarketing and identify conditions or characteristics that exacerbate these risks. As a recent example, immune checkpoint inhibitor therapy has been approved for a variety of malignant conditions including melanoma, renal cell carcinoma, non–small cell lung cancer, and urothelial carcinoma. As of 2018, about 44% of patients with cancer were eligible for treatment with one of these therapies.⁵⁵ Evidence from observational studies indicated an association between immune checkpoint inhibitors and drug-associated nephrotoxicity, in particular acute tubulointerstitial nephritis.^56,57 Yet, the size of these observational studies remains small compared with the scope of their use, which challenges the ability to draw conclusions about true incidence and risk factors for nephrotoxicity. Because drug-associated nephrotoxicity can be studied in claims data using validated algorithms,⁵⁸ Sentinel’s capabilities can be readily leveraged to evaluate acute and chronic nephrotoxicity associated with immune checkpoint inhibitor therapy in a more diverse, generalizable population.

Sentinel can also be used to study real-world effectiveness of newly approved kidney protective medications. For example, whereas clinical trials of sodium-glucose cotransporter-2 (SGLT2) inhibitors demonstrated a reduction in risk for incident or worsening nephropathy in patients with diabetes,^59,60 few observational studies had yet evaluated their effectiveness in real-world settings until recently.⁶¹ Sentinel could be used to address unanswered questions, including the role for SGLT2 inhibition without RAAS blockade and the safety of SGLT2 inhibition in the presence of diuretics.⁶² Approximately 475,000 patients in Sentinel initiated treatment with canagliflozin, dapagliflozin, or empagliflozin between March 2013 and June 2018.⁶³ With this large sample size and the overall capabilities of the system, including the ability to extract detailed clinical information from full-text medical records, robust evidence on real-world safety and, potentially, effectiveness can be generated.

Sentinel can also be leveraged to evaluate interventions targeting optimization of delivery of care in patients with CKD by conducting embedded, pragmatic trials. Data suggest that suboptimal delivery of care leads to elevated risk for cardiovascular morbidity and mortality for patients with CKD.⁶⁴ As an example, an embedded, pragmatic trial—analogous to the IMPACT-AFib study described above—might be used to assess various approaches to introduce statin therapy in patients with CKD in a rapid and resource-efficient manner. Such interventions could delay progression of kidney disease and improve cardiovascular outcomes in these patients.⁶⁵

Opportunities for Researchers to Leverage Sentinel’s Capabilities

Use of Sentinel and its enhanced capabilities by non-FDA investigators requires the oversight of an institutional review board and depends on voluntary participation by the Sentinel data partners. Opportunities for researchers, delivery systems, and others interested in leveraging the capabilities of Sentinel are described below.

Researchers in academia and industry may use Sentinel for multisite observational or interventional studies through collaborations with partner organizations, working directly with individual Sentinel data partners, or engaging with the Sentinel Operations Center. Investigations can be facilitated via several entities, including the Reagan-Udall Foundation’s Innovation in Medical Evidence Development and Surveillance (IMEDS) program,^66,67 the National Institutes of Health (NIH) Collaboratory Distributed Research Network,^68–⇓70 and the Biologics and Biosimilars Collective Intelligence Consortium (BBCIC).⁷¹ Recent IMEDS studies have focused on the incidence of venous thromboembolism among patients receiving routine clinical care for rheumatoid arthritis,⁷² the safety of oral contraceptives, and the effect of a drug class label change on medication use and outcomes.⁷³ Investigators funded by the NIH or other not-for-profit sponsors can initiate or participate in research via the NIH Collaboratory Distributed Research Network. Collaboratory Distributed Research Network topics relevant to nephrology have included drug-induced toxicity,⁷⁴ medication use in the elderly,⁷⁵ and reducing polypharmacy in at-risk populations.⁷⁶ Investigations that focus on biologics or biosimilars can leverage the BBCIC. Similar consortiums can also be created to focus on organ system–related topics, like kidney disease. Additionally, external investigators may work directly with Sentinel data partners and/or the Operations Center; several studies have already taken place through these collaborations.^77,78 Costs of conducting a research project with Sentinel may vary with the complexity of the effort, number of data partners, timeliness of data access, need to obtain original medical records, and other requirements. Researchers interested in initiating any of the aforementioned collaborations may first contact the Sentinel Operations Center using https://www.sentinelinitiative.org/contact-us.

Researchers may also directly benefit from Sentinel by using its publicly available resources available at www.sentinelinitiative.org. The Sentinel Common Data Model,¹³ analytic tools,^18,79 and data source translation code^80,81 are publicly available for researchers to transform their own data and run queries. Study protocols, specifications, and analytic packages used for regulatory analyses are also publicly available on Sentinel’s website to facilitate adaptation by the broader research community in other environments.

Disclosures

R. Ball reports he is an author on US Patent 9,075,796, “Text mining for large medical text datasets and corresponding medical text classification using informative feature selection.” E. Barreto provides consultation to FAST BioMedical for work unrelated to this manuscript. All remaining authors have nothing to disclose.

Funding

This project was supported by US FDA under master agreement number HHSF223201400030I. E. Barreto (principal investigator) is supported by a National Institute of Allergy and Infectious Diseases grant K23AI143882.