Validation and customization of the SAPS 3 score in Brazilian ICUs
The severity of illness scores is routinely used to evaluate the performance and efficiency of intensive care units (ICUs) for the benchmarking and assessment of temporal severity-adjusted trending of mortality.(1–3) These scores are also used in clinical trials and observational studies to characterize and stratify subgroups of patients in terms of illness severity. Therefore, these instruments provide valuable clinical and administrative information.
However, model performance inherently varies across different settings because of differences in case-mix, clinical management, admission and discharge policies, among other factors.(3, 4) Moreover, the performance of these scores is also expected to deteriorate over time, particularly in terms of calibration.(5, 6) Therefore, the severity of illness scores must be validated prior to their use in a specific setting or geographic region and reassessed periodically to evaluate whether their performance remains appropriate.
The Simplified Acute Physiology (SAPS) 3 score was published in 2005, and it was developed from a database with 16,784 patients who were admitted to 303 ICUs from 35 hospitals in 35 countries, including Brazil.(4) Since 2009, the Brazilian Association of Intensive Care (Associação de Medicina Intensiva Brasileira, AMIB) has defined the SAPS 3 score as the recommended severity of illness score for assessing ICU performance and benchmarking Brazilian ICUs.(7) This decision has been supported by different multicenter studies.(8–10) The last large and multicenter validation study was published in 2017 and used data from 48,818 patients who were admitted to 70 ICUs in 50 hospitals during 2013.(8) In this study, the SAPS 3 standard equation (SAPS 3-SE) had good discrimination and calibration, but the customized equation for Central and South American countries overestimated mortality.(8) Recently, the critical care setting was seriously challenged by the worldwide COVID-19 pandemic, with significant changes in patient clinical management and ICU organization and management.(11) Therefore, validation of the SAPS 3 in a post-pandemic critical care population is needed to assess whether this model still performs well. In addition, the SAPS 3 provides information for evaluating ICU efficiency using standardized resource use (SRU). Nevertheless, the parameters and metrics used to estimate the SRU were reported in 2007 by Rothen et al. using the SAPS 3 original dataset, and to the best of our knowledge, they have not been revalidated.(12) The present study aimed to evaluate SAPS 3 performance in a contemporary cohort of patients who were admitted to ICUs participating in the Brazilian ICU Registry (UTIs Brasileiras)(13) and investigate whether it needs to be customized.
