21 May ACS Surgical Risk Calculator Reliably Guides Decision Making
MedicalResearch.com Interview with:
Dr. Mark Cohen
Mark E. Cohen, PhD
Statistical Manager
Continuous Quality Improvement
Division of Research and Optimal Patient Care
American College of Surgeons
Chicago, IL
MedicalResearch.com: What is the background for this study?
Dr. Cohen: The ACS NSQIP Surgical Risk Calculator (built from 2.7 million patient records from nearly 600 hospitals) has been widely adopted as a decision aid and informed consent tool by surgeons and patients. Predictive accuracy can be assessed in terms of discrimination, calibration, and combined discrimination and calibration. In this study, we focused primarily on calibration. Calibration refers to the consistency of agreement between observed and predicted risk across the range of predicted risk. One would not want, for example, a model that dramatically overestimates risk for low-risk patients and underestimates risk for high-risk patients – this sort of systematic error, if of sufficient magnitude, would make a risk calculator unacceptable for clinical use. We also assessed the potential benefits of statistical recalibration using restricted cubic splines.
MedicalResearch.com: What are the main findings?
Dr. Cohen: Without recalibration, the Risk Calculator was shown to have excellent calibration, though there was, at times, a slight tendency for predicted risk to be overestimated for lowest- and highest-risk patients and underestimated for moderate-risk patients. After recalibration this distortion was eliminated.
MedicalResearch.com: What should readers take away from your report?
Dr. Cohen: While no risk calculator can provide perfect prediction, users of the ACS NSQIP Surgical Risk Calculator can presume that estimates are accurate for their intended purposes – they can be relied upon to guide surgical decision-making and generally inform patients of their probabilities for important adverse events.
MedicalResearch.com: What recommendations do you have for future research as a result of this study?
Dr. Cohen: In the literature, there are frequent comparisons between the ACS NSQIP Surgical Risk Calculator and other, special-purpose, risk calculators. These latter calculators target specific types of operations or specific types of patients and often use risk predictors unique to those operations and patients. While these “special” calculators might suggest the opportunity for more accurate prediction, they often rely on smaller datasets (potentially less reliable) from fewer hospitals (potentially less generalizable). At the present time, domains where the “general” (ACS NSQIP Surgical Risk Calculator) versus “special” risk calculators offer predictive advantages are not well established; this remains an area of active research.
MedicalResearch.com: Thank you for your contribution to the MedicalResearch.com community.
Citation:
Clifford Y. Ko, MD, MS, MSHS, FACS et al. Evaluation and Enhancement of Calibration in the American College of Surgeons NSQIP Surgical Risk Calculator. Journal of the American College of Surgeons, May 2016 DOI: 10.1016/j.jamcollsurg.2016.03.040
Note: Content is Not intended as medical advice. Please consult your health care provider regarding your specific medical condition and questions.
More Medical Research Interviews on MedicalResearch.com
[wysija_form id=”5″]
Last Updated on May 21, 2016 by Marie Benz MD FAAD