https://doi.org/10.4081/ecj.2025.13829
Evaluating the predictive accuracy of ChatGPT in risk stratification for chest pain in the emergency department
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.
Published: 6 June 2025
Chest pain is a frequent cause of emergency department (ED) visits, yet accurately assessing the risk of major adverse cardiac events (MACE) remains challenging. This study evaluated the potential of ChatGPT 4.0 as a clinical decision-support tool for predicting MACE in patients with chest pain. We conducted a prospective observational study at the Rovereto Hospital ED from March to August 2024, analyzing 178 patients. ChatGPT received patient data in three sequential phases: initial clinical history and ECG, first troponin test, and second troponin test. Its predictive performance improved with additional data, with the area under the receiver operating characteristic curve (AUROC) increasing from 0.699 in the first phase to 0.776 after the second troponin test (p=0.039). However, ChatGPT misclassified several MACE cases as non-urgent, raising concerns about sensitivity and risk stratification. While ChatGPT demonstrated potential in identifying MACE cases, its current performance does not support routine ED implementation. Further refinements in AI-based models are needed to improve real-time risk assessment and ensure safer, more reliable clinical integration.
Downloads
1. Byrne RA, Rossello X, Coughlan JJ, et al.; ESC Scientific Document Group. 2023 ESC Guidelines for the management of acute coronary syndromes. Eur Heart J 2023;44:3720-826.
2. Virani SS, Newby LK, Arnold SV, et al.; Peer Review Committee Members. 2023 AHA/ACC/ACCP/ASPC/NLA/PCNA Guideline for the Management of Patients With Chronic Coronary Disease: A Report of the American Heart Association/American College of Cardiology Joint Committee on Clinical Practice Guidelines. Circulation 2023;148:e9-e119.
3. Writing Committee Members; Gulati M, Levy PD, Mukherjee D, et al. 2021 AHA/ACC/ASE/CHEST/SAEM/SCCT/SCMR Guideline for the Evaluation and Diagnosis of Chest Pain: A Report of the American College of Cardiology/American Heart Association Joint Committee on Clinical Practice Guidelines. J Am Coll Cardiol 2021;78:e187-e285.
4. Zaboli A, Ausserhofer D, Sibilio S, et al. Effect of the emergency department assessment of chest pain score on the triage performance in patients with chest pain. Am J Cardiol 2021;161:2-18.
5. Backus BE, Six AJ, Kelder JC, et al. A prospective validation of the HEART score for chest pain patients at the emergency department. Int J Cardiol 2013;168:2153-8.
6. Laureano-Phillips J, Robinson RD, Aryal S, et al.. HEART score risk stratification of low-risk chest pain patients in the emergency department: a systematic review and meta-analysis. Ann Emerg Med 2019;74:187-203.
7. Nilsson T, Johannesson E, Lundager Forberg J, et al. Diagnostic accuracy of the HEART Pathway and EDACS-ADP when combined with a 0-hour/1-hour hs-cTnT protocol for assessment of acute chest pain patients. Emerg Med J 2021;38:808-13.
8. Thangaraj PM, Khera R. Accelerating chest pain evaluation with machine learning. Eur Heart J Acute Cardiovasc Care 2023;12:753-4.
9. Stewart J, Lu J, Goudie A, et al. Applications of machine learning to undifferentiated chest pain in the emergency department: A systematic review. PLoS One 2021;16:e0252612.
10. Günay S, Öztürk A, Özerol H, et al. Comparison of emergency medicine specialist, cardiologist, and chat-GPT in electrocardiography assessment. Am J Emerg Med 2024;80:51-60.
11. Hinson JS, Taylor RA, Venkatesh A, et al. Accelerated chest pain treatment with artificial intelligence-informed, risk-driven triage. JAMA Intern Med 2024;184:1125-7.
12. Dawson LP, Smith K, Cullen L, et al. Care models for acute chest pain that improve outcomes and efficiency: JACC state-of-the-art review. J Am Coll Cardiol 2022;79:2333-48.
13. Zhang PI, Hsu CC, Kao Y, et al. Real-time AI prediction for major adverse cardiac events in emergency department patients with chest pain. Scand J Trauma Resusc Emerg Med 2020;28:93.
14. Liu N, Chee ML, Koh ZX, et al. Utilizing machine learning dimensionality reduction for risk stratification of chest pain patients in the emergency department. BMC Med Res Methodol 2021;21:74.
15. Ting Sim JZ, Fong QW, Huang W, Tan CH. Machine learning in medicine: what clinicians should know. Singapore Med J 2023;64:91-7.
16. Sidey-Gibbons JAM, Sidey-Gibbons CJ. Machine learning in medicine: a practical introduction. BMC Med Res Methodol 2019;19:64.
17. Tan S, Xin X, Wu D. ChatGPT in medicine: prospects and challenges: a review article. Int J Surg 2024;110:3701-6.
18. Zaboli A, Brigo F, Sibilio S, et al. Human intelligence versus Chat-GPT: who performs better in correctly classifying patients in triage? Am J Emerg Med 2024;79:44-7.
19. Heston TF, Lewis LM. ChatGPT provides inconsistent risk-stratification of patients with atraumatic chest pain. PLoS One 2024;19:e0301854.
20. Wu CC, Hsu WD, Islam MM, et al. An artificial intelligence approach to early predict non-ST-elevation myocardial infarction patients with chest pain. Comput Methods Programs Biomed. 2019;173:109-17.
21. Tsai DJ, Tsai SH, Chiang HH, et al. Development and validation of an artificial intelligence electrocardiogram recommendation system in the emergency department. J Pers Med 2022;12:700.
How to Cite
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
PAGEPress has chosen to apply the Creative Commons Attribution NonCommercial 4.0 International License (CC BY-NC 4.0) to all manuscripts to be published.
