Published online Aug 28, 2024. doi: 10.4329/wjr.v16.i8.329
Revised: July 22, 2024
Accepted: August 5, 2024
Published online: August 28, 2024
Processing time: 211 Days and 1.4 Hours
With the increasingly extensive application of artificial intelligence (AI) in medical systems, the accuracy of AI in medical diagnosis in the real world deserves attention and objective evaluation.
To investigate the accuracy of AI diagnostic software (Shukun) in assessing ischemic penumbra/core infarction in acute ischemic stroke patients due to large vessel occlusion.
From November 2021 to March 2022, consecutive acute stroke patients with large vessel occlusion who underwent mechanical thrombectomy (MT) post-Shukun AI penumbra assessment were included. Computed tomography angiography (CTA) and perfusion exams were analyzed by AI, reviewed by senior neurointerventional experts. In the case of divergences among the three experts, discussions were held to reach a final conclusion. When the results of AI were inconsistent with the neurointerventional experts’ diagnosis, the diagnosis by AI was consi
A total of 22 patients were included in the study. The vascular recanalization rate was 90.9%, and 63.6% of patients had modified Rankin scale scores of 0-2 at the 3-month follow-up. The computed tomography (CT) perfusion diagnosis by Shu
AI (Shukun) has limits in assessing ischemic penumbra. Integrating clinical and imaging data (CT, CTA, and even magnetic resonance imaging) is crucial for MT decision-making.
Core Tip: Shukun CerebralDoc system is a Chinese artificial intelligence (AI) post-processing software for computed tomography angiography (CTA) and cerebral perfusion. It is currently the most widely used vascular post-processing AI software in hospitals in mainland China. There is still a lack of relevant laws or regulations in China regarding the clinical application of AI. We found that Shukun AI computed tomography (CT) perfusion imaging has certain limitations in assessing the ischemic penumbra with an inaccuracy rate of 13.6%. We highlight that comprehensive evaluation of the ischemic penumbra in combination with the clinical symptoms, signs, and imaging findings (CT, CTA, and even magnetic resonance imaging), rather than totally relying on AI results, is a key imperative prior to decision-making for mechanical thrombectomy.
- Citation: Li ZQ, Liu W, Luo WL, Chen SQ, Deng YP. Artificial intelligence software for assessing brain ischemic penumbra/core infarction on computed tomography perfusion: A real-world accuracy study. World J Radiol 2024; 16(8): 329-336
- URL: https://www.wjgnet.com/1949-8470/full/v16/i8/329.htm
- DOI: https://dx.doi.org/10.4329/wjr.v16.i8.329
Stroke is the second most common cause of death worldwide and the most common cause of death and disability in China[1]. Ischemic stroke accounts for approximately 85% of all cases of acute stroke, of which acute ischemic stroke (AIS) caused by large vessel occlusion (LVO) accounts for approximately 20% and is the leading cause of moderate to severe disability and death[2,3]. The use of alteplase for intravenous thrombolysis (IVT) is an effective treatment for AIS within the thrombolytic window[4]. However, for AIS with LVO (AIS-LVO), the recanalization rate is low, and the efficacy of IVT is unsatisfactory. A meta-analysis of five randomized trials on mechanical thrombectomy (MT) in 2016 indicated that for patients with anterior circulation AIS-LVO within 6 hours of onset, IVT combined with MT was associated with a higher immediate vascular recanalization rate and significantly reduced disability at 90 days compared with IVT, irrespective of the patient characteristics or geographical location[5]. In 2018, on the basis of the DAWN and DEFUSE3 studies, the therapeutic window for MT in patients with anterior circulation AIS-LVO was extended to 16-24 hours[6,7]. IVT combined with MT is currently the standard treatment for AIS-LVO[8,9]. Guidelines recommend computed tomography perfusion (CTP) imaging to evaluate the ischemic penumbra to select patients suitable for MT[8,9]. In the DEFUSE3 study, patients were screened based on perfusion images processed by RAPID artificial intelligence (AI)[7].
The RAPID system has indeed improved clinical accuracy and efficiency. However, many district hospitals, especially those in economically underdeveloped regions in China, are unable to afford the cost of purchasing RAPID software. Therefore, other similar and more affordable products can serve as alternative choices. Shukun (CerebralDoc, Beijing Shukun Technology Co., Ltd., Beijing, China), a Chinese AI processing software for computed tomography (CT) images of arteries and cerebral perfusion, was approved by the State Drug Administration in February 2021. In a comparative study, Shukun AI processing software was found to be superior to volume computed tomographic digital subtraction angiography (DSA) in displaying the carotid artery and extracranial segments of the vertebral artery[10]. In a study focused on head and neck arteries, the Shukun AI system achieved 100% accuracy in diagnosing aneurysms and arterial stenosis, as confirmed by DSA[11]. However, there are no reports on the reliability of Shukun AI in evaluating the ischemic penumbra to screen eligible patients for MT. Therefore, we performed a single-center study to investigate the accuracy of the Shukun software in evaluating the ischemic penumbra.
This was a retrospective single-center clinical study conducted at Huizhou Central People’s Hospital. Consecutive AIS patients with anterior circulation LVO who underwent MT after assessment of the ischemic penumbra using Shukun AI software between November 2021 and March 2022 were included.
Patients were required to meet the following conditions: Age ≥ 18 years; National Institute of Health Stroke Scale (NIHSS) score > 2; time from the last normal time to femoral artery puncture 6-24 hours; CTA findings indicative of occlusion of large vessels in the anterior circulation; core infarct volume [defined as cerebral blood flow (CBF) < 30%] < 70 mL; low perfusion volume [time to maximum (Tmax) > 6 seconds] > 15 mL; and low perfusion volume (Tmax > 6 seconds)/core infarct volume (CBF < 30%) > 1.8.
According to the contraindications for intravascular diagnosis and treatment of acute large vessel occlusive ischemic stroke in the Chinese Guidelines for Stroke Prevention and Treatment (2021 version)[12], patients with the following conditions were excluded from the study: (1) A history of intracranial hemorrhage in the preceding 3 weeks, or the presence of cerebral artery malformations or aneurysms without interventional or surgical treatment; (2) Intractable hypertension not controlled by medication (systolic blood pressure > 185 mmHg or diastolic blood pressure > 110 mmHg); (3) Known allergy to iodized contrast media; (4) Blood glucose < 2.8 mmol/L or > 22.0 mmol/L; (5) Acute hemorrhagic constitution, including coagulation factor deficiency, an international normalized ratio > 3.0 or a platelet count < 40 × 109/L; (6) A history of arterial puncture at an incompressible site within the last 7 days, a history of major surgery or severe trauma within the preceding 14 days, gastrointestinal or urethral bleeding within the preceding 21 days, a history of any disease in the preceding 3 months that increases the risk of bleeding (such as severe craniocerebral trauma, severe liver diseases, or ulcerative gastrointestinal diseases), and operations, biopsy of solid organs, or active bleeding within one month; (7) Suspected septic emboli or bacterial endocarditis; (8) Life expectancy < 90 days; and (9) Severe renal dysfunction.
One-stop multimode CT examination (256 slices Revolution CT Xtream, GE) was performed as soon as possible for these AIS patients. The images were postprocessed with Shukun (AI) software. Regions with a CBF < 30% represent the core infarct volume, and regions with a Tmax > 6 seconds represent a low perfusion volume. Iodofol (Jiangsu Hengrui Pharmaceutical Co. Ltd., China) was used as the contrast agent, with a total volume of 45 mL injected at a rate of 4.5 mL/second. Scanning was started 5 seconds after injection of the contrast agent, with a total scanning time of 60 seconds and a scanning thickness of 5 mm. The analysis output of AI (Shukun) was independently reviewed by three or more senior neurointerventional experts, and a comprehensive analysis based on the patient’s clinical manifestations, cranial CTA or magnetic resonance angiography was conducted to calculate the accuracy of AI (Shukun). In the cases of divergences among the three experts, discussions were held to reach a final conclusion. When the results of AI were inconsistent with the neurointerventional experts’ diagnosis, the diagnosis of AI was considered inaccurate.
All the data were analyzed and processed with SPSS 22.0 software. Continuous variables are expressed as the means ± SD, whereas categorical variables are expressed as frequencies (%).
A total of 22 patients (20 males, 2 females; mean age: 59 years) who met the inclusion criteria and underwent MT were enrolled. The mean NIHSS score was 10.7, and the vascular recanalization rate was 90.9%. At the 3-month follow-up, 63.6% of patients had a modified Rankin scale (mRS) score of 0-2. In 3 patients, the ischemic penumbra results assessed by the Shukun system were inconsistent with the signs, symptoms, or imaging findings. This corresponded to an inaccuracy rate of 13.6%.
Case 1: A 53-year-old man complained of speech disfluency and right limb weakness for 7 hours. Physical examination revealed mixed aphasia, right central facial and lingual paralysis, right limb muscle strength of Grade 1, right Babinski sign (+), and an NIHSS score of 17. One-stop multimode CT examination was performed immediately. The patient had some head movement during the examination. On plain CT scan, mild cortical sulcal effacement of the left cerebral hemisphere was found. AI did not suggest left cerebral hemisphere core infarction (Figure 1A). Preoperative CT per
Case 2: A 93-year-old man who had a history of atrial fibrillation but not anticoagulant therapy was admitted to the hospital due to “being found unconscious for 4 hours”, with 11 hours elapsed since he was last seen well. On physical examination, the patient was lethargic, with a Glasgow coma scale (GCS) score of 9, mixed aphasia, right limb muscle strength of Grade 0, right Babinski sign (+), and an NIHSS score of 19. CTP revealed that the volume of the core infarct area was 5.7 mL, the mismatch volume was negative, the low perfusion volume was 4.2 mL, and the mismatch ratio was 0.7 (Figure 2A). CTA revealed occlusion of the M1 segment of the left middle cerebral artery (Figure 2B). The patient had impaired consciousness, with an NIHSS score of 19, and disappearance of the sulci and gyri of the left cerebral hemis
Case 3: A 64-year-old man was transferred from a local hospital to our hospital due to left limb weakness and unclear speech for 12 hours. Physical examination revealed lethargy, a GCS score of 14, gaze to the right, a flattened nasolabial fold on the left, Grade 1 muscle strength of the left limb, and Babinski sign (+) on the left side. The patient’s NIHSS score was 13. CT revealed a low-density focus in the right basal ganglia. AI revealed that the core infarction volume was 30.4 mL, which was located mainly in the left cerebral hemisphere. The volume of the hypoperfusion area was 119.3 mL, and the mismatch ratio was 3.9 (Figure 3A). CTA revealed occlusion of both the initial segment of the right internal carotid artery and the M1 segment of the right middle cerebral artery (Figure 3B). The right internal carotid artery and right middle cerebral artery were completely recanalized after emergency MT, and the forward blood flow recovered to mTICI grade 3. A cranial CT scan performed 5 days after the operation revealed that the infarcted area was located in the right basal ganglia and that there was no obvious infarction in the left cerebral hemisphere (Figure 3C). AI misjudged the location of the core infarction in this patient.
The RAPID software developed by Stanford University in 2010 defines the infarct core area with an apparent diffusion coefficient threshold of 600 × 10-6 mm2/second, and a PWI Tmax > 6 second defines hypoperfusion. The software automatically applies these criteria to assess DWI/PWI mismatch in patients with AIS within 5 to 7 minutes, with 100% sensitivity and 91% specificity[13]. However, there are several disadvantages of selecting patients based on MR mismatch, as follows: (1) In acute stroke patients, the MRI scan time is 10-20 minutes longer than that with CT, which is completed in 10-15 minutes; (2) Most stroke centers do not have a 24 hours × 7 days facility for MRI; and (3) Metal implants (such as pacemakers) and claustrophobia are contraindications to MRI examination.
In contrast, CT perfusion image processing software can quickly and accurately assess LVO, the vascular path, collateral compensation, and the ischemic penumbra, and it has become the main imaging modality for the preoperative evaluation of LVO before MT. In our cohort, the proportion of patients with post-MT mTICI Grade 2b or 3 anterior circulation LVO who had undergone Shukun CT perfusion imaging was 90.9%, which is greater than that reported in the Defuse3 trial (69%)[7]. The percentage of patients with favorable functional outcomes, defined as an mRS score of 0-2 at 90 days, was 63.6%, which was also higher than that reported in the Defuse3 trial (44%)[7]. This discrepancy may be attributable to the continuous update of thrombectomy equipment at our center in recent years, the use of the Penumbra thrombus suction system for patients with cardiogenic cerebral embolism, greater institutional experience, and the strict screening and selection of patients via multimodality imaging. Shukun (China) automated CT perfusion imaging software is safe and effective in selecting patients who require MT. Furthermore, it is inexpensive and easy to popularize.
Although the diagnosis of core infarction and penumbra made by AI (Shukun) is mostly reliable, it still has a certain interpretative error rate. The interpretative error rate in our single-center study was 13.6%, indicating the presence of certain limitations and pitfalls of AI. The diagnosis of core infarction and ischemic penumbra by AI depends on the setting of the parameters. Not surprisingly, the use of different AI parameters to define core infarcts results in different core infarcts and penumbras, sometimes leading to false ischemic penumbras[14]. The baseline core infarct area suggested by AI is not necessarily the final infarct size. The earlier the recanalization, the smaller is the final infarct size. With late or no recanalization, the final infarct size will be larger than the baseline core infarct area assessed by AI. In the Shukun cerebral perfusion CT image processing software, the area with a CBF < 30% was defined as the core infarction, and a Tmax > 6 seconds was defined as the ischemic area. Some studies have suggested that for patients who have undergone thrombectomy within 90 minutes of onset, areas with a CBF < 20% are much closer to the final infarct size after surgery[15]. Even if optimized with an increasing interpretative rate of AI in the future, clinical physicians and radiologists should use AI as a medical device rather than completely relying on the data provided by AI[16].
The CTP scanning time can also affect the judgment of the AI system pertaining to the volume of the ischemic penumbra. AI software takes into account the entire process from the inflow of contrast agent from arteries to outflow through veins to obtain the arterial inflow and venous outflow curves for accurately calculating the blood volume in the ischemic area. For patients with LVO, the time for contrast agents to flow into the arteries is relatively delayed; thus, the already established inappropriate CTP scanning time is liable to have a certain impact on the lesion volume. If the scanning time is too short, the volume of the ischemic penumbra cannot be accurately estimated, whereas if the scanning time is too long, the patient will be exposed to unnecessary additional radiation. Currently, the recommended optimal scanning time for CTP is 60-70 seconds[17]. The scanning time in the present study was 60 seconds. However, the scanning time may be shortened by the operator if a patient is irritable or does not cooperate with the examination. Additionally, a low contrast medium concentration or low injection rate is also likely to cause errors in the CT perfusion results.
Finally, the CT perfusion results generated by AI can also be affected by artifacts. The essence of perfusion imaging is enhanced CT examination. Since CT perfusion examination is a multiphase examination, any situation that causes CT artifacts can cause inaccurate CT perfusion results. Stroke patients with anterior circulation LVO often have dysphoria and cannot cooperate with the CT perfusion examination in a quiet state. Therefore, the appropriate application of sedatives is recommended to reduce the occurrence of artifacts in restless patients.
Although CTP imaging of the brain is a clinically useful tool in multimodality imaging of acute stroke, there are limitations and pitfalls of AI. Clinical physicians and radiologists should not rely completely on the data provided by AI. In addition to the CTP results provided by AI, a comprehensive analysis of the clinical picture, plain CT findings, CTA, and even MRI should be performed before deciding whether to perform MT. In addition, it is critical to pay attention to factors that may lead to AI inaccuracies, such as head movement during the CT examination and the speed of contrast agent injection.
We are grateful to all the participants in this study.
1. | Wu S, Wu B, Liu M, Chen Z, Wang W, Anderson CS, Sandercock P, Wang Y, Huang Y, Cui L, Pu C, Jia J, Zhang T, Liu X, Zhang S, Xie P, Fan D, Ji X, Wong KL, Wang L; China Stroke Study Collaboration. Stroke in China: advances and challenges in epidemiology, prevention, and management. Lancet Neurol. 2019;18:394-405. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 645] [Cited by in F6Publishing: 830] [Article Influence: 207.5] [Reference Citation Analysis (0)] |
2. | Khandelwal P, Yavagal DR, Sacco RL. Acute Ischemic Stroke Intervention. J Am Coll Cardiol. 2016;67:2631-2644. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 82] [Cited by in F6Publishing: 98] [Article Influence: 14.0] [Reference Citation Analysis (0)] |
3. | Ospel JM, Holodinsky JK, Goyal M. Management of Acute Ischemic Stroke Due to Large-Vessel Occlusion: JACC Focus Seminar. J Am Coll Cardiol. 2020;75:1832-1843. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 17] [Cited by in F6Publishing: 14] [Article Influence: 3.5] [Reference Citation Analysis (0)] |
4. | National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group. Tissue plasminogen activator for acute ischemic stroke. N Engl J Med. 1995;333:1581-1587. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 8192] [Cited by in F6Publishing: 7823] [Article Influence: 269.8] [Reference Citation Analysis (0)] |
5. | Goyal M, Menon BK, van Zwam WH, Dippel DW, Mitchell PJ, Demchuk AM, Dávalos A, Majoie CB, van der Lugt A, de Miquel MA, Donnan GA, Roos YB, Bonafe A, Jahan R, Diener HC, van den Berg LA, Levy EI, Berkhemer OA, Pereira VM, Rempel J, Millán M, Davis SM, Roy D, Thornton J, Román LS, Ribó M, Beumer D, Stouch B, Brown S, Campbell BC, van Oostenbrugge RJ, Saver JL, Hill MD, Jovin TG; HERMES collaborators. Endovascular thrombectomy after large-vessel ischaemic stroke: a meta-analysis of individual patient data from five randomised trials. Lancet. 2016;387:1723-1731. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 4230] [Cited by in F6Publishing: 4771] [Article Influence: 596.4] [Reference Citation Analysis (0)] |
6. | Nogueira RG, Jadhav AP, Haussen DC, Bonafe A, Budzik RF, Bhuva P, Yavagal DR, Ribo M, Cognard C, Hanel RA, Sila CA, Hassan AE, Millan M, Levy EI, Mitchell P, Chen M, English JD, Shah QA, Silver FL, Pereira VM, Mehta BP, Baxter BW, Abraham MG, Cardona P, Veznedaroglu E, Hellinger FR, Feng L, Kirmani JF, Lopes DK, Jankowitz BT, Frankel MR, Costalat V, Vora NA, Yoo AJ, Malik AM, Furlan AJ, Rubiera M, Aghaebrahim A, Olivot JM, Tekle WG, Shields R, Graves T, Lewis RJ, Smith WS, Liebeskind DS, Saver JL, Jovin TG; DAWN Trial Investigators. Thrombectomy 6 to 24 Hours after Stroke with a Mismatch between Deficit and Infarct. N Engl J Med. 2018;378:11-21. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 3099] [Cited by in F6Publishing: 3403] [Article Influence: 567.2] [Reference Citation Analysis (0)] |
7. | Albers GW, Marks MP, Kemp S, Christensen S, Tsai JP, Ortega-Gutierrez S, McTaggart RA, Torbey MT, Kim-Tenser M, Leslie-Mazwi T, Sarraj A, Kasner SE, Ansari SA, Yeatts SD, Hamilton S, Mlynash M, Heit JJ, Zaharchuk G, Kim S, Carrozzella J, Palesch YY, Demchuk AM, Bammer R, Lavori PW, Broderick JP, Lansberg MG; DEFUSE 3 Investigators. Thrombectomy for Stroke at 6 to 16 Hours with Selection by Perfusion Imaging. N Engl J Med. 2018;378:708-718. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 2796] [Cited by in F6Publishing: 3050] [Article Influence: 508.3] [Reference Citation Analysis (0)] |
8. | Warner JJ, Harrington RA, Sacco RL, Elkind MSV. Guidelines for the Early Management of Patients With Acute Ischemic Stroke: 2019 Update to the 2018 Guidelines for the Early Management of Acute Ischemic Stroke. Stroke. 2019;50:3331-3332. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 73] [Cited by in F6Publishing: 119] [Article Influence: 23.8] [Reference Citation Analysis (0)] |
9. | Chinese Stroke Society, Neurointerventional Branch of Chinese Stroke Society; Interventional Group of Stroke Prevention and Control Professional Committee of Chinese Preventive Medicine Association. [Chinese guidelines for the endovascular treatment of acute ischemic stroke 2018]. Zhongguo Zuzhong Zazhi. 2018;13:706-729. [DOI] [Cited in This Article: ] |
10. | Jiang X, Li WJ, Wei M, Lv FJ. [Comparison of Volume Computed Tomographic Digital Subtraction Angiography and Artificial Intelligence Technology in the Subtraction Effect of the Head and Neck CTA]. Zhongguo Yiliao Shebei. 2020;35:113-116. [DOI] [Cited in This Article: ] |
11. | Huang XY, Bao YF, Li XM, Guo FK, Li ZF, Shan CH, Chen YM. [Application of artificial intelligence in vascular reconstruction based on cerebral CT perfusion data]. Zhonghua Fangshexue Zazhi. 2021;55:817-822. [DOI] [Cited in This Article: ] |
12. | Medical Administration and Hospital Authority. Chinese Guidelines for Stroke Prevention and Treatment (2021 version). Aug 31, 2021. [cited 3 August 2024]. Available from: http://www.nhc.gov.cn/yzygj/s3593/202108/50c4071a86df4bfd9666e9ac2aaac605.shtml. [Cited in This Article: ] |
13. | Straka M, Albers GW, Bammer R. Real-time diffusion-perfusion mismatch analysis in acute stroke. J Magn Reson Imaging. 2010;32:1024-1037. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 300] [Cited by in F6Publishing: 316] [Article Influence: 24.3] [Reference Citation Analysis (0)] |
14. | Mangla R, Ekhom S, Jahromi BS, Almast J, Mangla M, Westesson PL. CT perfusion in acute stroke: know the mimics, potential pitfalls, artifacts, and technical errors. Emerg Radiol. 2014;21:49-65. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 30] [Cited by in F6Publishing: 30] [Article Influence: 2.7] [Reference Citation Analysis (0)] |
15. | Bivard A, Kleinig T, Miteff F, Butcher K, Lin L, Levi C, Parsons M. Ischemic core thresholds change with time to reperfusion: A case control study. Ann Neurol. 2017;82:995-1003. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 62] [Cited by in F6Publishing: 72] [Article Influence: 12.0] [Reference Citation Analysis (0)] |
16. | Prabhakar B, Singh RK, Yadav KS. Artificial intelligence (AI) impacting diagnosis of glaucoma and understanding the regulatory aspects of AI-based software as medical device. Comput Med Imaging Graph. 2021;87:101818. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 10] [Cited by in F6Publishing: 14] [Article Influence: 4.7] [Reference Citation Analysis (0)] |
17. | Kasasbeh AS, Christensen S, Straka M, Mishra N, Mlynash M, Bammer R, Albers GW, Lansberg MG. Optimal Computed Tomographic Perfusion Scan Duration for Assessment of Acute Stroke Lesion Volumes. Stroke. 2016;47:2966-2971. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 18] [Cited by in F6Publishing: 19] [Article Influence: 2.4] [Reference Citation Analysis (0)] |