Comparison of ChatGPT-3.5 and GPT-4 as potential tools in artificial intelligence-assisted clinical practice in renal and liver transplantation

Table 4 Comparative performance of ChatGPT and GPT-4 in case reports on liver transplantation, detailing agreement levels by task type

Ref.	Question number	Task	Performance, ChatGPT/GPT-4	Physicians course of action/ground truth	Agreement status, ChatGPT/GPT-4
Rubin et al[22]	1	Case presentation/provide a DD for the patient	Provided a DD that included the final diagnosis/provided a DD that included the final diagnosis	CMV	A/A
	2	Provide the most likely diagnosis for the patient	Suggested post-transplant infection, particularly a viral infection (CMV, EBV, or VZV)/CMV	CMV	PA/A
	3	Justify the recurrence of CMV infections despite treatment	Suggested resistance to ganciclovir/suggested resistance to ganciclovir or/and inadequate duration of initial treatment-secondary infections	Ganciclovir resistant infection	A/A
	4	Suggest alternative treatment for the patient	Suggested foscarnet/suggested foscarnet or cidofovir or letermovir or/and CMV immunoglobulins	Foscarnet was administered	A/A
Okeke et al[31]	5	Case presentation/suggest treatment for the patient given no arterial flow in the liver	Suggested interventional radiology procedures or/and surgical revascularization/suggested interventional radiology procedures or/and surgical revascularization (thrombectomy or re-anastomosis)	Interventional radiology procedure (thrombolysis) was performed. Then revascularization was achieved intraoperatively (infra-aortic jump was performed)	PA/PA
	6	Suggest the diagnostic tests needed for the patient following re-thrombosis	Suggested doppler ultrasound, CT angiogram, coagulation profile-thrombophilia testing/suggested thrombophilia workup, repeat imaging (doppler ultrasound, CT/MRI angiography), and autoimmune markers	Hypercoagulable workup was performed	A/A
	7	Provide a DD behind re-thrombosis	Provided a DD that included the final diagnosis/provided a DD that included the final diagnosis	Antiphospholipid syndrome	A/A
	8	Provide the most likely diagnosis for the patient	Suggested hepatic artery thrombosis/suggested antiphospholipid syndrome	Antiphospholipid syndrome	D/A
Eubank et al[32]	9	Case presentation/determine the most likely microorganism to be identified by the swab	Suggested Staphylococcus aureus, Streptococcus species, Enterococcus species, and Pseudomonas aeruginosa, and fungi like Candida albicans/suggested Staphylococcus aureus, Enterococcus species, Pseudomonas aeruginosa, Escherichia coli, fungi like Candida or Aspergillus, viruses like CMV, and mycobacteria	94% Enterococcus faecalis, 93% Rhizopus oryzae, and 5% Aspergillus flavus	D/PA
	10	Suggest treatment for the patient given the pathogens identified	Suggested intravenous liposomal amphotericin B at an appropriate dosage, along with surgical debridement of infected tissue/suggested intravenous liposomal amphotericin B at an appropriate dosage, oral posaconazole along with surgical debridement of infected tissue.	Oral posaconazole 300 mg and IV amphotericin B and micafungin daily. Amphotericin B deoxycholate irrigation in the wound vacuum	PA/A
Kim et al[33]	11	Case presentation/provide a DD for the patient’s shock	Provided a DD that did not include the final diagnosis/provided a DD that included the final diagnosis	GVHD	D/A
	12	Provide the most likely diagnosis for the patient	Suggested a surgical complication, specifically duodenal perforation/suggested duodenal perforation or drug-induced kidney injury/neutropenia	GVHD	D/D
	13	Suggest the further diagnostic tests needed for the patient	Suggested blood cultures, peritoneal fluid analysis, endoscopy or upper GI imaging/suggested blood and urine cultures, viral and fungal tests, peritoneal fluid analysis, laboratory tests, and endoscopy or upper GI imaging	Mixed chimerism studies and skin biopsy were performed	D/D
	14	Suggest further treatment for the patient given the mixed chimerism studies results	The following treatment options were suggested: Systemic corticosteroids, adjusting tacrolimus dose, consider additional immunosuppressives such as mycophenolate, and phototherapy/suggested considering the following treatment options: High-dose corticosteroids, ATG, ECP, infliximab, ruxolitinib, MSC transplantation, additional immunosuppressive agents, and IL-2 diphtheria toxin	Steroids were administrated for 4 days followed by ruxolitinib due to patient not responding to treatment	PA/A
	15	Guess the survival of the patient	Suggested that the patient did not, most likely, survive/suggested that the patient did not, most likely, survive	The patient died on day 16 of re-admission, 45 days following transplantation	A/A
Kim et al[33], (b)	16	Case presentation/provide a DD for the patient	Provided a DD that did not include the final diagnosis/provided a DD that included the final diagnosis	GVHD	D/A
	17	Provide the most likely diagnosis for the patient	Suggested Clostridioides difficile colitis/suggested GVHD	GVHD	D/A
	18	Suggest treatment for the patient	The following treatment options were suggested: Glucocorticoids, CNIs, ATG, T-cell depleting agents such as basiliximab/high-dose corticosteroids, adjust immunosuppression, consider second line treatments such as ATG, ECP, sirolimus, infliximab, and basiliximab	Steroids were administrated for 2 days followed by ruxolitinib due to patient not responding to treatment	PA/PA
	19	Guess the survival of the patient	Declined to make a prediction/suggested that the patient did not, most likely, survive	The patient died 29 days after transplant	D/A
Ramírez de la Piscina et al[34]	20	Case presentation/Provide a DD for the patient	Provided a DD that included the final diagnosis/ provided a DD that included the final diagnosis	Budd-Chiari syndrome secondary to ADPKD	A/A
	21	Provide the most likely diagnosis for the patient	Suggested Budd-Chiari syndrome/suggested Budd-Chiari syndrome secondary to the compression from ADPKD cysts	Budd-Chiari syndrome secondary to ADPKD	A/A
	22	Suggest treatment for the patient	Provided a list of suitable treatment options including only liver transplantation/provided a list of suitable treatment options including combined transplantation	A combined liver and renal transplantation was performed	PA/A
Arstikyte et al[35]	23	Case presentation/provide a DD for the patient	Provided a DD that included the final diagnosis/provided a DD that included the final diagnosis	Venous air embolism	A/A
	24	Provide the most likely diagnosis for the patient	Suggested that information given is insufficient to single out a specific diagnosis/suggested that based on given information hemorrhage or venous air embolism are the two most likely diagnoses	Venous air embolism	D/A
	25	Suggest appropriate diagnostic test for the patient	Suggested TEE/suggested TEE	TEE	A/A
Aucejo et al[36]	26	Case presentation/provide a DD for the patient	Provided a DD that did not include the final diagnosis/provided a DD that did not include the final diagnosis	Narrowing of the RHV at the level of the cava-caval anastomosis	D/D
	27	Provide the most likely diagnosis for the patient	Suggested adhesions, anastomotic leakage, or biliary complications/suggested PVT	Narrowing of the RHV at the level of the cava-caval anastomosis	D/D
	28	Given the RHV stenosis diagnosis, suggest treatment for the patient	Suggested considering stent placement, TIPS or surgical revision/suggested considering stent placement, TIPS or surgical revision	A wall stent 14 mm in diameter by 40 mm in length was placed across the RHV stenosis	A/A
Ichimura et al[37]	29	Case presentation/provide a DD for the patient	Provided a DD that included the final diagnosis/provided a DD that included the final diagnosis	VOD/SOS	A/A
	30	Provide the most likely diagnosis for the patient	Suggested GVHD/suggested VOD/SOS	VOD/SOS	D/A
	31	Suggest treatment for the patient given VOD/SOS	Suggested considering defibrotide, anticoagulant medications, and liver transplantation/suggested considering defibrotide, anticoagulant medications, TIPS, and liver transplantation	The physicians performed a liver transplantation since defibrotide had not yet been approved	A/A
	32	Provide a new differential diagnosis for the patient’s deterioration postoperatively	Provided a DD that did not include the final diagnosis/provided a DD that included the final diagnosis	GVHD, several infections	D/A
Trevizoli et al[38]	33	Case presentation/suggest appropriate treatment for the patient	Suggested considering corticosteroids, aminosalicylates, immunomodulators such as azathioprine, biologic agents such as infliximab, diuretics, variceal bleeding prophylaxis and liver transplant evaluation/suggested considering corticosteroids, aminosalicylates, immunomodulators such as azathioprine, biologic agents such as infliximab, consider surgical management (colectomy), diuretics, variceal bleeding prophylaxis and liver transplant evaluation	Sodium restriction, diuretic therapy, hydrocortisone 300 mg was started without adequate response, vedolizumab	PA/PA
	34	Suggest appropriate treatment for the patient given the DVT progression	Suggested LMWH and IVF/suggested LMWH	He underwent hemodynamic intervention with the placement of a vena cava filter	A/D

A: Agreement; ADPKD: Autosomal dominant polycystic kidney disease; ATG: Anti-thymocyte globulin; CMV: Cytomegalovirus; CNI: Calcineurin inhibitor; CT: Computed tomography; D: Disagreement; DD: Differential diagnosis; EBV: Epstein-Barr virus; ECP: Extracorporeal photopheresis; GI: Gastrointestinal; GVHD: Graft-versus-host disease; IVF: Inferior vena cava; IL: Interleukin; LMWH: Low-molecular-weight heparin; MRI: Magnetic resonance imaging; MSC: Mesenchymal stem cells; PA: Partial agreement; PVT: Portal vein thrombosis; RHV: Right hepatic vein; SOS: Sinusoidal obstruction syndrome; TEE: Transesophageal echocardiogram; TIPS: Transjugular intrahepatic portosystemic shunt; VOD: Veno-occlusive disease; VZV: Varicella-zoster virus.