Comparison of ChatGPT-3.5 and GPT-4 as potential tools in artificial intelligence-assisted clinical practice in renal and liver transplantation

Table 5 Comparative performance of ChatGPT and GPT-4 in department cases on renal transplantation, detailing agreement levels by task type

Case ID	Question number	Task	Performance, ChatGPT/GPT-4	Physicians course of action/ground truth	Agreement status, ChatGPT/GPT-4
1	1	Case presentation/provide the diagnostic tests needed to investigate refractory ascites in patient with ADPKD	Suggested abdominal ultrasound, paracentesis with fluid analysis, LF tests, tumor marker tests, CT scan, serologic testing, genetic testing/ suggested paracentesis with fluid analysis, LF tests, abdominal ultrasound, CT scan, echocardiogram, and endoscopy, further evaluation for elevated markers	Paracentesis (ascites fluid was send for cytology, culture, TB investigation, SAAG calculation), abdominal CT, liver ultrasound, LF tests, tumor marker tests, serologic testing, echocardiogram, and endoscopy	PA/A
	2	Provide a DD for the patient	Provided a DD that included the final diagnosis/provided a DD that included the final diagnosis	Tuberculous peritonitis	A/A
	3	Provide the most likely diagnosis for the patient	Suggested malignancy (most likely ovarian cancer) or SBP are the most likely diagnoses/suggested tuberculous peritonitis or malignancy or SBP as the most likely diagnoses	Tuberculous peritonitis	D/A
2	4	Case presentation/provide a differential diagnosis for the patient	Provided a DD that included the final diagnosis/provided a DD that included the final diagnosis	Acute PE	A/A
	5	Provide the most probable diagnosis for the patient	Suggested myocardial infraction as the most probable diagnosis/suggested PE as the most probable diagonal	Acute PE	D/A
	6	What diagnostic test is more suitable for this patient	Suggested CTPA and ECG be performed/suggested CTPA, ECG, and d-dimers tests be performed	CTPA was performed	A/A
	7	What treatment do you recommend for this patient, given PE is confirmed	Suggested a choice among LMWH, DOACs, and warfarin. No discrimination between short and long-term anticoagulation was made. Suggested initial anticoagulation with either LMWH or DOACs including apixaban followed by a long-term anticoagulation with either a DOAC or warfarin	10 mg apixaban BD was commenced followed by 5 mg BD for 6 months	PA/A
3	8	Case presentation/provide a DD given the post-operative signs/symptoms of the patient	Provided a DD that included the final diagnosis/provided a DD that included the final diagnosis	Post-operative bleeding	A/A
	9	Provide the most probable diagnosis	Suggested exacerbation or progression of her underlying thrombocytopenic disorder/suggested post-transplant acute thrombotic microangiopathy	Post-operative bleeding	D/D
	10	Predict the next diagnostic test that the patient requires	Suggested coagulation studies, renal function test, peripheral blood smear, infectious testing and imaging including ultrasound and CT/suggested peripheral blood smear, LDH level, Coombs test, renal function, immunosuppressive level tests, and infection screening.	Abdominal ultrasound and abdomen/pelvis CT with contrast	PA/D
	11	Appropriate treatment given the evidence of active bleeding	Suggested stabilization with intravenous fluids and blood products, surgical intervention, and close monitoring/suggested stabilization with intravenous fluids and blood products, surgical intervention, and close monitoring	The patient was transfused and was re-explored	A/A
4	12	Case presentation/provide a DD for the patient	Provided a DD that included the final diagnosis/provided a DD that included the final diagnosis	Acute graft thrombosis due to renal vein thrombosis	A/A
	13	Provide the most probable diagnosis	Suggested acute graft thrombosis due to either renal artery or vein thrombosis/suggested acute graft thrombosis due to renal vein thrombosis	Acute graft thrombosis due to renal vein thrombosis	A/A
	14	Provide the most suitable diagnostic test	Suggested choosing among transplant duplex US, CT angiography, and renal scintigraphy/suggested choosing among transplant duplex US, CT angiography, and renal scintigraphy	Transplant doppler US	A/A
	15	Given the transplant US findings, provide the patient’s diagnosis	Acute renal allograft rejection/acute renal artery thrombosis or artery stenosis	Renal vein thrombosis	D/D
	16	Given the transplant US findings, suggest a diagnostic modality that could verify diagnosis	Renal biopsy/suggested CT angiography	CT angiography was performed	D/A
	17	Suggest treatment options for the patient	Suggested considering high-dose corticosteroids, antithymocyte globulin, calcineurin inhibitors, mycophenolate mofetil, basiliximab or alemtuzumab, and plasmapheresis with intravenous immunoglobulin/suggested surgical revascularization	Patient was re-explored	D/A
	18	Findings of reperfusion during benchwork after explanation	Suggested inadequate restoration of tissue perfusion and significant vascular compromise and tissue damage/suggested extensive, vascular thrombosis with poor kidney perfusion, and evidence of parenchymal damage	Artery perfusion required high pressure, kidney became turgid, swollen, and a capsular tear was seen	A/A
5	19	Case presentation/provide DD for the patient	Provided a DD that included the final diagnosis/provided a DD that included the final diagnosis	Post-transplant obstructive LUTS due to clot retention	A/A
	20	Provide most probable diagnosis	Suggested urinary tract obstruction most probably at the side of the anastomosis/suggested urinary tract obstruction due to blood clot formation as the most probable diagnosis	Post-transplant obstructive LUTS due clot retention	PA/A
	21	Suggest next diagnosis test to verify the diagnosis	Suggested considering transplant US, abdominal CT or renal scintigraphy/suggested transplant US as the first-line image modality. Suggested that other option include abdominal CT, MRI, and nuclear medicine scans	A transplant US was performed	A/A
	22	Given findings of US/suggest a suitable treatment option for the patient	Suggested considering manual irrigation, catheter flushing, cystoscopic clot evaluation, and monitoring/suggested replacing the foley catheter to flush out smaller clots, cystoscopic clot evaluation, consider percutaneous nephrostomy, and monitoring	A 3-way irrigation system was applied	PA/PA
	23	Despite resolved hematuria patient’s clearance did not improved/provide a DD	Provided a DD that included the final diagnosis/provided a DD that included the final diagnosis	Acute kidney injury with acute tubular necrosis	A/A
	24	Provide most probable diagnosis	Suggested acute kidney injury with acute tubular necrosis as the most probable diagnosis/suggested acute kidney injury with acute tubular necrosis as the most probable diagnosis	Acute kidney injury with acute tubular necrosis	A/A
	25	Case progression update/poor renal function 3 months post-operatively provide DD for patient’s signs and symptoms	Provided a DD that included the final diagnosis/provided a DD that included the final diagnosis	Recurrence of underlying disease	A/A
	26	Provide most probable diagnosis	Suggested chronic allograft dysfunction as the most probable diagnosis/suggested chronic allograft dysfunction and recurrence of the underlying disease as the two most probable diagnoses	Recurrence of underlying disease	D/PA

A: agreement; ADPKD: Autosomal dominant polycystic kidney disease; BD: Twice a day; CT: Computed tomography; CTPA: Computed tomography pulmonary angiogram; D: Disagreement; DD: Differential diagnosis; DOAC: Direct oral anticoagulants; ECG: Electrocardiogram; LDH: Lactate dehydrogenase; LF: Liver function; LMWH: Low molecular weight heparin; LUTS: Lower urinary tract symptoms; MRI: Magnetic resonance imaging; PA: Partial agreement; PE: Pulmonary embolism; SAAG: Serum ascites albumin gradient; SBP: Spontaneous bacterial peritonitis; TB: Tuberculosis; US: Ultrasound.