Comparison of ChatGPT-3.5 and GPT-4 as potential tools in artificial intelligence-assisted clinical practice in renal and liver transplantation

Table 3 Comparative performance of ChatGPT and GPT-4 in case reports on renal transplantation, detailing agreement levels by task type

Ref.	Question number	Task	Performance, ChatGPT/GPT-4	Physicians course of action/ground truth	Agreement status, ChatGPT/GPT-4
Alharbi et al[21]	1	Provide a list of suitable antibiotics for pseudomonas aeruginosa urinary tract infection.	Provided a list of suitable antibiotics including the one used by physicians (meropenem)/provided a list of suitable antibiotics including the one used by physicians (meropenem)	Meropenem was administrated	A/A
	2	Suggest the next diagnostic test(s) needed for the patient	Suggested a renal ultrasound and a stool culture/suggested a renal ultrasound, abdominal CT, blood cultures, and a stool culture	Abdominal ultrasound and abdominal CT scan were conducted	PA/A
	3	Identify the most probable diagnosis for the patient	Renal allograft malignancy/renal allograft malignancy	Eosinophilic chromophobe renal cell carcinoma was confirmed by the histopathological examination of the graft	A/A
Rubin et al[22]	4	Provide a DD for the patient	Provided a DD that included the final diagnosis/provided a DD that included the final diagnosis	CMV viremia	A/A
	5	Provide the most likely diagnosis for the patient	Post-influenza bacterial pneumonia/CMV reactivation	CMV viremia was demonstrated by antigenemia and PCR assay	D/A
	6	Suggest treatment for the patient	Suggested ganciclovir, valganciclovir, foscarnet, and cidofovir (most preferable ganciclovir or valganciclovir)/suggested ganciclovir, valganciclovir, foscarnet, and cidofovir (most preferable ganciclovir or valganciclovir)	Intravenous ganciclovir followed by oral valganciclovir at a dose of 900 mg/day was administered	A/A
Molina-Andújar et al[23]	7	Provide a DD for the patient	Provided a DD that included the final diagnosis/ Provided a DD that included the final diagnosis	Acute post-infectious glomerulonephritis	A/A
	8	Provide the most likely diagnosis for the patient	Acute post-infectious glomerulonephritis/acute post-infectious glomerulonephritis	Acute post-infectious glomerulonephritis	A/A
Baker et al[24]	9	Provide the next step patient’s management	Suggested hemodynamic stabilization with transfusion of blood products and bleeding control including surgical intervention, if necessary, followed by continuous monitoring/Suggested hemodynamic stabilization with transfusion of blood products and surgical exploration if bleeding if suspected to be within the surgical site. Suggested medication reevaluation focus on anticoagulants, prophylactic treatment for infection prevention and continuous monitoring.	The patient was taken back to theatre for exploration where ligation of the bleeding artery, removal of blood clots and blood transfusion took place. Postoperative monitoring was performed	A/A
	10	Suggest the next diagnostic test needed for the patient	Suggested an abdominal CT scan or an ultrasound/suggested imaging such as abdominal CT with contrast, an ultrasound or an angiogram is performed. Suggested evaluating the patient with new laboratory tests and for the need of re-exploration	An urgent CT angiogram was performed	PA/A
	11	Provide a DD for the bleeding	Provided a DD that included the final diagnosis/provided a DD that included the final diagnosis	Bleeding from a small branch of the renal artery	A/A
	12	Provide the most likely diagnosis for the patient	Failure or dislodgement of a surgical clip: Bleeding from a small branch of the renal artery where a surgical clip had come off during the re-exploration surgery/Failure or dislodgement of a surgical clip: Bleeding from a small branch of the renal artery where a surgical clip had come off during the re-exploration surgery	Bleeding was noticed from a small branch of the renal artery	A/A
Gewehr et al[25]	13	Provide a DD for the patient	Provided a DD that included the final diagnosis/provided a DD that included the final diagnosis	Fungal infection	A/A
	14	Provide the most likely diagnosis for the patient	Fungal Infection/fungal Infection, and specifically sporotrichosis	Fungal Infection (sporotrichosis)	A/A
Vassallo et al[26]	15	Provide a DD for the patient	Provided a DD that included the final diagnosis/provided a DD that included the final diagnosis	Active hepatitis E virus infection	A/A
	16	Provide the most likely diagnosis for the patient	NAFLD/NAFLD or drug induced	Active hepatitis E virus infection	D/D
	17	Suggest the next diagnostic test needed for the patient	Suggested liver biopsy along with further imaging and laboratory investigations/suggested liver biopsy along with further imaging and laboratory investigations	Liver biopsy	A/A
	18	Suggest the next diagnostic test needed for the patient after the biopsy results	Suggested extensive viral serologic tests, PCR for suspected viruses, immunostaining of liver biopsy, and continuous monitoring of liver function/suggested extensive viral serologic tests, PCR for suspected viruses, immunostaining of liver biopsy, and continuous monitoring of liver function	A more extensive viral screen was conducted	A/A
Olsen et al[27]	19	Provide a DD for the patient	Provided a DD that included the final diagnosis/provided a DD that included the final diagnosis	Epstein-Barr virus-negative, diffuse, large B-cell lymphoma	A/A
	20	Provide the most likely diagnosis for the patient	Suggested that infectious aetiologies such as disseminated tuberculosis or fungal infections are more likely. It implied that diagnosis is difficult without further diagnostic investigations/determined as PTLD as the most likely diagnosis followed by infectious aetiologies	Epstein-Barr virus-negative, diffuse, large B cell lymphoma	D/A
	21	Suggest the next diagnostic test needed for the patient	Suggested sputum and/or BAL cultures, Mantoux test or IGRA, Blood tests, further imaging and laboratory tests, and lung biopsy/suggested liver biopsy, sputum and/or BAL cultures, Mantoux test or IGRA, Blood tests, further imaging and laboratory tests, and lung biopsy	Biopsy from one of the liver lesions	D/A
Allam et al[28]	22	Suggest the next diagnostic test needed for the patient	Suggested a kidney biopsy/suggested a kidney biopsy and further laboratory tests	Transplant biopsy was performed	A/A
	23	Provide a DD for the patient	Provided a DD that did not include the final diagnosis/provided a DD including vascular complications such as vein stenosis	Biopsy-induced arteriovenous fistula and venous stenosis	D/PA
	24	Suggest treatment for the patient	Suggested intervention to address the arteriovenous fistula and stenosis of the main renal vein (embolization, angioplasty, stenting)/suggested intervention to address the arteriovenous fistula and stenosis of the main renal vein (embolization, angioplasty, stenting)	Embolization of fistula (coil occlusion)	A/A
Subramanian et al[29]	25	Provide a DD for the patient	Provided a DD that did not include the final diagnosis/provided a DD that included the final diagnosis	A small basal ganglia infarct and an infarct of the spinal cord was found	D/A
	26	Provide the most likely diagnosis for the patient	Suggested ischemic injury or infarction of the spinal cord/suggested spinal cord ischemia or infarction	A small basal ganglia infarct and an infarct of the spinal cord was found	A/A
	27	Suggest the next diagnostic test needed for the patient	Suggested spine MRI, NCS and EMG to assess peripheral nerves and muscles, lumbar puncture if infections suspected, and transplant biopsy if rejection or ischemia is suspected/suggested spine MRI-MRA, neurond physiological studies (SSEP, NCS and EMG), lumbar puncture if infections suspected	A CTAP, and spine/brain MRI were performed	PA/PA
Ainsworth et al[30]	28	Provide a DD for the patient	Provided a DD that included immune-mediated hemolysis but did not specifically include PLS/provided a DD that included the final diagnosis	PLS	PA/A
	29	Provide the most likely diagnosis for the patient	Suggested hemolysis due to mismatched blood type of the donor/suggested PLS	PLS	D/A

CT: Computed tomography; DD: Differential diagnosis; A: Agreement; BAL: Bronchoalveolar lavage; D: Disagreement; CMV: Cytomegalovirus; CTAP: Computed tomography arterial portography; EMG: Electromyography; IGRA: Interferon-gamma release assay; MRA: Magnetic resonance angiography; MRI: Magnetic resonance imaging; NAFLD: Non-alcoholic fatty liver disease; NCS: Nerve conduction studies; PA: Partial agreement; PCR: Polymerase chain reaction; PLS: Passenger lymphocyte syndrome; PTLD: Post-transplant lymphoproliferative disorder; SSEP: Somatosensory evoked potentials.