Accuracy of artificial intelligence in meta-analysis: A comparative study of ChatGPT 4.0 and traditional methods in data synthesis

Table 1 Comparison of effect measures and P values between original studies and ChatGPT

Ref.	Journal	Software used for analysis	Outcome measures	Type of effect measure	Effect measure (original)	P value (original)	Effect measure (ChatGPT)	P value (ChatGPT)
Sreenivasan et al[10]	Journal of American Heart Association	R studio	MACE	Risk ratio (95%CI)	0.67 (0.55 -0.82)	< 0.001	0.67 (0.56-0.81)	< 0.001
			Cardiac death	Risk ratio (95%CI)	0.49 (0.34 -0.71)	< 0.001	0.46 (0.29-0.72)	< 0.001
			All-cause death	Risk ratio (95%CI)	0.81 (0.61 -1.07)	0.135	0.80 (0.60-1.06)	0.124
			Myocardial infarction	Risk ratio (95%CI)	0.82 (0.62-1.07)	0.144	0.84 (0.64-1.10)	0.197
			Target vessel MI	Risk ratio (95%CI)	0.61 (0.42-0.89)	0.011	0.61 (0.42-0.90)	0.011
			Target lesion revascularization	Risk ratio (95%CI)	0.67 (0.49-0.91)	0.011	0.66 (0.48-0.91)	0.010
			Target vessel revascularization	Risk ratio (95%CI)	0.60 (0.45-0.80)	< 0.001	0.60 (0.45-0.80)	< 0.001
			Stent thrombosis	Risk ratio (95%CI)	0.63 (0.40-0.99)	0.046	0.67 (0.42-1.06)	0.090
Patel et al[11]	Circulation	R studio	MACE	Hazard ratio (95%CI)	0.91 (0.87-0.96)	< 0.0001	0.94 (0.90-0.98)	0.005
			Cardiovascular death	Hazard ratio (95%CI)	0.86 (0.81-0.92)	< 0.0001	0.88 (0.81-0.96)	0.0037
			Myocardial infarction	Hazard ratio (95%CI)	0.95 (0.87-1.04)	0.29	0.97 (0.90-1.05)	0.429
			Stroke	Hazard ratio (95%CI)	0.99 (0.91-1.07)	0.77	0.87 (0.65-1.18)	0.380
Sacoransky et al[12]	American Journal of Cardiology	RevMan	LV thrombus formation	Odds ratio (95%CI)	0.28 (0.11-0.73)	< 0.01	0.37 (0.16-0.86)	0.021
			Major bleeding events	Odds ratio (95%CI)	2.85 (1.13-7.24)	0.03	2.82 (1.30-6.13)	0.009
			Systemic embolism	Odds ratio (95%CI)	0.37 (0.12-1.13)	0.08	0.42 (0.13-1.29)	0.130
Paolucci et al[13]	European Heart Journal	RevMan	Relationship between PMI (UDMI and UDMI-like definitions) and mortality	Hazard ratio (95%CI)	1.61(1.32- 1.97)	< 0.00001	1.61 (1.32-1.97)	< 0.00001
			Relationship between PMI (ARC-2 definition) and mortality	Hazard ratio (95%CI)	2.07 (1.40-3.08)	0.0003	2.07 (1.40-3.08)	0.0003
			Relationship between PMI (SCAI and SCAI-like definitions) and mortality	Hazard ratio (95%CI)	3.24 (2.36-4.44)	< 0.00001	3.24 (2.36-4.44)	< 0.00001
Ang et al[14]	International Journal of Surgery	Stata	Major vascular complications	Risk ratio (95%CI)	2.32 (1.73-3.11)	< 0.001	2.30 (1.72-3.09)	< 0.001
			Major bleeding	Risk ratio (95%CI)	1.61 (1.27-2.05)	< 0.001	1.61 (1.27-2.05)	< 0.001
			Aortic annulus rupture	Risk ratio (95%CI)	4.66 (1.67-13.01)	< 0.001	4.66 (1.67-13.01)	0.0033
			Cardiac tamponade	Risk ratio (95%CI)	3.0 (1.31-6.89)	0.01	3.0 (1.31-6.89)	0.0095
			Minor vascular complications	Risk ratio (95%CI)	1.43 (1.00-2.04)	0.05	1.43 (1.00-2.04)	0.052
			Stroke	Risk ratio (95%CI)	1.37 (0.81-2.32)	0.24	1.41 (0.82-2.40)	0.210
			In-hospital mortality	Risk ratio (95%CI)	1.86 (0.74-4.70)	0.19	1.86 (0.74-4.70)	0.187
			30 days mortality	Risk ratio (95%CI)	1.14 (0.532.46)	0.74	1.22 (0.55-2.71)	0.628
			Pacemaker implantation	Risk ratio (95%CI)	0.91 (0.70-1.18)	0.47	0.98 (0.75-1.26)	0.848

MACE: Major adverse cardiac events; MI: Myocardial infarction; LV: Left ventricular.