Effect of training on resident inter-reader agreement with American College of Radiology Thyroid Imaging Reporting and Data System

Table 5 The relative sensitivity, specificity, positive predictive value, and negative predictive value per Thyroid Imaging Reporting and Data System Level on the pre-training assessment compared to the reference standard

Pre-training, Statistics	TI-RADS 1, %	TI-RADS 2, %	TI-RADS 3, %	TI-RADS 4, %	TI-RADS 5, %
Sensitivity
R1	9.1 (0.2-41.3)	33.3 (7.5-70.1)	44.4 (13.7-78.8)	30.8 (9.1-61.4)	87.5 (47.4-99.7)
R2	9.1 (0.2-41.3)	0 (0-33.6)	11.1 (0.3-48.3)	38.5 (13.9-68.4)	75 (34.9-96.8)
R3	90.9 (58.7-99.8)	33.3 (7.5-70.1)	44.4 (13.7-78.8)	38.5 (13.9-68.4)	37.5 (8.5-75.5)
Pooled	36.4 (20.4-54.9)	22.2 (8.6-42.3)	33.3 (16.5-54)	35.9 (21.2-52.8)	66.7 (44.7-84.4)
Specificity
R1	100 (91.0-100)	90.2 (76.9-97.3)	92.7 (80.1-98.5)	62.2 (44.8-77.5)	76.2 (60.6-88)
R2	100 (91-100)	97.6 (87.1-99.9)	80.5 (65.1-91.2)	81.1 (64.8-92)	50 (34.2-65.8)
R3	66.7 (49.8-80.9)	97.6 (87.1-99.9)	95.1 (83.5-99.4)	89.2 (74.6-97)	90.5 (77.4-97.3)
Pooled	88.9 (81.8-94)	95.1 (89.7-98.2)	89.4 (82.6-94.3)	76.6 (67.6-84.1)	72.2 (63.5-79.8)
Positive predictive value
R1	100	42.9 (16.8-73.6)	57.1 (26.4-83.2)	22.2 (10.3-41.6)	41.2 (27.7-56.1)
R2	100	0	11.1 (1.8-46.8)	41.7 (21.5-65.1)	22.2 (14.8-32.1)
R3	43.5 (32.2-55.5)	75 (26-96.2)	66.7 (30.1-90.3)	55.6 (28.3-79.8)	42.9 (17.1-73.2)
Pooled	48 (31.8-64.6)	50 (25.9-74.1)	40.9 (24.8-59.2)	35 (23.9-48)	31.4 (23.5-40.5)
Negative predictive value
R1	79.6 (76.4-82.5)	86.1 (79.4-90.8)	88.4 (80.8-93.2)	71.9 (62.2-79.9)	97 (83.5-99.5)
R2	79.6 (76.4-82.5)	81.6 (80.9-82.4)	80.5 (75.8-84.5)	79 (70.4-85.6)	91.3 (75.3-97.3)
R3	96.3 (79.8-99.4)	87 (80.7-91.4)	88.6 (81.2-93.4)	80.5 (72.6-86.5)	88.4 (81.5-92.9)
Pooled	83.2 (79.2-86.6)	84.8 (81.9-87.3)	85.9 (82.3-88.9)	77.3 (72.5-81.5)	91.9 (86.5-95.3)

TI-RADS: Thyroid Imaging Reporting and Data System; RS: Reference standard; R1: Reader 1; R2: Reader 2; R3: Reader 3.