Development and validation of a risk assessment model for prediabetes in China national diabetes survey

Abstract

BACKGROUND

Prediabetes risk assessment models derived from large sample sizes are scarce.

AIM

To establish a robust assessment model for prediabetes and to validate the model in different populations.

METHODS

The China National Diabetes and Metabolic Disorders Study (CNDMDS) collected information from 47325 participants aged at least 20 years across China from 2007 to 2008. The Thyroid Disorders, Iodine Status and Diabetes Epidemiological Survey (TIDE) study collected data from 66108 participants aged at least 18 years across China from 2015 to 2017. A logistic model with stepwise selection was performed to identify significant risk factors for prediabetes and was internally validated by bootstrapping in the CNDMDS. External validations were performed in diverse populations, including populations of Hispanic (Mexican American, other Hispanic) and non-Hispanic (White, Black and Asian) participants in the National Health and Nutrition Examination Survey (NHANES) in the United States and 66108 participants in the TIDE study in China. C statistics and calibration plots were adopted to evaluate the model’s discrimination and calibration performance.

RESULTS

A set of easily measured indicators (age, education, family history of diabetes, waist circumference, body mass index, and systolic blood pressure) were selected as significant risk factors. A risk assessment model was established for prediabetes with a C statistic of 0.6998 (95%CI: 0.6933 to 0.7063) and a calibration slope of 1.0002. When externally validated in the NHANES and TIDE studies, the model showed increased C statistics in Mexican American, other Hispanic, Non-Hispanic Black, Asian and Chinese populations but a slightly decreased C statistic in non-Hispanic White individuals. Applying the risk assessment model to the TIDE population, we obtained a C statistic of 0.7308 (95%CI: 0.7260 to 0.7357) and a calibration slope of 1.1137. A risk score was derived to assess prediabetes. Individuals with scores ≥ 7 points were at high risk of prediabetes, with a sensitivity of 60.19% and specificity of 67.59%.

CONCLUSION

An easy-to-use assessment model for prediabetes was established and was internally and externally validated in different populations. The model had a satisfactory performance and could screen individuals with a high risk of prediabetes.

Key Words: Hyperglycemia, Prediabetes, Risk assessment model, Risk scores

Core Tip: This was the first study to utilize easily-measured metrics to develop prediabetes assessment model in a large population and validated the model in different populations. Data of the China National Diabetes and Metabolic Disorders Study survey with 47325 participants was used to establish the risk assessment model for prediabetes. External validation was performed in a broad spectrum of populations that have marked racial and demographical differences, and the satisfactory discrimination and calibration performance enhance the model’s generalizability across nations. Risk score was derived to assess prediabetes. Stratified individuals at ≥ 7 points were at high risk of prediabetes, with sensitivity of 60.19% and specificity of 67.59%.

INTRODUCTION

Type 2 diabetes is a burdensome chronic disease. It is associated with a high prevalence and high morbidity. In China, the prevalence of type 2 diabetes rose from 5.5% in 2000 to 9.7% in 2008 and 12.8% in 2020[1-4]. Prediabetes, also known as intermediate hyperglycemia, is a high-risk state in which glycemic levels are elevated above the healthy level but are lower than the threshold of diabetes. Prediabetes is also speculated to be a preceding phase in the pathological development of type 2 diabetes[5]. Prediabetes has been reported to increase the risk of mortality and cardiovascular disease in the general population as well as in patients with atherosclerotic cardiovascular disease[6]. Nearly 150 million Chinese adults have prediabetes[2]. If left untreated, prediabetes is likely to progress into diabetes and impose a great burden on individuals and society.

Evidence from the well-known Da Qing study in China has demonstrated that lifestyle interventions can effectively decrease progression to diabetes and delay the onset of diabetes[7,8]. Lifestyle interventions can also prevent cardiovascular or microvascular events in people with impaired glucose tolerance and reduce cardiovascular-related or all-cause mortality during the 30-year follow-up[8]. Thus, early lifestyle changes and prompt intervention are important for effective diabetes prevention. Given the preceding phase of prediabetes in diabetes progression and its modifiable nature through effective early intervention, screening high-risk individuals for prediabetes has significant implications for diabetes prevention and health promotion.

Nevertheless, existing risk scores and screening instruments for hyperglycemia, e.g., the Finnish Diabetes Risk Score[9], the diabetes score developed from data of the National Health and Nutrition Examination Survey[10], the Qingdao Diabetes Risk Score[11] and the New Chinese Diabetes Risk Score[12], are mainly focused on diabetes. Most of these studies are based on self-assessments from patients to make risk assessments for diabetes, while risk algorithms specific to prediabetes are still rare. Prior studies on prediabetes assessment are restricted to studies with small sample sizes or low sensitivity or specificity[13,14]. A prediabetes risk score was developed in a sample of 892 Indian subjects with a sensitivity/specificity of 84.37%/58.47%[14]. Another prediabetes risk score was derived from 21,720 Indonesian individuals with a sensitivity of 50.03% and specificity of 65.81%[13]. The Indonesian prediabetes risk score had low sensitivity, while the Indian score did not have satisfactory specificity due to the small number of subjects.

To address the problem of scarcity in the robust assessment model of prediabetes in a large sample, we established a prediabetes risk assessment model based on the China National Diabetes and Metabolic Disorders Study (CNDMDS), which was a population-based survey involving nearly 48000 participants across China from 2007 to 2008[2]. The dataset from the National Health and Nutrition Examination Survey (NHANES) in the United States was used to externally validate the model. NHANES is an ongoing national health survey in the United States. Its aim is to assess health and nutritional status in the general population. Data are released in 2-year cycles (https://www.cdc.gov/nchs/nhanes/index.htm). NHANES started to oversample the Asian population in 2011-2012 and released data for Asian Americans in a single race category, which enabled us to analyze diverse populations for external validation. The model was further externally validated using data from the Thyroid disorders, Iodine status and Diabetes Epidemiological survey (TIDE) study in 2017. The TIDE study is the latest national epidemic survey of diabetes in the Chinese population[4].

MATERIALS AND METHODS

Study population and design

A prediabetes assessment model was developed and internally validated in the CNDMDS. Information on the CNDMDS design has been reported previously[2]. Briefly speaking, the CNDMDS was a cross-sectional population survey conducted from 2007-2008 in China. A total of 47325 participants aged at least 20 years old from 152 urban districts and 112 rural villages across the country completed the survey. External validation was performed in the Asian population in the NHANES. NHANES is an ongoing national health survey to assess health and nutritional status in the general population in America. The NHANES study design and data on glucose and hypoglycemic medication were accessed via the website (https://www.cdc.gov/nchs/nhanes/index.htm). Details of the two datasets are summarized in Supplementary Table 1. External validation was further performed using data from the TIDE study, which is the latest population survey of diabetes in China. The TIDE study was conducted in 31 provinces in mainland China from 2015 to 2017 to investigate the prevalence of thyroid disorders, iodine status and diabetes in Chinese adults. The design of the TIDE study was reported in detail previously[4]. All participants in the CNDMDS and the TIDE study provided informed consent and signed written informed consent.

To derive and validate the prediabetes assessment model, we included adults with available glucose measurements in the surveys and excluded individuals who were previously or newly diagnosed with type 2 diabetes. Diabetes was defined as fasting glucose ≥ 7 mmol/L, 2-h glucose level of oral glucose tolerance test (OGTT) ≥ 1.1 mmol/L, or use of glucose-lowering medications in individuals with a diabetes history.

Outcomes

Prediabetes was defined as a fasting glucose level between 6.10 and 7.00 mmol/L or a 2-h glucose level of the OGTT between 7.80 and 11.10 mmol/L and not meeting the diagnostic criteria of diabetes. A normal blood glucose state was defined as a fasting glucose level lower than 6.10 mmol/L and a 2-h OGTT glucose level lower than 7.80 mmol/L[15]. In the CNDMDS, blood samples after overnight fasting were collected, and a standard OGTT was conducted to evaluate the glucose metabolic state of the participants[2]. In NHANES 2012, fasting plasma glucose and the 2-h glucose level of the OGTT were also measured to assess normal blood glucose state, prediabetes and diabetes mellitus (DM) https://www.cdc.gov/nchs/nhanes/index.htm.

Potential risk factors

To develop the prediabetes assessment model, candidate risk factors were determined on the basis of data availability, ease of measurement, risk scores for diabetes, and risk factors for hyperglycemia in published literature[9,10,12,16-18]. The following variables obtained from the CNDMDS were incorporated for candidate risk factors: demographic information (age, sex, education, residency), DM family history, smoking habit, anthropometric measurements (waist circumference, height, body weight), blood pressure [systolic blood pressure (SBP), diastolic blood pressure (DBP)], and laboratory dyslipidemia biomarkers (cholesterol, triglycerides, high-density lipoprotein cholesterol, low-density lipoprotein cholesterol). A family history of DM was defined as any first-degree relatives, including parents, brothers and sisters, having ever been diagnosed with diabetes. Smoking history was defined as ever smoking at least 100 cigarettes in one’s lifetime. Education (illiteracy, primary school, middle school and university or above) was grouped as a binary variable: Primary or lower education vs middle school or higher level. Waist circumference was categorized into normal or abdominal obesity, which was defined as ≥ 90 cm for men and ≥ 85 cm for women in Chinese people. Abdominal obesity in non-Hispanic Asians was determined using the same criteria as that used for Chinese people. Abdominal obesity was defined as ≥ 102 cm in men and ≥ 88 cm in women for Mexican American, other Hispanic, non-Hispanic White and Black populations given the racial differences in criteria for clinical diagnosis of metabolic syndrome.

For external validation in the NHANES and the TIDE study, we gathered information related to potential risk factors. Moreover, we collected data on demographic characteristics, lifestyle factors, family history of diabetes, anthropometric measurements, blood pressure and glucose levels in the OGTT. A self-reported family history of diabetes in NHANES was considered an affirmative answer to the following question: were any of your close biological relatives, including father, mother, sisters and brothers, ever told by a health professional that they had diabetes? This was consistent with the definition of DM family history in the CNDMDS and the TIDE studies.

Statistical analysis

Data are summarized as numbers (percentages) for categorical variables and means± SDs or medians [interquartile ranges (IQRs)] for continuous variables where appropriate. We adopted T test and Mann‒Whitney’s test for normally distributed and non-normally distributed continuous variables, respectively. We used the chi-square test to analyze categorical variables. Based on univariable analysis of group differences in characteristics and clinical relationships of potential risk factors with prediabetes, the following variables were treated as candidate risk factors: age, sex, middle school or higher education, family history of DM, abdominal obesity, body mass index (BMI), SBP, and DBP. A logistic model with stepwise selection was used to select significant risk factors.

Considering the potential influence of outliers, we analyzed trimmed data in which the highest 0.01% and lowest 0.01% of key continuous variables (waist circumference, BMI, SBP, DBP) were excluded. The predictive ability of the model was evaluated by assessing discrimination and calibration in internal and external validations. Discrimination evaluated the model’s ability to discriminate individuals with prediabetes from those with normal glucose status. The C statistic, also known as the area under the curve (AUC), quantified the discriminative ability. Calibration assessed the agreement between the observed and predicted risk of prediabetes. Slope and intercept in a calibration plot quantified the calibration ability.

To obtain an unbiased estimate of model performance in internal validation, the optimism-corrected C statistic and calibration slope were estimated with 200 bootstrapped resamples of CNDMDS data, accounting for possible overfitting. The bootstrapped resampling sample had the same size as the derivation data. The modeling process was repeated with the same variable selection in each resampling. The apparent performance of the model derived from the bootstrap sample was compared with the test performance when applying the model to the original dataset (CNDMDS). Differences in apparent and test performances across all models were averaged to estimate overall optimism. Moreover, calibration slopes from the bootstrap samples were averaged. Slope and intercept in the calibration plot were estimated in a logistic regression model, with prediabetes as the outcome and a linear indicator as the independent variable. Linear indicators were calculated by multiplying the beta coefficients of indicators with the corresponding values for each individual. Together with the intercept, the slope was used to evaluate the amount of mis-calibration of the original model. Ideally, the slope and intercept should be 1 and 0, respectively, with the calibration curve close to the diagonal. If the curve deviates, the slope would be regarded as a shrinkage factor to adjust the original model by multiplying the factor with the original coefficients. The intercept was re-estimated given the decreased coefficients to maintain the overall calibration.

When externally validated in the NHANES and the TIDE study, risk factor parameters obtained in the CNDMDS were used to estimate prediabetes risk in the NHANES and the TIDE study. Actual prediabetes events were regressed on the predicted risk to yield C statistics. Beta coefficients were multiplied by the corresponding indicators in the risk equation to predict the log odds of prediabetes and then transformed into absolute risk by natural logarithm: prediabetes risk=1/(1+e-predicted log odds of prediabetes). A decile calibration plot visually exhibited the agreement between observed and predicted probabilities of prediabetes in the NHANES population and the TIDE population. The Hosmer-Leme show test was used to assess the calibration and calculated whether the observed prediabetes prevalence matched the expected prevalence by the model in the NHANES sample and the TIDE sample. The slope and intercept were estimated in a logistic regression model with prediabetic events in the NHANES and TIDE studies as outcomes and linear indicators as independent variables.

According to beta coefficients in the final assessment model, each indicator’s risk point was calculated, and risk scores were derived by summing the points[19]. Based on the risk score, individuals were stratified into high- and low-risk groups. A decision curve analysis was carried out to analyze the net benefit of screening a high-risk population for blood testing, assuming a threshold of prediabetes probability above which individuals were considered high-risk and opt for blood testing to detect prediabetes early.

All statistical analyses were performed using SAS 9.4 (SAS Institute Inc., Cary, North Carolina, United States) with statistical significance established at two-tailed P < 0.05. The study was conducted in accordance with the Transparent Reporting of a Multivariable assessment Model for Individual Prognosis or Diagnosis (TRIPOD) statement[20].

The CNDMDS was approved by the Ethics Review Board of China-Japan Friendship Hospital and ethics committee of local institutions. The TIDE study was approved by the medical ethics committee of China Medical University.

RESULTS

Study participants

In the CNDMDS study, 47325 participants completed the survey. After excluding 6035 participants diagnosed with diabetes and 887 with missing information on key variables, there were 40403 nondiabetic participants available for our analysis, comprising individuals with prediabetes and those with normal blood glucose levels. To reduce the influence of outliers, we excluded the following groups of patients: (1) 5 participants with waist circumference > the 99.99 percentile (172 cm in men, 163 cm in women) or < the 0.01 percentile (49 cm in men, 50 cm in women); (2) 8 participants with BMI > the 99.99 percentile (47.7 kg/m²) or < the 0.01 percentile (13.3 kg/m²); (3) 5 participants with SBP > the 99.99 percentile (222 mm/Hg) or < the 0.01 percentile (70 mm/Hg); and (4) 4 participants with DBP > the 99.99 percentile (140 mm/Hg) or < the 0.01 percentile (40 mm/Hg). Ultimately, 40381 participants were included in the derivation dataset for model development and internal validation (Figure 1A).

Figure 1 Flowchart for trimmed data in the China National Diabetes and Metabolic Disorders Study derivation dataset, data cleaning in the National Health and Nutrition Examination Survey validation dataset and data arrangement in the Thyroid Disorders, Iodine Status and Diabetes Epidemiological Survey study. A: In the China National Diabetes and Metabolic Disorders Study (CNDMDS) study, 47325 completed the survey. After excluding 6035 participants diagnosed with diabetes and 887 with missing information on key variables, there were 40403 nondiabetic participants available for our analysis. After excluding those with outlier influences of waist circumference, body mass index, systolic blood pressure and diastolic blood pressure, 40381 participants were ultimately included in the derivation dataset for model development and internal validation; B: In National Health and Nutrition Examination Survey, data from 9364 participants were available. After excluding 894 participants diagnosed with diabetes and 6359 with missing fasting glucose level or oral glucose tolerance test data, there were 2111 nondiabetic participants available for our analysis. After excluding the 586 participants with missing key variables, 1525 nondiabetic individuals were available for external validation; C: In the TIDE study, 80937 subjects participated in the survey. A total of 5057 subjects were excluded due to a lack of information on age, sex, glucose levels or HbA1c. A total of 9772 participants diagnosed with diabetes were further excluded, leaving 66108 participants in the analysis. CNDMDS: China National Diabetes and Metabolic Disorders Study; DBP: Diastolic blood pressure; OGTT: Oral glucose tolerance test; NHANES: National Health and Nutrition Examination Survey; TIDE: Thyroid disorders, Iodine status and Diabetes Epidemiological survey.

Among these 40381 participants, 6810 (16.9%) individuals had prediabetes. The overall age was 44 ± 13 years, 39.2% were men, 54% had received a middle school education, and 63% were urban residents. Of the analyzed participants, 11.7% reported a family history of DM, and 24.5% had a smoking history. The average waist circumference was 84.8 cm in men and 78.7 cm in women. Characteristics were compared between prediabetic and normal individuals (Table 1). Sex and smoking behaviors did not differ significantly between groups. They were not significantly associated with prediabetes in univariable logistic regression (Supplementary Table 2). Given that sex was an important demographic factor, it remained to be analyzed as a risk factor with other significant variables. The characteristics of the study populations in the CNDMDS and NHANES are summarized in Supplementary Table 3. Populations in NHANES had different demographic profiles from those in the CNDMDS, with more males and a higher proportion of DM family history. Except for Asians, individuals of other racial populations in NHANES had consistently greater BMI and a higher proportion of abdominal obesity than Chinese people in CNDMDS. Fasting plasma glucose levels and the prevalence of prediabetes were both higher in the populations of NHANES (Supplementary Table 3).

Table 1 Characteristics of participants with available prediabetes risk factor data in the China National Diabetes and Metabolic Disorders Study population.

Variable	All	Normal glucose tolerance	Prediabetes	P value
mean ± SD/n (%)	n = 40381	n = 33571	n = 6810
Age, yr	44 ± 13	43 ± 13	50 ± 13	< 0.001
Men	15848 (39.2)	13156 (39.2)	2692 (39.5)	0.6
Education
University or above	9375 (23.2)	8245 (24.6)	1130 (16.6)	< 0.001
Middle school	21800 (54.0)	18279 (54.4)	3521 (51.7)
Primary school	6656 (16.5)	5129 (15.3)	1527 (22.4)
Illiteracy	2550 (6.3)	1918 (5.7)	632 (9.3)
Residency, urban	25425 (63.0)	21294 (63.4)	4131 (60.7)	< 0.001
DM family history, yes	4736 (11.7)	3756 (11.2)	980 (14.4)	< 0.001
Ever smoker, yes	9874 (24.5)	8169 (24.3)	1705 (25.0)	0.22
WC, cm	81.1 ± 10.6	80.3 ± 10.4	85.4 ± 10.5	< 0.001
WC in men	84.8 ± 10.5	84.1 ± 10.2	88.5 ± 10.7
WC in women	78.7 ± 10.0	77.8 ± 9.8	83.4 ± 9.8
BMI, kg/m2	23.9 ± 3.6	23.6 ± 3.5	25.4 ± 3.7	< 0.001
Cholesterol, mmol/L	4.7 ± 1.0	4.6 ± 1.0	5.0 ± 1.0	< 0.001
Triglycerides, mmol/L	1.5 ± 1.1	1.4 ± 1.0	1.9 ± 1.3	< 0.001
HDLC, mmol/L	1.3 ± 0.3	1.3 ± 0.3	1.3 ± 0.3	< 0.001
LDLC, mmol/L	2.7 ± 0.8	2.7 ± 0.8	2.9 ± 0.8	< 0.001
SBP, mmHg	121.1 ± 18.6	119.5 ± 17.8	129.0 ± 20.7	< 0.001
DBP, mmHg	77.9 ± 11.2	77.2 ± 10.9	81.3 ± 11.6	< 0.001
Fasting plasma glucose, mmol/L	5.1 ± 0.6	4.9 ± 0.5	5.6 ± 0.7	< 0.001
2-h plasma glucose, mmol/L	6.1 ± 1.6	5.6 ± 1.1	8.3 ± 1.5	< 0.001

DM: Diabetes mellitus; BMI: Body mass index; SBP: Systolic blood pressure; DBP: Diastolic blood pressure; HDLC: High-density lipoprotein cholesterol; LDLC: Low-density lipoprotein cholesterol; WC: Waist circumference.

Model development and internal validation

Through a multivariable logistic regression test using eight candidate risk factors for prediabetes in the CNDMDS dataset, six factors, including age, education level, DM family history, waist circumference, BMI and SBP, reached statistical significance and were included in the risk assessment model of prediabetes (Table 2). The C statistic of the model was 0.6998 (95%CI: 0.6933 to 0.7063) (Supplementary Table 4, Figure 2A). After bootstrapping 200 times with replacement, optimism was estimated as 0.0006, and the optimism-corrected C statistic was 0.6992, indicating that this risk assessment model could discriminate adults with and without prediabetes, with a C statistic of 0.6992 (Supplementary Table 4). The calibration plot showed good agreement between the observed and predicted probabilities of prediabetes with an apparent calibration slope of 1.0002 (95%CI: 0.9615 to 1.039) (Supplementary Table 4, Figure 2B). The curve is seen to be close to diagonal, with statistical significance indicated by the Hosmer-Leme show test (P < 0.0001). The optimism-corrected slope was 0.9978, which was also a shrinkage factor to adjust the coefficients in the model. The intercept was recalculated as -6.1262 to maintain the apparent calibration (Supplementary Table 5). Since the ordering of predicted probabilities for models with original or decreased coefficients are identical and discrimination does not change substantially after shrinkage[21], we focused on calibration to investigate the improvement in assessment after model shrinkage. Using an assessment model with an adjusted intercept of -6.1262 and a coefficient decrease of 0.9978, the slope and intercept in the CNDMDS became 1.0049 and 0.0177, respectively (Supplementary Figure 1A).

Figure 2 Discrimination and calibration of the prediabetes assessment model in the original population of the China National Diabetes and Metabolic Disorders Study, Asian population of the National Health and Nutrition Examination Survey and population of the Thyroid disorders, Iodine status and Diabetes Epidemiological survey study. A: Area under the curve (AUC) of the original assessment model in the original population of the China National Diabetes and Metabolic Disorders Study (CNDMDS): The assessment model demonstrated acceptable discrimination, with apparent C statistics of 0.6998 (0.6933, 0.7063) and optimism-corrected C statistics of 0.6992. After recalibration, the C statistics remained the same, and the estimate (95%CI) was 0.6999 (0.6933 to 0.7064); B: Calibration plot of the assessment model in the original population of the CNDMDS: the calibration slope was 1.0002, and the intercept was 0.000378; C: AUC of the prediabetes assessment model: The C statistic was 0.7090 (95%CI: 0.6324 to 0.7857) in the Asian population in National Health and Nutrition Examination Survey (NHANES); D: Calibration plot of the assessment model: The calibration slope was 0.9238, and the intercept was 0.000376 in the Asian population in the NHANES; E: AUC of the prediabetes assessment model: The C statistic was 0.7308 (95%CI: 0.7260 to 0.7357) in the population of the Thyroid disorders, Iodine status and Diabetes Epidemiological survey (TIDE) study; F: Calibration plot of the assessment model: the calibration slope was 1.1137, and the intercept was 0.1315 in the population of the TIDE study.

Table 2 Multivariable logistic regression for prediabetes in China National Diabetes and Metabolic Disorders Study¹.

Variable	β coefficients	P value	OR (95%CI)
Intercept	-6.1358	< 0.0001	-
Age, yr	0.0274	< 0.0001	1.028 (1.025-1.030)
Middle school or higher education	-0.2032	< 0.0001	0.816 (0.766-0.870)
DM family history	0.3108	< 0.0001	1.364 (1.261-1.477)
Waist circumference ≥ normal upper limit	0.1959	< 0.0001	1.216 (1.134-1.305)
BMI	0.0761	< 0.0001	1.079 (1.069-1.089)
SBP	0.0117	< 0.0001	1.012 (1.010-1.013)

¹The initial set of risk factors for variable selection in logistic regression model: Age, gender, middle school or higher education, DM family history, waist circumference above normal upper limit, BMI, SBP, DBP. DM: Diabetes mellitus; BMI: Body mass index; SBP: Systolic blood pressure; DBP: Diastolic blood pressure.

External validation

In NHANES, 166 Mexican American, 160 other Hispanic, 631 non-Hispanic White, 337 non-Hispanic Black, and 231 Asian nondiabetic individuals were available for inclusion in the external validation study (Supplementary Table 1, Figure 1B). The C statistics were 0.7613, 0.7776, 0.6926, 0.7441 and 0.7090 in Mexican American, other Hispanic, non-Hispanic White, non-Hispanic Black and Asian populations, respectively (Supplementary Table 6, Figure 2C, Figure 3). The model was well calibrated with nonsignificant Hosmer-Leme show test results in Mexican (P = 0.4053), other Hispanic (P = 0.1583), non-Hispanic Black (P = 0.2200) and Asian (P = 0.1541) individuals but poorly calibrated in White individuals with overestimated prediabetes risk (P = 0.016) (Figure 2D, Figure 3). When applying the shrunken model to the Asian population, the slope and intercept were 0.9285 and 0.0171, respectively. The Hosmer-Leme show test, as mentioned above, was also significant in the CNDMDS (P < 0.001) but nonsignificant in the NHANES (P = 0.1467) (Supplementary Figure 1B). Comparing the calibration slopes and intercepts between the original model and the shrunken model, we preferred the original model as the final assessment model since its slope and intercept were closer to 1 and 0, respectively. Applying the final risk assessment model to the Asian population, we obtained a C statistic of 0.7090 (95%CI: 0.6324 to 0.7857) and a calibration plot with a slope slightly lower than 1 (0.9238) and intercept of 0.000376 (Figure 2C and D). The Hosmer-Leme show test was statistically nonsignificant (P = 0.1541) (Figure 2C and D).

Figure 3 Discrimination and calibration of the prediabetes prediction model in external validation among diverse populations in National Health and Nutrition Examination Survey. A: Area under the curve (AUC) of the model validated in Mexican American. The estimates (95% confidence interval) were 0.7613 (0.6725, 0.8502) in Mexican American; B: Calibration plot of the model in Mexican American. Hosmer-Leme show tests indicated good calibration of the model in Mexican American (P > 0.05); C: AUC of the model validated in other Hispanic. The estimates (95% confidence interval) were 0.7776 (0.7022, 0.8530) in other Hispanic; D: Calibration plot of the model in other Hispanic. Hosmer-Leme show tests indicated good calibration of the model in other Hispanic (P > 0.05); E: AUC of the model validated in non-Hispanic White. The estimates (95% confidence interval) were 0.6926 (0.6449, 0.7404) in non-Hispanic White; F: Calibration plot of the model in non-Hispanic White. Hosmer-Leme show tests indicated poor calibration in the White population with overestimated prediabetes risk (P = 0.016); G: AUC of the model validated in non-Hispanic Black. The estimates (95% confidence interval) were 0.7441 (0.6828, 0.8053) in non-Hispanic Black; H: Calibration plot of the model in non-Hispanic Black. Hosmer-Leme show tests indicated good calibration of the model in non-Hispanic Black (P > 0.05).

In the TIDE study, 80937 subjects participated in the survey. A total of 5057 subjects were excluded due to a lack of information on age, sex, glucose levels or HbA1c. A total of 9772 participants diagnosed with diabetes were further excluded, leaving 66108 participants in the analysis (Figure 1C). Among the 66108 participants, 27230 (41.2%) were diagnosed with prediabetes. Applying the final risk assessment model to the TIDE population, we obtained a C statistic of 0.7308 (95%CI: 0.7260 to 0.7357) and a calibration plot with a slope of 1.1137 and intercept of 0.1315 (Figure 2E and F).

External validation of the model among non-Asian populations in NHANES was indicated in Figure 3. The Area under the curve (AUC) of the model in all non-Asian populations was statistically significant. The estimates (95% confidence interval) of AUC were 0.7613 (0.6725, 0.8502), 0.7776 (0.7022, 0.8530), 0.6926 (0.6449, 0.7404), and 0.7441 (0.6828, 0.8053) in Mexican American (Figure 3A), other Hispanic (Figure 3C), non-Hispanic White (Figure 3E), and non-Hispanic Black (Figure 3G) populations, respectively. Hosmer-Leme show tests indicated good calibration of the model in Mexican, other Hispanic and Black populations (all P > 0.05) but poor calibration in the White population with overestimated prediabetes risk (P = 0.016) (Figure 3B, D, F, and H).

Prediabetes risk score and stratification

Based on the beta coefficients of the final assessment model, each risk factor’s score and total score were calculated (Table 3, Supplementary Table 7). The risk factors included age, middle/college/higher education, DM family history, waist circumference greater than the normal limit, BMI and SBP (Table 3). The risk score of each risk factor is indicated in Table 3. The total risk score theoretically ranged from -5 to 25, and it actually ranged between -5 and 23 in the CNDMDS (Supplementary Table 7). To obtain similarly high sensitivity and specificity, we determined a score of 7 points as the threshold, which was determined to be the cutoff point of risk points for prediabetes assessment (Table 3). In the CNDMDS, 14979 (37.09%) participants had a total risk ≥ 7 points, and they were defined as the high-risk subgroup for prediabetes. Using the threshold, the sensitivity and specificity were 60.19% (59.03-61.35%) and 67.59% (67.09-68.09%), respectively. Accordance between the observed and predicted probability of prediabetes and their changes over the risk point were demonstrated (Supplementary Figure 2, Supplementary Table 7). The observed and predicted probability both increased with increasing risk score (Supplementary Figure 2). The observed and predicted probabilities of prediabetes were mostly in accordance except for the score of approximately 21 due to the bias caused by a minimal number of participants with risk scores of approximately 21 (Supplementary Figure 2, Supplementary Table 7). In net benefit analysis, a blood test for individuals at increased risk (total point ≥ 7) could produce better net benefit when their predicted risk was between 0.12 and 0.28 (Supplementary Figure 3).

Table 3 Risk scores of risk factors for prediabetes.

Predictor	Categories	Reference value	Base reference	Estimates of beta coefficients	P value	Points
Intercept				-6.1358	< 0.0001
Age	20-29	24.5	24.5	0.0274	< 0.0001	0
	30-39	34.5	24.5	0.0274	< 0.0001	2
	40-49	44.5	24.5	0.0274	< 0.0001	4
	50-59	54.5	24.5	0.0274	< 0.0001	6
	60-69	64.5	24.5	0.0274	< 0.0001	8
	70-79	74.5	24.5	0.0274	< 0.0001	10
	80-100	90	24.5	0.0274	< 0.0001	13
Middle, college or higher education	0	0	0	-0.2032	< 0.0001	0
	1	1	0	-0.2032	< 0.0001	-1
DM family history	0	0	0	0.3108	< 0.0001	0
	1	1	0	0.3108	< 0.0001	2
Waist circumference greater than normal limit1	0	0	0	0.1959	< 0.0001	0
	1	1	0	0.1959	< 0.0001	1
BMI	< 18.5	17.8	21.2	0.0761	< 0.0001	-2
	18.5-23.9	21.2	21.2	0.0761	< 0.0001	0
	24.0-27.9	25.95	21.2	0.0761	< 0.0001	3
	≥ 28	30.9	21.2	0.0761	< 0.0001	5
SBP	< 120	105	125	0.0117	< 0.0001	-2
	120-129	125	125	0.0117	< 0.0001	0
	130-139	135	125	0.0117	< 0.0001	1
	140-159	150	125	0.0117	< 0.0001	2
	≥ 160	169	125	0.0117	< 0.0001	4

¹Waist circumference greater than normal limit: ≥ 90 for men and ≥ 85 for women for Asian population and ≥ 102 cm in men and ≥ 88 cm in women for Mexican American, other Hispanic, non-Hispanic White and Black populations. DM: Diabetes mellitus, BMI: Body mass index, SBP: Systolic blood pressure.

DISCUSSION

In the CNDMDS involving over 40000 study participants, nearly 17% of Chinese people had prediabetes. A similarly high prevalence of prediabetes was observed in another independent population participating in a national survey in the United States. Easily measured indicators were utilized for modeling. The identified risk factors for prediabetes included nonmodifiable (age, family history of diabetes mellitus) and modifiable indicators (e.g., education, waist circumference, BMI and SBP). Internal and external validation in two independent population-based surveys demonstrated good discriminative and calibration performance of the assessment model, with overfitting taken into consideration. The risk score derived from the established model was applied to screen the high-risk population, with ≥ 7 points as the threshold. he large sample size involved in model development and external validation across diverse populations increased the scientific rigorousness of this study. The readily available risk factors make it convenient for the model to be applied in a clinical setting as a simple predictive tool.

Prediabetes shared similar risk factors with diabetes. Some risk factors for prediabetes in this study were the same variables incorporated in the prior risk score for diabetes, such as age, family history, waist circumference, and SBP[10,18]. In a previous diabetes risk score yielded from the CNDMDS[12], almost all the risk factors were the same variables for prediabetes assessment in our study. The overlap in risk factors might be attributed to the progressive nature of prediabetes preceding diabetes[5] and the pathophysiological mechanism of hyperglycemia.

In contrast to diabetes assessment, DBP and sex were not significant risk factors for prediabetes in our study. There was heterogeneity in the risk factors for prediabetes or diabetes in existing studies[10,16-18]. High blood pressure is a common coexisting condition in diabetic patients. The coexistence of hypertension and diabetes mutually increased cardiovascular risk[22]. Compared to DBP, SBP has a greater effect on individuals’ glycemic disorder and is considered the major determinant of cardiovascular outcomes in people aged over 50 years[23]. SBP control was the focus of outcome improvement in patients receiving glycemic interventions[22,24]. SBP variability significantly increased the risk of all-cause mortality and cardiovascular events in type 2 diabetes patients, while diastolic blood pressure did not, suggesting the importance of controlling SBP in type 2 diabetes patients[25]. In the Daqing Diabetes Prevention Study, hypertension was one risk factor for diabetes and increased long-term risk of cardiovascular diseases in prediabetes patients, suggesting the importance of blood pressure control in prediabetes patients[26]. With respect to sex, males had a tendency toward increased prediabetes risk, although this was nonsignificant. Similar results were observed in a cohort of Malays in Singapore, in which sex did not differ significantly between individuals with incident prediabetes and those with normal glycemic levels. Sex was also not associated with diabetes[17]. Men had different risk factors for prediabetes than women[27]. Subgroup analyses have been performed to identify gender-specific risk factors for prediabetes or diabetes[16,28]. Further studies are warranted to elucidate the role of sex in glucose disorders.

Since our assessment model was developed in a representative sample from a nationwide survey in China, the large original sample enabled our findings to be generalizable to other populations with good external validity. Moreover, owing to the large sample size, the discordance between the observed and predicted risks was statistically significant when internally evaluating the calibration performance. The calibration ability of our assessment model remained acceptable regarding the close overlap between the calibration curve and diagonal in the calibration plot. In addition, discrimination and calibration performance were improved when externally validating our model in diverse populations that had marked racial differences in demographics and metabolic syndromes, which strengthened the validity of the model. Similar to the CNDMDS, the NHANES is a population-based survey, while the TIDE study is the latest epidemiological survey of diabetes in China. The representative nature of these surveys increases the generalizability of our findings. Moreover, risk scores derived from the assessment model can be used as an algorithm for risk stratification and identifying individuals at high risk of prediabetes, thereby implementing proper risk reduction strategies in the population. Additionally, the risk score could be used as a tool to select a subgroup of patients or define high-risk individuals in research.

As for the limitations of this study, since the assessment model for prediabetes was developed in a cross-sectional study, temporal associations between some risk factors and prediabetes were unclear (e.g., relationship between SBP and glucose level). Considering the inherent methodological limitation, a cohort study might be needed to further validate the discriminative accuracy of the model. Data on long-term outcomes, e.g., the occurrence of prediabetes during follow-up, need to be collected longitudinally to evaluate the accuracy of assessment.

The identified risk factors, although some were unmodifiable, characterized a subgroup of individuals susceptible to hyperglycemia, which would help identify a targeted population for early and intensive interventions to prevent diabetes and subsequent cardiovascular diseases. It can also help guide the appropriate allocation of resources to improve population health. Lifestyle interventions and education need to be delivered promptly to high-risk individuals, not only diabetic patients, given their shared risk factors.

CONCLUSION

Overall, based on easily measured variables, an assessment model for prediabetes was developed and externally validated to quantify the absolute risk of prediabetes in the general population. It could be easily implemented in clinical practice to screen high-risk populations and guide early interventions for diabetes prevention.

ARTICLE HIGHLIGHTS

Research background

Existing risk scores and screening instruments for hyperglycemia are mainly focused on diabetes. Prior studies on prediabetes assessment are restricted to studies with small sample sizes or low sensitivity or specificity.

Research motivation

To address the problem of scarcity in the robust assessment model of prediabetes in a large sample, we established a prediabetes risk assessment model based on the China National Diabetes and Metabolic Disorders Study (CNDMDS), which was a population-based survey involving nearly 48000 participants across China from 2007 to 2008. External validation was performed in a broad spectrum of populations that have marked racial and demographical differences.

Research objectives

This study aims to establish a robust assessment model for prediabetes and to validate the model in different populations.

Research methods

A logistic model with stepwise selection was performed to identify significant risk factors for prediabetes and was internally validated by bootstrapping in the China National Diabetes and Metabolic Disorders Study. External validations were performed in diverse populations, including populations of Hispanic (Mexican American, other Hispanic) and non-Hispanic (White, Black and Asian) participants in the National Health and Nutrition Examination Survey (NHANES) in the United States and 66108 participants in the Thyroid Disorders, Iodine Status and Diabetes Epidemiological Survey (TIDE) study in China. C statistics and calibration plots were adopted to evaluate the model’s discrimination and calibration performance.

Research results

A set of easily measured indicators (age, education, family history of diabetes, waist circumference, body mass index, and systolic blood pressure) were selected as significant risk factors. A risk assessment model was established for prediabetes with a C statistic of 0.6998 (95%CI: 0.6933 to 0.7063) and a calibration slope of 1.0002. External validation was performed in a broad spectrum of populations that have marked racial and demographical differences, and the satisfactory discrimination and calibration performance enhance the model’s generalizability across nations. A risk score was derived to assess prediabetes. Individuals with scores ≥ 7 points were at high risk of prediabetes, with a sensitivity of 60.19% and specificity of 67.59%.

Research conclusions

An easy-to-use assessment model for prediabetes was established and was internally and externally validated in different populations. The model had a satisfactory performance and could screen individuals with a high risk of prediabetes.

Research perspectives

Considering the inherent methodological limitation, a cohort study might be needed to further validate the discriminative accuracy of the model. Data on long-term outcomes, e.g., the occurrence of prediabetes during follow-up, need to be collected longitudinally to evaluate the accuracy of assessment.

ACKNOWLEDGEMENTS

We are grateful to the China National Diabetes and Metabolic Disorders Study (CNDMDS) Group and all the colleagues who contributed to the CNDMDS. We are thankful to all the colleagues who conducted the TIDE study. We appreciate all the participants who participated in the CNDMDS, NHANES and TIDE studies.