Rapid pathologic grading-based diagnosis of esophageal squamous cell carcinoma via Raman spectroscopy and a deep learning algorithm

Abstract

BACKGROUND

Esophageal squamous cell carcinoma is a major histological subtype of esophageal cancer. Many molecular genetic changes are associated with its occurrence. Raman spectroscopy has become a new method for the early diagnosis of tumors because it can reflect the structures of substances and their changes at the molecular level.

AIM

To detect alterations in Raman spectral information across different stages of esophageal neoplasia.

METHODS

Different grades of esophageal lesions were collected, and a total of 360 groups of Raman spectrum data were collected. A 1D-transformer network model was proposed to handle the task of classifying the spectral data of esophageal squamous cell carcinoma. In addition, a deep learning model was applied to visualize the Raman spectral data and interpret their molecular characteristics.

RESULTS

A comparison among Raman spectral data with different pathological grades and a visual analysis revealed that the Raman peaks with significant differences were concentrated mainly at 1095 cm^-1 (DNA, symmetric PO, and stretching vibration), 1132 cm^-1 (cytochrome c), 1171 cm^-1 (acetoacetate), 1216 cm^-1 (amide III), and 1315 cm^-1 (glycerol). A comparison among the training results of different models revealed that the 1D-transformer network performed best. A 93.30% accuracy value, a 96.65% specificity value, a 93.30% sensitivity value, and a 93.17% F1 score were achieved.

CONCLUSION

Raman spectroscopy revealed significantly different waveforms for the different stages of esophageal neoplasia. The combination of Raman spectroscopy and deep learning methods could significantly improve the accuracy of classification.

Key Words: Raman spectroscopy; Esophageal neoplasia; Early diagnosis; Deep learning algorithm; Rapid pathologic grading

Core Tip: Raman spectroscopy has become a new method for the early diagnosis of tumors. This study employed Raman spectroscopy to detect alterations in Raman spectral information across different stages of esophageal neoplasia, and a deep learning algorithm was designed to classify spectral data. In conclusion, Raman spectroscopy revealed significantly different waveforms for the different stages of esophageal neoplasia. The combination of Raman spectroscopy and deep learning methods could significantly improve the accuracy of classification and yielded high accuracy and specificity levels for the rapid pathologic grading-based diagnosis.

INTRODUCTION

Esophageal squamous cell carcinoma (ESCC) is a common malignancy with poor prognoses[1]. Early diagnosis is essential for effectively treating patients and improving their prognoses. In recent years, many studies have investigated the molecular genetic mechanism involved in the development of esophageal carcinoma, revealing the presence many molecular genetic changes[2]. Moreover, large esophageal cancer tissue samples were detected via exome sequencing, whole-genome association, and deep sequencing technology applied to the target region, and many molecular events have been identified[3-5]. These molecular changes have long been considered the theoretical basis for the early diagnosis of ESCC[6].

Raman spectroscopy is a new method for the early diagnosis of tumors that operates by tracking molecular-level changes caused by tissue malignancy. Owing to its immediate, objective and noninvasive characteristics, this detection method has been widely used to diagnose laryngeal cancer, gastric cancer, colorectal cancer, skin cancer and other lesions[7-10]. Recently, Raman spectroscopy studies concerning esophageal cancer have focused mostly on esophageal adenocarcinoma, which is common in Western countries[11,12], whereas ESCC, which is common in Chinese people, has rarely been reported. In addition, the existing studies on the tumor diagnosis applications of Raman spectroscopy have focused mainly on differentiating between tumors and nontumors but have seldom paid attention to studies of cohorts with carcinogenesis[13,14]. There is little understanding of the changes exhibited by Raman spectroscopy information in cohorts with normal, low-grade intraepithelial neoplasia (LGIN) and high-grade intraepithelial neoplasia (HGIN). However, the Raman spectroscopy information changes demonstrated by these cancer cohorts form the basis for the application of Raman spectroscopy in the task of providing early warnings and diagnoses of esophageal cancer risks. Therefore, the application of Raman spectroscopy technology for detecting spectral information changes in the tissue sections of these cancer cohorts will provide us with information for the early diagnosis of ESCC.

At present, the existing classification algorithms based on Raman spectroscopy can be roughly divided into two types: Traditional machine learning methods and deep learning methods. In a traditional machine learning method, it is often necessary to first extract the corresponding features of the input Raman spectrum data and then train different classifiers on the collected features to complete the Raman spectrum classification task. Huang et al[15] used four machine learning algorithms, namely, principal component analysis-linear discriminant analysis (PCA-LDA), PCA-extreme gradient boosting (XGB), PCA-support vector machine (SVM), and PCA-(LDA, XGB, SVM)-stacked gradient boosting machine, to analyze Raman spectral data. All four machine learning algorithms were able to distinguish esophageal squamous carcinoma cells from normal esophageal cells, and the PCA-XGB model achieved an overall prediction accuracy of 85% in terms of classifying ESCC and adjacent tissues. Bergholt et al[16] applied the LDA diagnostic model for analyzing Raman spectral data, resulting in an accuracy of 96.0% for the in vivo diagnosis of esophageal cancer. Sharma et al[17] used the partial least-squares (PLS)-SVM method to analyze prominent spectral regions containing key oral biomarkers. This approach can effectively distinguish oral squamous cell carcinoma tissues from non-oral squamous cell carcinoma tissues. However, when the sample source or spectral acquisition conditions change, the spectral response of a substance may exhibit nonlinear characteristics, which will affect the predictive performance of the model. Although traditional machine learning methods can be used to classify esophageal cancer, the feature extraction process may lead to the loss of part of the associated Raman information, thus limiting the resulting classification effect.

In recent years, with the rapid development of deep learning technology, an increasing number of studies have begun to apply it to Raman spectral analysis tasks. Owing to the strong adaptive spectral feature extraction and end-to-end learning capabilities of deep learning, we can efficiently address Raman spectral classification problems, especially when handling high-dimensional features and large-scale data. By simulating the structures and functions of biological neural networks, convolutional neural networks (CNNs) can accurately solve complex problems involving large amounts of data[18]. However, the performance of CNNs is highly dependent on the selected cores, which may result in the loss of part of the given time series information. In addition, CNNs are more suitable for processing two-dimensional image data, while their applicability to the classification and recognition of one-Villaman spectral data is weak. The original transformer model was proposed by Vaswani et al[19] of the Google Brain team in 2017. The transformer model uses a self-attention mechanism to effectively process one-dimensional text data without relying on traditional serialization operations, such as CNNs. Thus, it can efficiently model long-term dependencies in sequence data. Given the nature of the Raman spectrum with one-dimensional long series data, a transformer may be a better choice than a CNN as a model for Raman spectrum classification.

In this study, Raman spectroscopy was applied to detect Raman spectral informatics changes at all levels of ESCC in an occurrence cohort, and deep learning algorithms were designed to classify spectral data and ESCC. The transformation mechanism of ESCC was explored from the perspective of Raman spectroscopy-based detection, providing a theoretical basis for providing early diagnoses and risk warnings related to ESCC. Therefore, the main contributions of this paper are summarized as follows: (1) A transformer model was adopted to process the spectral data classification task concerning ESCC for the first time, which significantly improved the resulting classification accuracy. Additionally, we have at https://github.com/LLY-Bistu/RamanSystem provides the source code of the model; (2) Two deep learning models, a CNN and a transformer, were designed for the classification of ESCC, and the results of a comparison showed that the self-attention mechanism was more efficient in terms of processing the Raman spectrum; and (3) The deep learning models were applied to a comprehensive visual analysis and a molecular feature interpretation of Raman spectrum data.

MATERIALS AND METHODS

A new generation of high-resolution Raman spectrometers

In this study, a new generation of high-resolution Raman spectrometers (Japanese HORIBA Jobin Yvon, instrument model: LabRAM HR Evolution) built by the project team (Figure 1) was used to collect spectral data of esophageal cancer tissues. A 532-nm fiber-coupled diode laser was used as the excitation source, and the spectral resolution was ≤ 0.65 cm^-1. The device included SWIFT ultrafast imaging; thus, the time required for Raman imaging is greatly shortened, and the speed was tens of times greater than that of ordinary imaging. The open microscope was equipped with a high-definition color camera to view white light images and laser spots via LabSpec software, and it automatically switched between the white light illumination and Raman measurement modes. It was also equipped with the KnowItAll-HORIBA database for rapid chemical identification, designated peak searches, structural searches, etc.

Figure 1 A new generation of high-resolution Raman spectrometer.

Patient enrollment and sample preparation

Different grades of ESCC lesions, including 30 normal esophageal mucosal tissue sections, 30 LGIN tissue sections, and 30 HGIN tissue sections, were obtained during the processes of carcinogenesis and normal esophageal mucosa screening in the field population with a high incidence of esophageal cancer. Wax blocks from all esophageal cancer cohorts were collected and sorted in terms of their ages, genders, pathological diagnoses and other information. The pathological sections were then diagnosed by an experienced clinical pathologist to evaluate the type, stage and degree of differentiation of the disease of each patient. In this experiment, the examiner was blinded to the Raman spectrum of the measured tissue. The pathologist was also blinded to the Raman spectra of the tissue that needed to be examined for pathology. The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the Ethics Committee of Beijing Tiantan Hospital, Capital Medical University (No. KY2022-042-02) on April 28, 2022.

Acquisition of fiber Raman spectroscopy data

A paraffin-embedded tissue was sliced continuously, and two pieces of tissue were connected to prepare two paraffin sections. One piece was used for HE stain observation and Raman spectrum detection point monitoring; the other piece was used for dewaxing via the n-hexane method for Raman spectrum detection purposes. During the Raman spectroscopy detection process, we used a 50 × objective lens (whose corresponding beam spot diameter was 1.25 microns) and set the laser power to 100% (the 532-nm laser was equivalent to 13 mW/cm² at 100% power). The scanning time was set to 60 seconds and the number of accumulations was 1. For each sample, we randomly selected four different data acquisition points and collected 360 sets of Raman data in total.

Network model

Data preprocessing: In the preliminary processing stage, we first classified all the Raman spectral data into three types, namely, healthy, low-grade and high-grade data, and the spectral band covered 800-1800 cm^-1. To mitigate the effects of robust fluorescence signals and noise derived from different background sources, the spectral data needed to be preprocessed. This preprocessing work was divided into the following three steps: (1) Denoising: A Savitzky-Golay filter was used to smooth the spectrum to reduce the degree of noise interference and the window parameter is set to 5, and the polynomial order is 2; (2) Baseline correction: For the denoised data, the piecewise linear fitting method was used to fit the polynomial baseline, and this result was subtracted from the denoising spectrum to remove the fluorescent background and highlight the spectral features. The penalty factor is set to 10000, and the smoothing parameter is 0.01; and (3) Data normalization: Finally, the data was normalized to the 0-1 interval via maximum intensity normalization to facilitate comparisons between different samples.

PLS method for feature dimensionality reduction: The PLS technique is an effective supervised learning method that is aimed at reducing the dimensionality of data. By extracting the underlying components from the original space of the spectral data, PLS is able to reduce uncorrelated noise and reduce the complexity of the constructed model, resulting in a low-dimensional dataset that is suitable for deep learning model training. The number of components is a crucial parameter when optimizing PLS equations. We used the F1 score for classification tasks to evaluate the overall performance of the model. To determine the optimal number of components for the system, the effects of different values were explored and evaluated by the F1 score metric. Every 10 components, the number of components was systematically adjusted from 800 to 900, and all the test results of the deep learning model were recorded in detail. Figure 2 shows the relationship between the number of components selected and the F1 score. When the number of PLS components changed, the value of the F1 score did not change much, and when the number of components was 800, the F1 score reached a maximum value of 93.88%. Therefore, to optimize the use of subsequent resources, we decided to take 800 as the optimal number of components.

Figure 2 The relationship between the number of components selected and the F1 score.

Modeling and training: This paper proposes a 1D-transformer network model for identifying the healthy, low-grade, and high-grade states of esophageal tissue. The network structure is divided into five parts (Figure 3): Preprocessing + PLS, a linear embedding layer, a location coding layer, a coding layer, and a classification head. First, the collected Raman spectral data were denoised, baseline corrected and normalized, and then the feature dimensionality is reduced to 1 × 800 via PLS. Next, these data were fed into the linear embedding layer, and the processed spectral data were converted into low-dimensional vectors, highlighting the key peak relationships and other features. Given the continuous nature of Raman spectra, linear embedding is the best choice for understanding the complex relationships between vibration modes. Next, the position coding layer used sine and cosine functions to encode the position values that were related to the relative position of the spectrum and not to the Raman displacement. The resulting position coding matrix was combined with linearly embedded information to preserve and identify the spatial relationships within the spectrum. The rich spatial information was integrated into the encoder module, which consisted of two submodules: The first submodule integrated multihead attention mechanisms and addition and normalization functions, and the second submodule contained feedforward neural networks and addition and normalization functions, comprising a total of two encoder layers. Finally, the output derived from the encoder module was passed to the classification head, which replaced the decoder layer of the transformer, including two fully connected layers, resulting in a three-classification model.

Figure 3 Architecture of 1D-transformer network model. Raman data composed of 360 individual spectra are introduced to preprocessing + partial least-squares, a linear embedding layer, a location coding layer, a coding layer, and a classification head respectively. PLS: Partial least-squares; RELU: Rectified linear unit.

To complete the final classification process, we used the Softmax function to calculate the probability of the output. The learning rate was set to 0.01, the batch size was 16, and the training period was 100. We chose the stochastic gradient descent algorithm for optimization, with the momentum set to 0.9 and the weight decay rate set to 0.001. In addition, a cosine annealing strategy was used to dynamically adjust the learning rate so that it gradually decreased during training and was smoother in the later stages. The minimum learning rate was set to 0, the warm-up phase used a linear increase strategy, the number of warm-up iterations was 5, the initial learning rate ratio is 0.002, and the warm-up phase was carried out in an epochwise manner. To test the generalizability of the model, considering the small amount of data used when cross-validation was performed, we carried out a 10-fold cross-validation according to different categories. Specifically, health, low-level, and high-level health are each divided into 10 subsets. In each training phase, seven subsets from each of the three categories were used for training, two were employed for validation, and the remaining subset was used for testing. This process was repeated ten times; each time, a different subset was used for testing. Finally, the evaluation results of the ten models were averaged and used as model performance indicators.

To more clearly explain the power of the 1D-transformer model in Raman spectral data analyses, we used gradient-based class-weighted activation mapping (Grad-CAM). With this analytical approach, we were able to visualize the Raman spectral regions of concern for the 1D-transformer model, thereby better understanding its classification process. First, we input the spectral data into the 1D-transformer model and used Grad-CAM to plot and calculate the gradient of the feature map from the last convolutional layer. Next, we added these gradients to the feature map of the last convolutional layer via weighted summation, followed by global average pooling. After processing, the results were converted via a PLS inverse transformation into a thermal map that was consistent with the dimensions of the Raman spectrum, which was closely related to the target class. Finally, we put the heatmap on the same map as that of the original spectrogram, which could more intuitively show the spectral differences between different bands and help us better understand the characteristics and changes exhibited by the spectral data.

RESULTS

Patient information

Table 1 Lists the basic information of the subjects involved in this study, including 59 males and 31 females, ranging in age from 47 to 78 years. Tissue samples were collected from different esophageal sites, including 2 in the neck segment, 24 in the upper esophagus, 40 in the middle esophagus, and 16 in the lower esophagus. According to the pathological results, the subjects were divided into three groups, namely, healthy, low-grade neoplasia and high-grade neoplasia groups, with 30 samples in each group, for a total of 90 samples. For each sample, four different points were randomly selected, and a total of 360 spectral data points were collected.

Table 1 Summaries of the fundamental information of enrolled subjects and the corresponding number of collected Raman spectra.

Patient information		Number of patients	Number of spectra
Age	< 60 years	24
	≥ 60 years	66
Gender	Male	59
	Female	31
Test site	Neck	2
	Upper	24
	Middle	40
	Lower	16
Histological grading	Health	30	120
	Low grade	30	120
	High grade	30	120

Raman spectroscopy analysis results

According to their pathological grades, the 360 spectral data of all the tissues to be measured were divided into three categories, namely, healthy, LGIN and HGIN groups, with 120, 120 and 120 samples, respectively. The average Raman spectral data of these three categories are shown in Figure 4. The spectrograms derived from different subjects belonging to the same disease stage and pathological grade are very similar, which indicates that the same type of tissue to be measured has significant homogeneity; conversely, significant differences can be observed in the spectral data of patients with different disease stages and pathological grades. By comparing the Raman spectral data of different pathologic grades, the Raman peaks of significant differences were concentrated in the following regions (Table 2). When the low-grade disease changed to healthy, the Raman spectra were 891 cm^-1 (lauric acid), 1078 cm^-1 [v(C-C) for lipids], 1095 cm^-1 (DNA, symmetric PO, and stretching vibration), 1132 cm^-1 (cytochrome c), 1171 cm^-1 (acetoacetate), 1216 cm^-1 (amide III and CH₂ wagging), 1295 cm^-1 [amide III delta increased peaks at (CH)₂], 1369 cm^-1 (thymine), 1417 cm^-1 (CH₂ stretch), 1440 cm^-1 (lipin), 1596-1616 cm^-1 (phenylalanine C = C) and 1618 cm^-1 [v(C = C) for porphyrins][20-22]. Through a comprehensive analysis of the two types of differential Raman spectra (Figure 4), the peaks of the Raman spectra at 891, 1078, 1095, 1132, 1171, 1295, 1369, 1417 and 1618 cm^-1 of both types had corresponding changes.

Figure 4 The average Raman spectrogram of esophageal tissues with different pathological grades.

Table 2 Analysis of peak positions and assignment in Raman spectra during low-health and high-low progressions.

Progression type	Change direction	Raman shift, cm-1	Band assignments
Low-health	Increase	891	Lauric acid
		1078	v(C-C) of lipids
		1095	DNA, symmetric PO, stretching vibration
		1132	Cytochrome c
		1171	Acetoacetate
		1216	Amide III and CH2 wagging
		1295	Amide III delta (CH)2
		1369	Thymine
		1417	CH2 stretch
		1440	Lipin
		1596-1616	Phenylalanine C = C
		1618	v(C = C) of porphyrins
High-low	Decrease	891	Lauric acid
		1004	vs(C-C) ring breathing of phenylalanine
		1078	v(C-C) of lipids
		1095	DNA, symmetric PO, stretching vibration
		1132	Cytochrome c
		1171	Acetoacetate
		1295	Amide III delta (CH)2
		1315	Glycerol
		1369	Thymine
		1417	CH2 stretch
		1448-1468	CH3 deformation
		1618	v(C = C) of porphyrins

Network test results and analysis

1D-transformer: In this section, we take a more nuanced look at the predictive accuracy and error classifications associated with the 1D-transformer model across different classification categories. As shown in Figure 5, the cumulative confusion matrix summarizes the results of the 10 performed cross-validations. This protocol demonstrates the consistent and accurate performance of the model in the tumor staging classification task. In the histological grading task, the model accurately classified 117 patients as healthy, 99 as low-grade patients, and 118 as high-grade patients. Table 3 presents the results for the specificity, sensitivity, and F1 score achieved by the 1D-transformer model in the pathology grading task. As shown in the table, the maximum specificity, sensitivity and F1 score values produced for each category were 98.33% (low), 99.15% (healthy) and 95.55% (high), respectively. The minimum values were 95.0% (healthy), 83.19% (low) and 94.74% (high). By analyzing the specificity, sensitivity and F1 score values produced for the different categories, we were able to gain a more complete understanding of the performance of the model. The results showed that the model was weak in terms of classifying low classes and stronger when classifying high classes.

Figure 5 The cumulative confusion matrix summarizes the results of the 10 performed cross-validations.

Table 3 Results for the specificity, sensitivity, and F1 score achieved by the 1D-transformer model in the pathology grading task.

Classification	Category	Specificity	Sensitivity	F1 score
Histologic grading	Health	95.0	99.15	94.74
	Low	98.33	83.19	89.19
	High	96.62	97.52	95.55

In addition, we implemented 10 × cross-validation experiments to train and test the 1D-transformer network model, generating 10 different models in the process. Table 4 outlines the accuracy, specificity, sensitivity, and F1 score produced by each of the ten models in the pathology grading task. The experimental results revealed that the third model had the best performance, with accuracy, specificity, sensitivity, and F1 score values of 97.22%, 98.61%, 97.22%, and 97.22%, respectively. In contrast, the performance of the eighth model was the worst, with corresponding indices of 88.89%, 94.44%, 88.89%, and 88.30%, respectively. Through an analysis of these results, we could observe the classification performance differences among different models, thereby reflecting the influences of parameter selection and data segmentation during model training. Notably, the consistency of the performance of the 10 models negated the possibility of random variation, indicating that our model had good stability and reliability during training.

Table 4 Results for the accuracy, specificity, sensitivity, and F1 score produced by each of the ten models in the pathology grading task.

Classification	Model	Accuracy	Specificity	Sensitivity	F1 score
Histologic grading	Model 1	94.12	97.04	93.94	93.89
	Model 2	91.67	95.83	91.68	91.77
	Model 3	97.22	98.61	97.22	97.22
	Model 4	88.89	94.44	88.89	88.57
	Model 5	94.44	97.22	94.44	94.41
	Model 6	94.29	97.10	94.44	94.41
	Model 7	94.44	97.22	94.44	94.30
	Model 8	88.89	94.44	88.89	88.30
	Model 9	94.44	97.22	94.44	94.30
	Model 10	94.59	97.33	94.66	94.56

Overall evaluation results: Accuracy and the cross-entropy loss are commonly used as metrics when evaluating the performance and reliability of 1D-transformer models. As the iterative learning process progressed, the accuracy and cross-entropy loss curves attained on the validation set gradually stabilized, which indicates that the model did not have overfitting problems. Figure 6A and B show the accuracy curve and cross-entropy loss curve produced for the pathology grading task in this study. To conduct a comparison with other methods, considering the relatively small amount of available data, we chose ResNet18, Long Short-Term Memory (LSTM), Gated Recurrent Units (GRU), EfficientNet, an SVM and XGB from machine learning as comparison models. ResNet18 is a classic CNN that makes it easier for complex functions to learn function mappings by introducing a residual block structure and mitigates the gradient vanishing and explosion problems. LSTM can effectively solve the long-term dependence problem of traditional RNN by introducing input gate, forget gate and output gate mechanism, and can capture the timing characteristics of Raman spectral data, which is suitable for spectral sequence modeling. GRU, as a simplified version of LSTM, combines the gating structure into update gate and reset gate, reduces the number of parameters and improves the training efficiency, and is more suitable for lightweight Raman spectral classification tasks. EfficientNet uses a composite scaling method to uniformly adjust the depth, width and resolution of the network, achieving a balance between computational efficiency and classification accuracy, and is an efficient CNN architecture suitable for feature extraction and classification of Raman spectra. DenseNet uses a dense cross-layer connection structure to force each layer to receive the feature input of all previous layers, strengthening the local feature multiplexing capability of the spectrum. Combined with transition layer compression and global pooling strategies, Densenet can improve the classification robustness of high-resolution Raman spectra. SVMs, on the other hand, are supervised learning models based on statistical learning theory that are widely used for classification tasks. XGB is an integrated learning algorithm that progressively improves the accuracy of the constructed model through a combination of additive models and decision trees. When these machine learning methods were used, we still adopted the PLS method for feature dimensionality reduction to ensure the validity of the features and reduce the computational complexity level. By evaluating these comparison models, we could comprehensively analyze the performance of the 1D-transformer model in the Raman spectral data analysis task.

Figure 6 The accuracy curve, cross-entropy loss curve and receiver operating characteristic curve produced for the pathology grading task of different models. A and B: The accuracy curve and cross-entropy loss curve produced for the pathology grading task; C: Receiver operating characteristic curve and corresponding area under the curve value of the test set.

Table 5 shows the performance metrics produced by our 1D-transformer model, the 1D-ResNet50 model, 1D-LSTM model, 1D-GRU model, 1D-EfficientNet model, 1D-DenseNet model, the SVM, and XGB in the esophageal tumor classification task, including the accuracy, specificity, sensitivity, and F1 score values. The comparative analysis revealed that our 1D-transformer model had superior performance in the pathology grading task. The accuracy, specificity, sensitivity and F1 score generated by our model were 93.30% ± 2.53%, 94.15% ± 1.91%, 93.30% ± 2.54% and 93.17% ± 2.67%, respectively. To quantitatively evaluate the classification performance of the 1D-transformer model, we plotted the receiver operating characteristic curve for the pathological grading classification task and calculated the area under the curve value, which was 0.9927 (Figure 6C). These results further validated the superior performance of the 1D-transformer model in this task. To more intuitively observe the prediction accuracies and misjudgment rates of our classification model for different categories, we provide the confusion matrices of the tenfold cross-validation results corresponding to three different classification tasks in Figure 5. These confusion matrices can help us analyze the predictions provided by the classification model for each category and intuitively understand the classification performance of the model.

Table 5 The performance metrics.

Algorithm types	Accuracy	Specificity	Sensitivity	F1 score
PLS-Transformer	93.30 ± 2.53	96.65 ± 1.27	93.30 ± 2.54	93.17 ± 2.67
PLS-ResNet18	88.52 ± 5.44	94.28 ± 2.72	88.51 ± 5.50	88.57 ± 5.38
PLS-LSTM	80.18 ± 5.60	90.06 ± 3.11	80.09 ± 6.79	78.80 ± 6.43
PLS-GRU	88.58 ± 7.39	94.22 ± 4.02	88.47 ± 7.63	87.87 ± 7.46
PLS-EfficientNet	81.01 ± 6.10	90.64 ± 3.32	81.94 ± 5.91	80.72 ± 5.64
PLS-DenseNet	90.79 ± 3.94	95.29 ± 2.15	90.73 ± 4.09	90.62 ± 3.68
PLS-SVM	87.46 ± 7.51	94.04 ± 2.89	87.20 ± 7.16	87.30 ± 7.20
PLS-XGB	74.27 ± 5.86	85.00 ± 4.03	74.36 ± 5.09	74.4 ± 5.5

PLS: Partial least-squares; LSTM: Long Short-Term Memory; GRU: Gated Recurrent Units; SVM: Support vector machine; XGB: Extreme gradient boosting.

Finally, Figure 7A-C show the corresponding histopathological images of the three histopathological grading states of esophageal cancer: Healthy tissue, low-grade pathological tissue, and high-grade pathological tissue. Figure 7D-F present Raman spectrograms of healthy, low-grade and high-grade lesions, respectively, with their corresponding spectral ranges superimposed. These images are intended to provide users with an intuitive visual reference that can help them better understand the differences between different pathological grades. By looking at these images, users can clearly see the significant morphology and characteristic changes between healthy tissue and low- or high-grade tissue. For example, healthy tissue usually presents a more uniform cell distribution and a clearer organizational structure, whereas when neoplasia occurs, the structure and cells of the squamous epithelium are abnormal. Structural abnormalities are characterized by the destruction of the normal epithelial structure, maturation disorders, the disappearance of the cell arrangement pole and nuclear overlap. Cell abnormalities include nuclear enlargement; hyperchromatic, polymorphic, and increased karyotoplasmic ratios; increased mitotic images; and pathological mitotic images. A distinct boundary is often present between the cell and the surrounding nonneoplastic epithelium. The grade of esophageal squamous epithelial dysplasia should be based on the level of tumor cells involved in the squamous epithelium. Low-grade dysplasia was diagnosed when the lesion involved the lower half of the squamous epithelium. High-grade dysplasia was diagnosed when the lesion was present in more than 1/2 of the squamous epithelium.

Figure 7 Corresponding histopathological images and Raman spectrograms with their corresponding spectral ranges superimposed of the three histopathological grading states of esophageal cancer. A-C: The corresponding histopathological images of the three histopathological grading states of esophageal cancer: Healthy tissue, low-grade pathological tissue, and high-grade pathological tissue; D-F: Raman spectrograms of healthy, low-grade and high-grade lesions, respectively, with their corresponding spectral ranges superimposed.

Visual analysis via Grad-CAM

For our 1D-transformer model, we provided Raman spectral visualizations of healthy tissue, low-grade tissue and high-grade tissue via the Grad-CAM tool. Moreover, to more intuitively compare the different tissues, the average Raman spectra of healthy tissue, low-grade tissue and high-grade tissue are shown in Figure 8, and the heatmap of the neural network is visualized via Grad-CAM. In the visualization results, a redder color indicates that the corresponding region has a stronger influence on the target category predicted by the model, and a lighter color indicates that the region has a relatively small influence on the target category predicted by the model. By visualizing the Raman spectrum, we can observe that the 1D-transformer model focused on different Raman shift regions for different datasets. Specifically, for healthy tissue, the 1D-transformer model tended to focus on the 1080-1210 cm^-1 and 1325-1380 cm^-1 bands. For low-grade tissues, the model paid more attention to the 800-900 cm^-1, 1090-1240 cm^-1, 1300-1360 cm^-1, and 1440-1500 cm^-1 bands. For high-grade tissue, the model paid more attention to three bands, i.e., 1060-1140 cm^-1, 1285-1380 cm^-1, and 1460-1525 cm^-1. Through the experimental visualization results, we can explain the differences between different tissue components, which in turn helps us understand the effect of Raman spectral classification. In Table 6, the previously reported important biomolecules corresponding to the regions of concern that are closely related to this study and focused on the Raman shift indicated by Grad-CAM are listed. These potential biomolecules are capable of characterizing the biochemical characteristics of different biological tissues and provide interpretability for classification purposes.

Figure 8 Raman spectra of healthy tissue, low-grade tissue and high-grade tissue and the heatmap of the neural network visualized via Grad-CAM. A: Healthy tissue; B: Low-grade tissue; C: High-grade tissue.

Table 6 Analysis of peak positions and assignment in Raman spectra of health, low, and high tissues.

Raman shift, cm-1	Intensity	Band assignment
891	w	Lauric acid
1004	w	vs(C-C) ring breathing of phenylalanine
1078	w	v(C-C) of lipids
1095	w, m	DNA; symmetric PO, stretching vibration
1132	m	Cytochrome c
1171	w, m	Acetoacetate
1216	m	Amide III and CH2 wagging
1295	w	Amide III delta (CH)2
1315	m	Glycerol
1369	w	Thymine
1440	w	Lipin
1448-1468	w	CH3 deformation

Intensity: “w” means weak (0.4-0.6) and “m” means mid (0.6-0.8).

DISCUSSION

China is one of the countries with the highest incidence of esophageal cancer, especially ESCC, with new cases and deaths accounting for nearly half of the world’s esophageal cancer cases during the same period[1,23]. Early diagnosis and treatment form an effective way to improve the effects of esophageal cancer therapies[24]. Research has shown that the occurrence of ESCC involves a slow, multistage, multistep and progressive evolution process from normal esophageal mucosa to esophageal precancerous lesions, early cancer and middle and advanced cancer[25,26]. The precancerous lesion phase of ESCC is esophageal squamous intraepithelial neoplasia, which can be divided into two stages according to their degrees of nuclear dysplasia and their depths of epithelial involvement: LGIN and HGIN. The evolution process of these lesions is often accompanied by internal molecular changes[2]. Therefore, if molecular events and markers that are closely related to the occurrence and development of esophageal cancer can be detected and identified and the risk statuses of esophageal precancerous lesions can be more objectively determined via pathological diagnoses, the detection rate and diagnostic accuracy achieved for esophageal precancerous lesions can be improved, and clinical interventions and treatments can be guided.

The advantage of Raman spectroscopy is that it can detect molecular changes inside lesions and directly reveal genetic molecular changes inside tissues, especially molecular changes in the early stage of tissue cancer. The detection of these changes in living tissues can enable the prediction of the risk of esophageal cancer and guide the diagnosis process[16]. Farooq et al[27] used single-cell Raman spectroscopy in combination with microfluidic techniques to characterize the development trends of esophageal adenocarcinoma through the progression of healthy epithelial, Barrett’s esophagus and esophageal adenocarcinoma cell lines, and they could differentiate between healthy and cancerous cells with an accuracy of 97%. Huang et al[15] carried out Raman spectroscopy detection and a machine learning algorithm analysis on ESCC tissues and normal tissues with different degrees of differentiation, and the results revealed that an overall prediction accuracy of 85% could be achieved. In addition, the overall prediction accuracy reached 90.3% in terms of distinguishing between areas with low differentiation and high differentiation. Previous studies have distinguished benign and malignant tissues by analyzing Raman spectrum data and spectral characteristics. In this study, we, for the first time, used Raman spectroscopy in combination with a deep learning algorithm to analyze lesions with different grades, including changes in the Raman spectroscopy information of normal, LGIN, and HGIN tissues, in a cohort of early ESCC patients.

In this study, we compared the Raman spectral data and visual analysis data of different pathological grades. We found that the Raman peaks with significant differences were concentrated mainly at 1095 cm^-1 (DNA, symmetric PO, and stretching vibration), 1132 cm^-1 (cytochrome c), 1171 cm^-1 (acetoacetate), 1216 cm^-1 (amide III), and 1315 cm^-1 (glycerol), which is similar to previous findings[16,28,29]. These Raman peak value changes corresponded mainly to DNA, proteins, nucleic acids, amino acids and lipids in tumor cells[21,22]. The atypia of esophageal tumor cells was manifested mainly in changes in the nucleus, an increase in the nuclear volume and an increase in DNA in the nucleus. An increase in the nucleic acid content increased the intensity of Raman scattering, which also resulted in a Raman peak with obvious characteristics in the Raman spectrum of esophageal cancer tissues. During the carcinogenesis of tissue cells, owing to the relative increase in nucleic acid content in the carcinogenic tissue, the main chain structure changed, and the conformation, content and various amino acids of the protein changed, resulting in the emergence of characteristic peaks. In addition, fat cells play crucial roles in cancer and promote tumor progression by regulating the metabolism of cancer cells. Therefore, the lipid content in esophageal tumor tissues increased significantly. These results suggest that there are cell metabolism differences between different grades of esophageal intraepithelial neoplasia and normal esophageal cells in the tumor-generating cohort, resulting in characteristic peak changes corresponding to Raman spectroscopy.

On the basis of the above findings, to improve the efficiency of Raman spectroscopy data processing, we combined Raman spectroscopy technology with two deep learning methods (1D-ResNet18 and a 1D-transformer) and two classic machine learning methods (an SVM and XGB). The transformer was designed to distinguish between normal cells, LGIN, and HGIN. Before the data were fed into the model, PLS was used to reduce the dimensionality of the Raman spectral data. A comparison among the training results of different models revealed that the 1D-transformer network had the best performance. A 93.30% accuracy, a 96.65% specificity, a 93.30% sensitivity, and a 93.17% F1 score were achieved. Therefore, the combination of Raman spectroscopy data with the 1D-transformer network model can significantly improve the accuracy of cell classification prediction; this finding is similar to the detection results produced by Raman transformers in other fields[30]. In addition, the transformer network model was applied to the task of esophageal cancer classification for the first time in this study, providing a feasible new deep learning approach in this field. By comparing the transformer model with the ResNet model, we found that although ResNet demonstrated strong feature extraction capabilities, the self-attention mechanism (the core idea of the transformer) significantly improved the performance of the model in terms of processing Raman spectral data[31]. The transformer model could effectively capture the long-distance dependencies and important features in data through its self-attention mechanism, whereas ResNet is excellent at processing local features but limited with respect to capturing long-distance features[32].

The above studies indicate that the 1D-transformer model has high application value for the classification of esophageal intraepithelial neoplasia. To better facilitate clinical applications, we continued to optimize the visualizations produced for the model. Grad-CAM generates activation maps through gradient information, which can provide visualizations of the decision regions for specific categories in image classification tasks, thus improving the interpretability of the model[33]. For the application of the Grad-CAM method, we provide the activation diagram of the 1D-transformer model for esophageal cancer, which is combined with the corresponding Raman shift and molecular structure. However, there is a growing demand for model interpretability in qualitative Raman spectroscopy analysis tasks, and the applications of the Grad-CAM method are still in the exploratory stage. Herein, we verified the validity of the proposed method only by correlating the molecular structure with the Raman shift. Future studies will be aimed at further verifying the reliability of this method by other means and further improving the interpretability and practicability of the model.

This study possesses several limitations. First, the sample size of this study was relatively small, and future studies with larger samples are still needed to provide more objective indicators. Our next-phase multi-center trial will include larger samples and specifically address generalizability across geographic regions. Second, in this study, esophageal pathological sections were used for Raman spectroscopy detection, but this approach cannot provide real-time diagnostic results via Raman spectroscopy. In the future, research and the development of Raman spectroscopy-based real-time detection probes can be carried out to detect suspected esophageal lesions in vivo during gastroscopy, and a deep learning model can be applied to interpret the results in real time to guide treatment plans. This is highly important for the early diagnosis of ESCC. In the follow-up research, we plan to work together with multidisciplinary forces to gradually overcome technical problems, such as the miniaturization and integration of the probe, the stability and anti-interference of the signal, and then constantly improve the classification model, and move toward realizing the goal of real-time in vivo diagnosis based on Raman spectroscopy combining the 1D-transformer model.

CONCLUSION

Our results revealed that the Raman spectrum changed significantly at different pathological stages in the ESCC cohort, reflecting significant DNA, protein, nucleic acid, amino acid and lipid content changes. As shown in this study, Raman spectroscopy can be used to understand the evolutionary mechanism of pathological grade changes from the perspective of biochemistry and to predict the risk of esophageal cancer and guide the diagnosis process. The combination of Raman spectroscopy and deep learning methods, as shown in this study, can significantly improve the accuracy of cell classification and prediction, resulting in high diagnostic accuracy and specificity. The metrics attained in this study included an accuracy of 93.30%, a specificity of 96.65%, a sensitivity of 93.30%, and an F1 score of 93.17%. In this study, a transformer network model was applied to classify esophageal intraepithelial neoplasia for the first time, providing a feasible new deep learning method for this field.