Rectal cancer (RC) is one of the most common malignant tumors of the digestive system and has an alarmingly high incidence and mortality rate globally. Compared to conventional imaging examinations, radiomics can extract quantitative features that reflect tumor heterogeneity and mine data from medical images. In this review, we discuss the potential value of multimodal MRI-based radiomics in the diagnosis and treatment of RC, with a special emphasis on the role of peritumoral tissue characteristics in clinical decision-making. Existing studies have shown that a radiomics model integrating intratumoral and peritumoral characteristics has good application prospects in RC staging evaluation, efficacy prediction, metastasis monitoring, recurrence early warning, and prognosis judgment. At the same time, this paper also objectively analyzes the existing methodological limitations in this field, including insufficient data standardization, inadequate model validation, limited sample size and poor reproducibility of results. By combining existing evidence, this review aimed to enhance the attention of clinicians and radiologists on the characteristics of peritumoral tissues and promote the translational application of radiomics technology in the individualized treatment of RC.

To construct a deep learning model (DL) based on high-resolution T2-weighted images for preoperative differentiation between T2 and T3 stage rectal cancer (RC), and to compare its performance with experienced radiologists.

This retrospective study included 281 patients with pathologically confirmed RC from four centers (January 2017-December 2022). A DenseNet model was developed using 255 patients from three centers (training:validation ratio=8:2) and externally tested on 26 patients from a fourth center. Two experienced radiologists independently assessed T staging. Diagnostic performance was evaluated using accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUC).

The DL model outperformed radiologists in differentiating T2 and T3 stages across all datasets. In the training set, the DL model achieved an AUC of 0.810, compared to 0.578 and 0.625 for radiologists A and B, respectively. In the external test set, the DL model maintained superior diagnostic performance (AUC=0.715) compared to radiologist A (AUC=0.549) and radiologist B (AUC=0.493). The DL model demonstrated higher accuracy for T2 staging (0.625-0.787) and T3 staging (0.611-0.814) compared to radiologists (0.373-0.526 for T2; 0.611-0.783 for T3), who showed a tendency to over-stage T2 tumors. Inter-observer agreement between radiologists was moderate (kappa=0.451).

The DenseNet-based DL model demonstrated superior accuracy and diagnostic efficiency than radiologists in preoperative differentiation between T2 and T3 stages RC. This automated approach could potentially improve staging accuracy and support clinical decision-making in RC treatment planning.

Artificial intelligence (AI), particularly deep learning, has demonstrated remarkable performance in medical imaging across a variety of modalities, including X-ray, computed tomography (CT), magnetic resonance imaging (MRI), ultrasound, positron emission tomography (PET), and pathological imaging. However, most existing state-of-the-art AI techniques are task-specific and focus on a limited range of imaging modalities. Compared to these task-specific models, emerging foundation models represent a significant milestone in AI development. These models can learn generalized representations of medical images and apply them to downstream tasks through zero-shot or few-shot fine-tuning. Foundation models have the potential to address the comprehensive and multifactorial challenges encountered in clinical practice. This article reviews the clinical applications of both task-specific and foundation models, highlighting their differences, complementarities, and clinical relevance. We also examine their future research directions and potential challenges. Unlike the replacement relationship seen between deep learning and traditional machine learning, task-specific and foundation models are complementary, despite inherent differences. While foundation models primarily focus on segmentation and classification, task-specific models are integrated into nearly all medical image analyses. However, with further advancements, foundation models could be applied to other clinical scenarios. In conclusion, all indications suggest that task-specific and foundation models, especially the latter, have the potential to drive breakthroughs in medical imaging, from image processing to clinical workflows.

Accurate identification of microvascular invasion (MVI) in hepatocellular carcinoma (HCC) is crucial for treatment and prognosis. Single-modality and feature fusion models using manual segmentation fail to provide insights into MVI. This study aims to develop a DeepLab V3+ model for automated segmentation of HCC magnetic resonance (MR) images and a decision fusion model to predict MVI and early recurrence (ER).

This retrospective study included 209 HCC patients (146 in the training and 63 in the test cohorts). The performance of DeepLab V3+ for HCC MR image segmentation was evaluated using Dice Loss and F1 score. Intraclass correlation coefficients (ICCs) assessed feature extraction reliability. Spearman's correlation analyzed the relationship between tumor volumes from automated and manual segmentation, with agreement evaluated using Bland-Altman plots. Model performance was assessed using the area under the receiver operating characteristic curve (ROC AUC), calibration curves, and decision curve analysis. A nomogram predicted ER of HCC after surgery, with Kaplan-Meier analysis for 2-year recurrence-free survival (RFS).

The DeepLab V3+ model demonstrated high segmentation accuracy, with strong agreement in feature extraction (ICC: 0.802-0.999). The decision fusion model achieved AUCs of 0.968 and 0.878 for MVI prediction, and the nomogram for predicting ER yielded AUCs of 0.782 and 0.690 in the training and test cohorts, respectively, with significant RFS differences between the risk groups.

The DeepLab V3+ model accurately segmented HCC. The decision fusion model significantly improved MVI prediction, and the nomogram offered valuable insights into recurrence risk for clinical decision-making.

The datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request.

This study aimed to address the limitations of conventional methods for measuring skeletal muscle mass for sarcopenia diagnosis by introducing an artificial intelligence (AI) system for direct computed tomography (CT) analysis. The primary focus was on enhancing simplicity, reproducibility, and convenience, and assessing the accuracy and speed of AI compared with conventional methods. A cohort of 3096 cases undergoing CT imaging up to the third lumbar (L3) level between 2011 and 2021 were included. Random division into preprocessing and sarcopenia cohorts was performed, with further random splits into training and validation cohorts for BMI_AI and Body_AI creation. Sarcopenia_AI utilizes the Skeletal Muscle Index (SMI), which is calculated as (total skeletal muscle area at L3)/(height)2. The SMI was conventionally measured twice, with the first as the AI label reference and the second for comparison. Agreement and diagnostic change rates were calculated. Three groups were randomly assigned and 10 images before and after L3 were collected for each case. AI models for body region detection (Deeplabv3) and sarcopenia diagnosis (EfficientNetV2-XL) were trained on a supercomputer, and their abilities and speed per image were evaluated. The conventional method showed a low agreement rate (κ coefficient) of 0.478 for the test cohort and 0.236 for the validation cohort, with diagnostic changes in 43% of cases. Conversely, the AI consistently produced identical results after two measurements. The AI demonstrated robust body region detection ability (intersection over Union (IoU) = 0.93), accurately detecting only the body region in all images. The AI for sarcopenia diagnosis exhibited high accuracy, with a sensitivity of 82.3%, specificity of 98.1%, and a positive predictive value of 89.5%. In conclusion, the reproducibility of the conventional method for sarcopenia diagnosis was low. The developed sarcopenia diagnostic AI, with its high positive predictive value and convenient diagnostic capabilities, is a promising alternative for addressing the shortcomings of conventional approaches.

Adaptive radiotherapy for patients with rectal cancer using a magnetic resonance-guided linear accelerator has limitations in managing bladder shape variations. Conventional couch shifts may result in missing the target while requiring a large margin. Conversely, fully adaptive strategy is time-consuming. Therefore, a more efficient strategy for online adaptive radiotherapy is required.

This retrospective study included 50 fractions from 10 patients with rectal cancer undergoing preoperative radiotherapy. The proposed method involved preparing a library of reference plans (LoRP) based on diverse bladder shapes. For each fraction, a plan from the LoRP was selected based on daily bladder filling. This plan was compared with those generated by conventional couch shift and fully adaptive strategies. The clinical acceptability of the plans (i.e., per protocol, variation-acceptable, or unacceptable) was assessed.

In per protocol criterion, 44 %, 6 %, and 100 % of the plans for LoRP, conventional couch shift, and fully adaptive strategies were achieved, respectively. In variation-acceptable criterion, 92 % of LoRP plans and 74 % of conventional couch shift plans were achieved. LoRP demonstrated 94 % target coverage (100 % prescription dose) in the fully adaptive strategy compared with 91 % in conventional couch shift strategy. The fully adaptive strategy had the best performance in sparing the intestine and colon. LoRP reduced the treatment session duration by more than a third (>20 min) compared with the fully adaptive strategy.

LoRP achieved adequate target coverage with a short treatment session duration, potentially increasing treatment efficiency and improving patient comfort.

The automatic segmentation approaches of rectal cancer from magnetic resonance imaging (MRI) are very valuable to relieve physicians from heavy workloads and enhance working efficiency. This study aimed to compare the segmentation accuracy of a proposed model with the other three models and the inter-observer consistency.

A total of 65 patients with rectal cancer who underwent MRI examination were enrolled in our cohort and were randomly divided into a training cohort (n = 45) and a validation cohort (n = 20). Two experienced radiologists independently segmented rectal cancer lesions. A novel segmentation model (AttSEResUNet) was trained on T2WI based on ResUNet and attention mechanisms. The segmentation performance of the AttSEResUNet, U-Net, ResUNet and U-Net with Attention Gate (AttUNet) was compared, using Dice similarity coefficient (DSC), Hausdorff distance (HD), mean distance to agreement (MDA) and Jaccard index. The segmentation variability of automatic segmentation models and inter-observer was also evaluated.

The AttSEResUNet with post-processing showed perfect lesion recognition rate (100%) and false recognition rate (0), and its evaluation metrics outperformed other three models for two independent readers (observer 1: DSC = 0.839 ± 0.112, HD = 9.55 ± 6.68, MDA = 0.556 ± 0.722, Jaccard index = 0.736 ± 0.150; observer 2: DSC = 0.856 ± 0.099, HD = 11.0 ± 10.1, MDA = 0.789 ± 1.07, Jaccard index = 0.673 ± 0.130). The segmentation performance of AttSEResUNet was comparable and similar to manual variability (DSC = 0.857 ± 0.115, HD = 10.0 ± 10.0, MDA = 0.704 ± 1.17, Jaccard index = 0.666 ± 0.139).

Comparing with other three models, the proposed AttSEResUNet model was demonstrated as a more accurate model for contouring the rectal tumours in axial T2WI images, whose variability was similar to that of inter-observer.

Colorectal cancer is a major public health issue, causing significant morbidity and mortality worldwide. Treatment for colorectal cancer often has a significant impact on patients' quality of life, which can vary over time and across individuals. The application of artificial intelligence and machine learning techniques has great potential for optimizing patient outcomes by providing valuable insights. In this paper, we propose a multimodal machine learning framework for the prediction of quality of life indicators in colorectal cancer patients at various temporal stages, leveraging both clinical data and computed tomography scan images. Additionally, we identify key predictive factors for each quality of life indicator, thereby enabling clinicians to make more informed treatment decisions and ultimately enhance patient outcomes. Our approach integrates data from multiple sources, enhancing the performance of our predictive models. The analysis demonstrates a notable improvement in accuracy for some indicators, with results for the Wexner score increasing from 24% to 48% and for the Anorectal Ultrasound score from 88% to 96% after integrating data from different modalities. These results highlight the potential of multimodal learning to provide valuable insights and improve patient care in real-world applications.

Lung and colon cancers are among the leading causes of cancer-related mortality worldwide. Early and accurate detection of these cancers is crucial for effective treatment and improved patient outcomes. False or incorrect detection is harmful. Accurately detecting cancer in a patient's tissue is crucial to their effective treatment. While analyzing tissue samples is complicated and time-consuming, deep learning techniques have made it possible to complete this process more efficiently and accurately. As a result, researchers can study more patients in a shorter amount of time and at a lower cost. Much research has been conducted to investigate deep learning models that require great computational ability and resources. However, none of these have had a 100% accurate detection rate for these life-threatening malignancies. Misclassified or falsely detecting cancer can have very harmful consequences. This research proposes a new lightweight, parameter-efficient, and mobile-embedded deep learning model based on a 1D convolutional neural network with squeeze-and-excitation layers for efficient lung and colon cancer detection. This proposed model diagnoses and classifies lung squamous cell carcinomas and adenocarcinoma of the lung and colon from digital pathology images. Extensive experiment demonstrates that our proposed model achieves 100% accuracy for detecting lung, colon, and lung and colon cancers from the histopathological (LC25000) lung and colon datasets, which is considered the best accuracy for around 0.35 million trainable parameters and around 6.4 million flops. Compared with the existing results, our proposed architecture shows state-of-the-art performance in lung, colon, and lung and colon cancer detection.

Rectal cancer (RC) is one of the most common types of cancer worldwide. With the development of artificial intelligence (AI), the application of AI in preoperative evaluation and follow-up treatment of RC based on magnetic resonance imaging (MRI) has been the focus of research in this field. This review was conducted to develop comprehensive insight into the current research progress, hotspots, and future trends in AI based on MRI in RC, which remains to be studied.

Literature related to AI based on MRI and RC, as of November 2023, was obtained from the Web of Science Core Collection database. Visualization and bibliometric analyses of publication quantity and content were conducted to explore temporal trends, spatial distribution, collaborative networks, influential articles, keyword co-occurrence, and research directions.

A total of 177 papers (152 original articles and 25 reviews) were identified from 24 countries/regions, 351 institutions, and 81 journals. Since 2019, the number of studies on this topic has rapidly increased. China and the United States have contributed the highest number of publications and institutions, cultivating the most intimate collaborative relationship. The highest number of articles derive from Sun Yat-sen University, while Frontiers in Oncology has published the highest number of relevant articles. Research on MRI-based AI in this field has mainly focused on preoperative diagnosis and prediction of treatment efficacy and prognosis.

This study provides an objective and comprehensive overview of the publications on MRI-based AI in RC and identifies the present research landscape, hotspots, and prospective trends in this field, which can provide valuable guidance for scholars worldwide.

To construct and compare merged models integrating clinical factors, MRI-based radiomics features and deep learning (DL) models for predicting pathological complete response (pCR) to neoadjuvant chemoradiotherapy (nCRT) in patients with locally advanced rectal cancer (LARC).

Totally 197 patients with LARC administered surgical resection after nCRT were assigned to cohort 1 (training and test sets); meanwhile, 52 cases were assigned to cohort 2 as a validation set. Radscore and DL models were established for predicting pCR applying pre- and post-nCRT MRI data, respectively. Different merged models integrating clinical factors, Radscore and DL model were constituted. Their predictive performances were validated and compared by receiver operating characteristic (ROC) and decision curve analyses (DCA).

Merged models were established integrating selected clinical factors, Radscore and DL model for pCR prediction. The areas under the ROC curves (AUCs) of the pre-nCRT merged model were 0.834 (95% CI: 0.737-0.931) and 0.742 (95% CI: 0.650-0.834) in test and validation sets, respectively. The AUCs of the post-nCRT merged model were 0.746 (95% CI: 0.636-0.856) and 0.737 (95% CI: 0.646-0.828) in test and validation sets, respectively. DCA showed that the pretreatment algorithm could yield enhanced clinically benefit than the post-nCRT approach.

The pre-nCRT merged model including clinical factors, Radscore and DL model constitutes an effective non-invasive tool for pCR prediction in LARC.

In online adaptive magnetic resonance image (MRI)-guided radiotherapy (MRIgRT), manual contouring of rectal tumors on daily images is labor-intensive and time-consuming. Automation of this task is complex due to substantial variation in tumor shape and location between patients. The aim of this work was to investigate different approaches of propagating patient-specific prior information to the online adaptive treatment fractions to improve deep-learning based auto-segmentation of rectal tumors.

243 T2-weighted MRI scans of 49 rectal cancer patients treated on the 1.5T MR-Linear accelerator (MR-Linac) were utilized to train models to segment rectal tumors. As benchmark, an MRI_only auto-segmentation model was trained. Three approaches of including a patient-specific prior were studied: 1. include the segmentations of fraction 1 as extra input channel for the auto-segmentation of subsequent fractions, 2. fine-tuning of the MRI_only model to fraction 1 (PSF_1) and 3. fine-tuning of the MRI_only model on all earlier fractions (PSF_cumulative). Auto-segmentations were compared to the manual segmentation using geometric similarity metrics. Clinical impact was assessed by evaluating post-treatment target coverage.

All patient-specific methods outperformed the MRI_only segmentation approach. Median 95th percentile Hausdorff (95HD) were 22.0 (range: 6.1-76.6) mm for MRI_only segmentation, 9.9 (range: 2.5-38.2) mm for MRI+prior segmentation, 6.4 (range: 2.4-17.8) mm for PSF_1 and 4.8 (range: 1.7-26.9) mm for PSF_cumulative. PSF_cumulative was found to be superior to PSF_1 from fraction 4 onward (p = 0.014).

Patient-specific fine-tuning of automatically segmented rectal tumors, using images and segmentations from all previous fractions, yields superior quality compared to other auto-segmentation approaches.

Rectal cancer (RC) is a globally prevalent malignant tumor, presenting significant challenges in its management and treatment. Currently, magnetic resonance imaging (MRI) offers superior soft tissue contrast and radiation-free effects for RC patients, making it the most widely used and effective detection method. In early screening, radiologists rely on patients' medical radiology characteristics and their extensive clinical experience for diagnosis. However, diagnostic accuracy may be hindered by factors such as limited expertise, visual fatigue, and image clarity issues, resulting in misdiagnosis or missed diagnosis. Moreover, the distribution of surrounding organs in RC is extensive with some organs having similar shapes to the tumor but unclear boundaries; these complexities greatly impede doctors' ability to diagnose RC accurately. With recent advancements in artificial intelligence, machine learning techniques like deep learning (DL) have demonstrated immense potential and broad prospects in medical image analysis. The emergence of this approach has significantly enhanced research capabilities in medical image classification, detection, and segmentation fields with particular emphasis on medical image segmentation. This review aims to discuss the developmental process of DL segmentation algorithms along with their application progress in lesion segmentation from MRI images of RC to provide theoretical guidance and support for further advancements in this field.

Segmentation of clinical target volumes (CTV) on medical images can be time-consuming and is prone to interobserver variation (IOV). This is a problem for online adaptive radiation therapy, where CTV segmentation must be performed every treatment fraction, leading to longer treatment times and logistic challenges. Deep learning (DL)-based auto-contouring has the potential to speed up CTV contouring, but its current clinical use is limited. One reason for this is that it can be time-consuming to verify the accuracy of CTV contours produced using auto-contouring, and there is a risk of bias being introduced. To be accepted by clinicians, auto-contouring must be trustworthy. Therefore, there is a need for a comprehensive commissioning framework when introducing DL-based auto-contouring in clinical practice. We present such a framework and apply it to an in-house developed DL model for auto-contouring of the CTV in rectal cancer patients treated with MRI-guided online adaptive radiation therapy.

The framework for evaluating DL-based auto-contouring consisted of 3 steps: (1) Quantitative evaluation of the model's performance and comparison with IOV; (2) Expert observations and corrections; and (3) Evaluation of the impact on expected volumetric target coverage. These steps were performed on independent data sets. The framework was applied to an in-house trained nnU-Net model, using the data of 44 rectal cancer patients treated at our institution.

The framework established that the model's performance after expert corrections was comparable to IOV, and although the model introduced a bias, this had no relevant impact on clinical practice. Additionally, we found a substantial time gain without reducing quality as determined by volumetric target coverage.

Our framework provides a comprehensive evaluation of the performance and clinical usability of target auto-contouring models. Based on the results, we conclude that the model is eligible for clinical use.

Colorectal cancer (CRC) is a leading cause of cancer-related deaths. Recent advancements in genomic technologies and analytical approaches have revolutionized CRC research, enabling precision medicine. This review highlights the integration of multi-omics, spatial omics, and artificial intelligence (AI) in advancing precision medicine for CRC. Multi-omics approaches have uncovered molecular mechanisms driving CRC progression, while spatial omics have provided insights into the spatial heterogeneity of gene expression in CRC tissues. AI techniques have been utilized to analyze complex datasets, identify new treatment targets, and enhance diagnosis and prognosis. Despite the tumor's heterogeneity and genetic and epigenetic complexity, the fusion of multi-omics, spatial omics, and AI shows the potential to overcome these challenges and advance precision medicine in CRC. The future lies in integrating these technologies to provide deeper insights and enable personalized therapies for CRC patients.

Rectal cancer (RC) is the third most frequently diagnosed cancer worldwide, and the status of its circumferential resection margin (CRM) is of paramount significance for treatment strategies and prognosis. CRM involvement is defined as tumor touching or within 1 mm from the outermost part of tumor or outer border of the mesorectal or lymph node deposits to the resection margin. The incidence of involved CRM varied from 5.4 % to 36 %, which may associate with an in consistent definition of CRM, the quality of surgeries, and the different examination modalities. Although T and N status are essential factors in determining whether a patient should receive neoadjuvant therapy before surgery, CRM status is a powerful predictor of local and distant recurrence as well as survival rate. This review explores the significance of CRM, the various assessment methods, and the role of magnetic resonance imaging (MRI) and artificial intelligence-based MRI in predicting CRM status. MRI showed potential advantage in predicting CRM status with a high sensitivity and specificity compared to computed tomography (CT). We also discuss MRI advancements in RC imaging, including conventional MRI with body coil, high-resolution MRI with phased-array coil, and endorectal MRI. Along with a discussion of artificial intelligence-based MRI techniques to predict the CRM status of RCs before and after treatments.

Perineural invasion (PNI) has been used as an important pathological indicator and independent prognostic factor for patients with rectal cancer (RC). Preoperative prediction of PNI status is helpful for individualized treatment of RC. Recently, several radiomics studies have been used to predict the PNI status in RC, demonstrating a good predictive effect, but the results lacked generalizability. The preoperative prediction of PNI status is still challenging and needs further study.

To establish and validate an optimal radiomics model for predicting PNI status preoperatively in RC patients.

This retrospective study enrolled 244 postoperative patients with pathologically confirmed RC from two independent centers. The patients underwent pre-operative high-resolution magnetic resonance imaging (MRI) between May 2019 and August 2022. Quantitative radiomics features were extracted and selected from oblique axial T2-weighted imaging (T2WI) and contrast-enhanced T1WI (T1CE) sequences. The radiomics signatures were constructed using logistic regression analysis and the predictive potential of various sequences was compared (T2WI, T1CE and T2WI + T1CE fusion sequences). A clinical-radiomics (CR) model was established by combining the radiomics features and clinical risk factors. The internal and external validation groups were used to validate the proposed models. The area under the receiver operating characteristic curve (AUC), DeLong test, net reclassification improvement (NRI), integrated discrimination improvement (IDI), calibration curve, and decision curve analysis (DCA) were used to evaluate the model performance.

Among the radiomics models, the T2WI + T1CE fusion sequences model showed the best predictive performance, in the training and internal validation groups, the AUCs of the fusion sequence model were 0.839 [95% confidence interval (CI): 0.757-0.921] and 0.787 (95%CI: 0.650-0.923), which were higher than those of the T2WI and T1CE sequence models. The CR model constructed by combining clinical risk factors had the best predictive performance. In the training and internal and external validation groups, the AUCs of the CR model were 0.889 (95%CI: 0.824-0.954), 0.889 (95%CI: 0.803-0.976) and 0.894 (95%CI: 0.814-0.974). Delong test, NRI, and IDI showed that the CR model had significant differences from other models (P < 0.05). Calibration curves demonstrated good agreement, and DCA revealed significant benefits of the CR model.

The CR model based on preoperative MRI radiomics features and clinical risk factors can preoperatively predict the PNI status of RC noninvasively, which facilitates individualized treatment of RC patients.

Artificial intelligence's (AI) emergence in radiology elicits both excitement and uncertainty. AI holds promise for improving radiology with regards to clinical practice, education, and research opportunities. Yet, AI systems are trained on select datasets that can contain bias and inaccuracies. Radiologists must understand these limitations and engage with AI developers at every step of the process - from algorithm initiation and design to development and implementation - to maximize benefit and minimize harm that can be enabled by this technology.

Integrating artificial intelligence (AI) into gastrointestinal (GI) endoscopy heralds a significant leap forward in managing GI disorders. AI-enabled applications, such as computer-aided detection and computer-aided diagnosis, have significantly advanced GI endoscopy, improving early detection, diagnosis and personalized treatment planning. AI algorithms have shown promise in the analysis of endoscopic data, critical in conditions with traditionally low diagnostic sensitivity, such as indeterminate biliary strictures and pancreatic cancer. Convolutional neural networks can markedly improve the diagnostic process when integrated with cholangioscopy or endoscopic ultrasound, especially in the detection of malignant biliary strictures and cholangiocarcinoma. AI's capacity to analyze complex image data and offer real-time feedback can streamline endoscopic procedures, reduce the need for invasive biopsies, and decrease associated adverse events. However, the clinical implementation of AI faces challenges, including data quality issues and the risk of overfitting, underscoring the need for further research and validation. As the technology matures, AI is poised to become an indispensable tool in the gastroenterologist's arsenal, necessitating the integration of robust, validated AI applications into routine clinical practice. Despite remarkable advances, challenges such as operator-dependent accuracy and the need for intricate examinations persist. This review delves into the transformative role of AI in enhancing endoscopic diagnostic accuracy, particularly highlighting its utility in the early detection and personalized treatment of GI diseases.

Diagnostic performance of prostate MRI depends on high-quality imaging. Prostate MRI quality is inversely proportional to the amount of rectal gas and distention. Early detection of poor-quality MRI may enable intervention to remove gas or exam rescheduling, saving time. We developed a machine learning based quality prediction of yet-to-be acquired MRI images solely based on MRI rapid localizer sequence, which can be acquired in a few seconds.

The dataset consists of 213 (147 for training and 64 for testing) prostate sagittal T2-weighted (T2W) MRI localizer images and rectal content, manually labeled by an expert radiologist. Each MRI localizer contains seven two-dimensional (2D) slices of the patient, accompanied by manual segmentations of rectum for each slice. Cascaded and end-to-end deep learning models were used to predict the quality of yet-to-be T2W, DWI, and apparent diffusion coefficient (ADC) MRI images. Predictions were compared to quality scores determined by the experts using area under the receiver operator characteristic curve and intra-class correlation coefficient.

In the test set of 64 patients, optimal versus suboptimal exams occurred in 95.3% (61/64) versus 4.7% (3/64) for T2W, 90.6% (58/64) versus 9.4% (6/64) for DWI, and 89.1% (57/64) versus 10.9% (7/64) for ADC. The best performing segmentation model was 2D U-Net with ResNet-34 encoder and ImageNet weights. The best performing classifier was the radiomics based classifier.

A radiomics based classifier applied to localizer images achieves accurate diagnosis of subsequent image quality for T2W, DWI, and ADC prostate MRI sequences.

Pancreatic cancer, an exceptionally malignant tumor of the digestive system, presents a challenge due to its lack of typical early symptoms and highly invasive nature. The majority of pancreatic cancer patients are diagnosed when curative surgical resection is no longer possible, resulting in a poor overall prognosis. In recent years, the rapid progress of Artificial intelligence (AI) in the medical field has led to the extensive utilization of machine learning and deep learning as the prevailing approaches. Various models based on AI technology have been employed in the early screening, diagnosis, treatment, and prognostic prediction of pancreatic cancer patients. Furthermore, the development and application of three-dimensional visualization and augmented reality navigation techniques have also found their way into pancreatic cancer surgery. This article provides a concise summary of the current state of AI technology in pancreatic cancer and offers a promising outlook for its future applications.

The esophageal adenocarcinoma is facing a worldwide challenge: early prediction and risk assessment in clinical Barrett's esophagus (BE). In recent years, the growing interests have been witnessed in prediction and risk assessment in clinical BE. However, the resolution is limited, and the system is huge and expensive for the existing devices. Inspired by the principle of collaboration between human eye vision and brain cortex in data processing, here we propose multimodal learning framework to tackle tasks from various modalities, which can benefit from each other. To our findings, the experimental result indicates that low-level modality can directly affect high-level modality and form the final risk grading based on contribution, which maximizes the clinical performance of medical professionals based on our findings.

The integration of artificial intelligence (AI) and magnetic resonance imaging in rectal cancer has the potential to enhance diagnostic accuracy by identifying subtle patterns and aiding tumor delineation and lymph node assessment. According to our systematic review focusing on convolutional neural networks, AI-driven tumor staging and the prediction of treatment response facilitate tailored treat-ment strategies for patients with rectal cancer.

This paper summarizes the current landscape of AI in the imaging field of rectal cancer, emphasizing the performance reporting design based on the quality of the dataset, model performance, and external validation.

AI-driven tumor segmentation has demonstrated promising results using various convolutional neural network models. AI-based predictions of staging and treatment response have exhibited potential as auxiliary tools for personalized treatment strategies. Some studies have indicated superior performance than conventional models in predicting microsatellite instability and KRAS status, offer-ing noninvasive and cost-effective alternatives for identifying genetic mutations.

Image-based AI studies for rectal can-cer have shown acceptable diagnostic performance but face several challenges, including limited dataset sizes with standardized data, the need for multicenter studies, and the absence of oncologic relevance and external validation for clinical implantation. Overcoming these pitfalls and hurdles is essential for the feasible integration of AI models in clinical settings for rectal cancer, warranting further research.

The segmentation of organs and structures is a critical component of radiation therapy planning, with manual segmentation being a laborious and time-consuming task. Interobserver variability can also impact the outcomes of radiation therapy. Deep neural networks have recently gained attention for their ability to automate segmentation tasks, with convolutional neural networks (CNNs) being a popular approach. This article provides a descriptive review of the literature on deep learning (DL) techniques for segmentation in radiation therapy planning. This review focuses on five clinical sub-sites and finds that U-net is the most commonly used CNN architecture. The studies using DL for image segmentation were included in brain, head and neck, lung, abdominal, and pelvic cancers. The majority of DL segmentation articles in radiation therapy planning have concentrated on normal tissue structures. N-fold cross-validation was commonly employed, without external validation. This research area is expanding quickly, and standardization of metrics and independent validation are critical to benchmarking and comparing proposed methods.

To determine whether the texture feature analysis of multi-phase abdominal CT can provide a robust prediction of benign and malignant, histological subtype, pathological stage, nephrectomy risk, pathological grade, and Ki67 index in renal tumor.

A total of 1051 participants with renal tumor were split into the internal cohort (850 patients from four different hospitals) and the external testing cohort (201 patients from another local hospital). The proposed framework comprised a 3D-kidney and tumor segmentation model by 3D-UNet, a feature extractor for the regions of interest based on radiomics and image dimension reduction, and the six classifiers by XGBoost. A quantitative model interpretation method called SHAP was used to explore the contribution of each feature.

The proposed multi-phase abdominal CT model provides robust prediction for benign and malignant, histological subtype, pathological stage, nephrectomy risk, pathological grade, and Ki67 index in the internal validation set, with the AUROC values of 0.88 ± 0.1, 0.90 ± 0.1, 0.91 ± 0.1, 0.89 ± 0.1, 0.84 ± 0.1, and 0.88 ± 0.1, respectively. The external testing set also showed impressive results, with AUROC values of 0.83 ± 0.1, 0.83 ± 0.1, 0.85 ± 0.1, 0.81 ± 0.1, 0.79 ± 0.1, and 0.81 ± 0.1, respectively. The radiomics feature including the first-order statistics, the tumor size-related morphology, and the shape-related tumor features contributed most to the model predictions.

Automatic texture feature analysis of abdominal multi-phase CT provides reliable predictions for multi-tasks, suggesting the potential usage of clinical application.

The automatic texture feature analysis framework, based on multi-phase abdominal CT, provides robust and reliable predictions for multi-tasks. These valuable insights can serve as a guiding tool for clinical diagnosis and treatment, making medical imaging an essential component in the process.

• The automatic texture feature analysis framework based on multi-phase abdominal CT can provide more accurate prediction of benign and malignant, histological subtype, pathological stage, nephrectomy risk, pathological grade, and Ki67 index in renal tumor. • The quantitative decomposition of the prediction model was conducted to explore the contribution of the extracted feature. • The study involving 1051 patients from 5 medical centers, along with a heterogeneous external data testing strategy, can be seamlessly transferred to various tasks involving new datasets.

According to the Global Cancer Statistics 2020, colorectal cancer has the third-highest diagnosis rate (10.0 %) and the second-highest mortality rate (9.4 %) among the 36 types. Rectal cancer accounts for a large proportion of colorectal cancer. The size and shape of the rectal tumor can directly affect the diagnosis and treatment by doctors. The existing rectal tumor segmentation methods are based on two-dimensional slices, which cannot analyze a patient's tumor as a whole and lose the correlation between slices of MRI image, so the practical application value is not high.

In this paper, a three-dimensional rectal tumor segmentation model is proposed. Firstly, image preprocessing is performed to reduce the effect caused by the unbalanced proportion of background region and target region, and improve the quality of the image. Secondly, a dual-path fusion network is designed to extract both global features and local detail features of rectal tumors. The network includes two encoders, a residual encoder for enhancing the spatial detail information and feature representation of the tumor and a transformer encoder for extracting global contour information of the tumor. In the decoding stage, we merge the information extracted from the dual paths and decode them. In addition, for the problem of the complex morphology and different sizes of rectal tumors, a multi-scale fusion channel attention mechanism is designed, which can capture important contextual information of different scales. Finally, visualize the 3D rectal tumor segmentation results.

The RTAU-Net is evaluated on the data set provided by Shanxi Provincial Cancer Hospital and Xinhua Hospital. The experimental results showed that the Dice of tumor segmentation reached 0.7978 and 0.6792, respectively, which improved by 2.78 % and 7.02 % compared with suboptimal model.

Although the morphology of rectal tumors varies, RTAU-Net can precisely localize rectal tumors and learn the contour and details of tumors, which can relieve physicians' workload and improve diagnostic accuracy.

In some surgical disciplines, navigation-assisted surgery has become standard of care, but in rectal cancer, indications for navigation and the utility of different technologies remain undetermined.

The NAVI-LARRC prospective study (NCT04512937; IDEAL Stage 2a) evaluated feasibility of navigation in patients with locally advanced primary (LARC) and recurrent rectal cancer (LRRC). Included patients had advanced tumours with high risk of incomplete (R1/R2) resection, and navigation was considered likely to improve the probability of complete resection (R0). Tumours were classified according to pelvic compartmental involvement, as suggested by the Royal Marsden group. The BrainlabTM navigation platform was used for preoperative segmentation of tumour and pelvic anatomy, and for intraoperative navigation with optical tracking. R0 resection rates, surgeons' experiences, and adherence to the preoperative resection plan were assessed.

Seventeen patients with tumours involving the posterior/lateral compartments underwent navigation-assisted procedures. Fifteen patients required abdominosacral resection, and 3 had resection of the sciatic nerve. R0 resection was obtained in 6/8 (75%) LARC and 6/9 (69%) LRRC cases. Preoperative segmentation was time-consuming (median 3.5 h), but intraoperative navigation was accurate. Surgeons reported navigation to be feasible, and adherence to the resection plan was satisfactory.

Navigation-assisted surgery using optical tracking was feasible. The preoperative planning was time-consuming, but intraoperative navigation was accurate and resulted in acceptable R0 resection rates. Selected patients are likely to benefit from navigation-assisted surgery.

Existing methods for quality assurance of the radiotherapy auto-segmentations focus on the correlation between the average model entropy and the Dice Similarity Coefficient (DSC) only. We identified a metric directly derived from the output of the network and correlated it with clinically relevant metrics for contour accuracy.

Magnetic Resonance Imaging auto-segmentations were available for the gross tumor volume for cervical cancer brachytherapy (106 segmentations) and for the clinical target volume for rectal cancer external-beam radiotherapy (77 segmentations). The nnU-Net's output before binarization was taken as a score map. We defined a metric as the mean of the voxels in the score map above a threshold (λ). Comparisons were made with the mean and standard deviation over the score map and with the mean over the entropy map. The DSC, the 95th Hausdorff distance, the mean surface distance (MSD) and the surface DSC were computed for segmentation quality. Correlations between the studied metrics and model quality were assessed with the Pearson correlation coefficient (r). The area under the curve (AUC) was determined for detecting segmentations that require reviewing.

For both tasks, our metric (λ = 0.30) correlated more strongly with the segmentation quality than the mean over the entropy map (for surface DSC, r > 0.65 vs. r < 0.60). The AUC was above 0.84 for detecting MSD values above 2 mm.

Our metric correlated strongly with clinically relevant segmentation metrics and detected segmentations that required reviewing, indicating its potential for automatic quality assurance of radiotherapy target auto-segmentations.

Diffusion weighted magnetic resonance imaging (DW-MRI) can be prognostic for response to neoadjuvant chemotherapy (nCRT) in patients with esophageal cancer. However, manual tumor delineation is labor intensive and subjective. Furthermore, noise in DW-MRI images will propagate into the corresponding apparent diffusion coefficient (ADC) signal. In this study a workflow is investigated that combines a denoising algorithm with semi-automatic segmentation for quantifying ADC changes.

Twenty patients with esophageal cancer who underwent nCRT before esophagectomy were included. One baseline and five weekly DW-MRI scans were acquired for every patient during nCRT. A self-supervised learning denoising algorithm, Patch2Self, was used to denoise the DWI-MRI images. A semi-automatic delineation workflow (SADW) was next developed and compared with a manually adjusted workflow (MAW). The agreement between workflows was determined using the Dice coefficients and Brand Altman plots. The prognostic value of ADCmean increases (%/week) for pathologic complete response (pCR) was assessed using c-statistics.

The median Dice coefficient between the SADW and MAW was 0.64 (interquartile range 0.20). For the MAW, the c-statistic for predicting pCR was 0.80 (95% confidence interval (CI):0.56-1.00). The SADW showed a c-statistic of 0.84 (95%CI:0.63-1.00) after denoising. No statistically significant differences in c-statistics were observed between the workflows or after applying denoising.

The SADW resulted in non-inferior prognostic value for pCR compared to the more laborious MAW, allowing broad scale applications. The effect of denoising on the prognostic value for pCR needs to be investigated in larger cohorts.

To assess the performance of convolutional neural networks (CNNs) for semiautomated segmentation of hepatocellular carcinoma (HCC) tumors on MRI.

This retrospective single-center study included 292 patients (237 M/55F, mean age 61 years) with pathologically confirmed HCC between 08/2015 and 06/2019 and who underwent MRI before surgery. The dataset was randomly divided into training (n = 195), validation (n = 66), and test sets (n = 31). Volumes of interest (VOIs) were manually placed on index lesions by 3 independent radiologists on different sequences (T2-weighted imaging [WI], T1WI pre-and post-contrast on arterial [AP], portal venous [PVP], delayed [DP, 3 min post-contrast] and hepatobiliary phases [HBP, when using gadoxetate], and diffusion-weighted imaging [DWI]). Manual segmentation was used as ground truth to train and validate a CNN-based pipeline. For semiautomated segmentation of tumors, we selected a random pixel inside the VOI, and the CNN provided two outputs: single slice and volumetric outputs. Segmentation performance and inter-observer agreement were analyzed using the 3D Dice similarity coefficient (DSC).

A total of 261 HCCs were segmented on the training/validation sets, and 31 on the test set. The median lesion size was 3.0 cm (IQR 2.0-5.2 cm). Mean DSC (test set) varied depending on the MRI sequence with a range between 0.442 (ADC) and 0.778 (high b-value DWI) for single-slice segmentation; and between 0.305 (ADC) and 0.667 (T1WI pre) for volumetric-segmentation. Comparison between the two models showed better performance in single-slice segmentation, with statistical significance on T2WI, T1WI-PVP, DWI, and ADC. Inter-observer reproducibility of segmentation analysis showed a mean DSC of 0.71 in lesions between 1 and 2 cm, 0.85 in lesions between 2 and 5 cm, and 0.82 in lesions > 5 cm.

CNN models have fair to good performance for semiautomated HCC segmentation, depending on the sequence and tumor size, with better performance for the single-slice approach. Refinement of volumetric approaches is needed in future studies.

• Semiautomated single-slice and volumetric segmentation using convolutional neural networks (CNNs) models provided fair to good performance for hepatocellular carcinoma segmentation on MRI. • CNN models' performance for HCC segmentation accuracy depends on the MRI sequence and tumor size, with the best results on diffusion-weighted imaging and T1-weighted imaging pre-contrast, and for larger lesions.

To use convolutional neural network for fully automated segmentation and radiomics features extraction of hypopharyngeal cancer (HPC) tumor in MRI.

MR images were collected from 222 HPC patients, among them 178 patients were used for training, and another 44 patients were recruited for testing. U-Net and DeepLab V3 + architectures were used for training the models. The model performance was evaluated using the dice similarity coefficient (DSC), Jaccard index, and average surface distance. The reliability of radiomics parameters of the tumor extracted by the models was assessed using intraclass correlation coefficient (ICC).

The predicted tumor volumes by DeepLab V3 + model and U-Net model were highly correlated with those delineated manually (p < 0.001). The DSC of DeepLab V3 + model was significantly higher than that of U-Net model (0.77 vs 0.75, p < 0.05), particularly in those small tumor volumes of < 10 cm3 (0.74 vs 0.70, p < 0.001). For radiomics extraction of the first-order features, both models exhibited high agreement (ICC: 0.71-0.91) with manual delineation. The radiomics extracted by DeepLab V3 + model had significantly higher ICCs than those extracted by U-Net model for 7 of 19 first-order features and for 8 of 17 shape-based features (p < 0.05).

Both DeepLab V3 + and U-Net models produced reasonable results in automated segmentation and radiomic features extraction of HPC on MR images, whereas DeepLab V3 + had a better performance than U-Net.

The deep learning model, DeepLab V3 + , exhibited promising performance in automated tumor segmentation and radiomics extraction for hypopharyngeal cancer on MRI. This approach holds great potential for enhancing the radiotherapy workflow and facilitating prediction of treatment outcomes.

• DeepLab V3 + and U-Net models produced reasonable results in automated segmentation and radiomic features extraction of HPC on MR images. • DeepLab V3 + model was more accurate than U-Net in automated segmentation, especially on small tumors. • DeepLab V3 + exhibited higher agreement for about half of the first-order and shape-based radiomics features than U-Net.

With the recent advancements in rapid imaging methods, higher numbers of contrasts and quantitative parameters can be acquired in less and less time. Some acquisition models simultaneously obtain multiparametric images and quantitative maps to reduce scan times and avoid potential issues associated with the registration of different images. Multiparametric magnetic resonance imaging (MRI) has the potential to provide complementary information on a target lesion and thus overcome the limitations of individual techniques. In this review, we introduce methods to acquire multiparametric MRI data in a clinically feasible scan time with a particular focus on simultaneous acquisition techniques, and we discuss how multiparametric MRI data can be analyzed as a whole rather than each parameter separately. Such data analysis approaches include clinical scoring systems, machine learning, radiomics, and deep learning. Other techniques combine multiple images to create new quantitative maps associated with meaningful aspects of human biology. They include the magnetic resonance g-ratio, the inner to the outer diameter of a nerve fiber, and the aerobic glycolytic index, which captures the metabolic status of tumor tissues.

Breast cancer is the most common and most commonly diagnosed non-skin cancer in women. There are several risk factors related to habits and heredity, and screening is essential to reduce the incidence of mortality. Thanks to screening and increased awareness among women, most breast cancers are diagnosed at an early stage, increasing the chances of cure and survival. Regular screening is essential. Mammography is currently the gold standard for breast cancer diagnosis. In mammography, we can encounter problems with the sensitivity of the instrument; in fact, in the case of a high density of glands, the ability to detect small masses is reduced. In fact, in some cases, the lesion may not be particularly evident, it may be hidden, and it is possible to incur false negatives as partial details that may escape the radiologist's eye. The problem is, therefore, substantial, and it makes sense to look for techniques that can increase the quality of diagnosis. In recent years, innovative techniques based on artificial intelligence have been used in this regard, which are able to see where the human eye cannot reach. In this paper, we can see the application of radiomics in mammography.

The aim of this study was to improve the accuracy of the clinical target volume (CTV) and organs at risk (OARs) segmentation for rectal cancer preoperative radiotherapy.

Computed tomography (CT) scans from 265 rectal cancer patients treated at our institution were collected to train and validate automatic contouring models. The regions of CTV and OARs were delineated by experienced radiologists as the ground truth. We improved the conventional U-Net and proposed Flex U-Net, which used a register model to correct the noise caused by manual annotation, thus refining the performance of the automatic segmentation model. Then, we compared its performance with that of U-Net and V-Net. The Dice similarity coefficient (DSC), Hausdorff distance (HD), and average symmetric surface distance (ASSD) were calculated for quantitative evaluation purposes. With a Wilcoxon signed-rank test, we found that the differences between our method and the baseline were statistically significant (P< 0.05).

Our proposed framework achieved DSC values of 0.817 ± 0.071, 0.930 ± 0.076, 0.927 ± 0.03, and 0.925 ± 0.03 for CTV, the bladder, Femur head-L and Femur head-R, respectively. Conversely, the baseline results were 0.803 ± 0.082, 0.917 ± 0.105, 0.923 ± 0.03 and 0.917 ± 0.03, respectively.

In conclusion, our proposed Flex U-Net can enable satisfactory CTV and OAR segmentation for rectal cancer and yield superior performance compared to conventional methods. This method provides an automatic, fast and consistent solution for CTV and OAR segmentation and exhibits potential to be widely applied for radiation therapy planning for a variety of cancers.

For locally advanced rectal cancers, in vivo radiological evaluation of tumor extent and regression after neoadjuvant therapy involves implicit visual identification of rectal structures on magnetic resonance imaging (MRI). Additionally, newer image-based, computational approaches (e.g., radiomics) require more detailed and precise annotations of regions such as the outer rectal wall, lumen, and perirectal fat. Manual annotations of these regions, however, are highly laborious and time-consuming as well as subject to inter-reader variability due to tissue boundaries being obscured by treatment-related changes (e.g., fibrosis, edema).

This study presents the application of U-Net deep learning models that have been uniquely developed with region-specific context to automatically segment each of the outer rectal wall, lumen, and perirectal fat regions on post-treatment, T2-weighted MRI scans.

In multi-institutional evaluation, region-specific U-Nets (wall Dice = 0.920, lumen Dice = 0.895) were found to perform comparably to multiple readers (wall inter-reader Dice = 0.946, lumen inter-reader Dice = 0.873). Additionally, when compared to a multi-class U-Net, region-specific U-Nets yielded an average 20% improvement in Dice scores for segmenting each of the wall, lumen, and fat; even when tested on T2-weighted MRI scans that exhibited poorer image quality, or from a different plane, or were accrued from an external institution.

Developing deep learning segmentation models with region-specific context may thus enable highly accurate, detailed annotations for multiple rectal structures on post-chemoradiation T2-weighted MRI scans, which is critical for improving evaluation of tumor extent in vivo and building accurate image-based analytic tools for rectal cancers.

Accurate measurement of the distance from the tumor's lowest boundary to the anal verge (DTAV) provides an important reference value for treatment of rectal cancer, but the standard measurement method (colonoscopy) causes substantial pain. Therefore, we propose a method for automatically measuring the DTAV on sagittal magnetic resonance (MR) images. We designed a boundary-guided transformer that can accurately segment the rectum and tumor. From the segmentation results, we estimated the DTAV by automatically extracting the anterior rectal wall from the tumor's lowest point to the anal verge and then calculating its physical length. Experiments were conducted on a rectal tumor MR imaging (MRI) dataset to evaluate the efficacy of our method. The results showed that our method outperformed surgeons with 6 years of experience (p < 0.001). Furthermore, by referring to our segmentation results, attending and resident surgeons could improve their measurement precision and efficiency.

Recently, radiomics has been widely used in colorectal cancer, but many variable factors affect the repeatability of radiomics research. This review aims to analyze the repeatability of radiomics studies in colorectal cancer and to evaluate the current status of radiomics in the field of colorectal cancer.

The included studies in this review by searching from the PubMed and Embase databases. Then each study in our review was evaluated using the Radiomics Quality Score (RQS). We analyzed the factors that may affect the repeatability in the radiomics workflow and discussed the repeatability of the included studies.

A total of 188 studies was included in this review, of which only two (2/188, 1.06%) studies controlled the influence of individual factors. In addition, the median score of RQS was 11 (out of 36), range-1 to 27.

The RQS score was moderately low, and most studies did not consider the repeatability of radiomics features, especially in terms of Intra-individual, scanners, and scanning parameters. To improve the generalization of the radiomics model, it is necessary to further control the variable factors of repeatability.

In the paradigm shift related to rectal cancer treatment, we have to understand a variety of new emerging topics to provide appropriate treatment for individual patients as precision medicine. However, information on surgery, genomic medicine, and pharmacotherapy is highly specialized and subdivided, creating a barrier to achieving thorough knowledge. In this review, we summarize the perspective for rectal cancer treatment and management from the current standard-of-care to the latest findings to help optimize treatment strategy.

Segmentation of rectal cancerous regions from Magnetic Resonance (MR) images can help doctor define the extent of the rectal cancer and judge the severity of rectal cancer, so rectal tumor segmentation is crucial to improve the accuracy of rectal cancer diagnosis. However, accurate segmentation of rectal cancerous regions remains a challenging task due to the shape of rectal tumor has significant variations and the tumor and surrounding tissue are indistinguishable. In addition, in the early research on rectal tumor segmentation, most deep learning methods were based on convolutional neural networks (CNNs), and traditional CNN have small receptive field, which can only capture local information while ignoring the global information of the image. Nevertheless, the global information plays a crucial role in rectal tumor segmentation, so traditional CNN-based methods usually cannot achieve satisfactory segmentation results. In this paper, we propose an encoder-decoder network named Dual Parallel Net (DuPNet), which fuses transformer and classical CNN for capturing both global and local information. Meanwhile, as for capture features at different scales as well as to avoid accuracy loss and parameters reduction, we design a feature adaptive block (FAB) in skip connection between encoder and decoder. Furthermore, in order to utilize the apriori information of rectal tumor shape effectively, we design a Gaussian Mixture prior and embed it in self-attention mechanism of the transformer, leading to robust feature representation and accurate segmentation results. We have performed extensive ablation experiments to verify the effectiveness of our proposed dual parallel encoder, FAB and Gaussian Mixture prior on the dataset from the Shanxi Cancer Hospital. In the experimental comparison with the state-of-the-art methods, our method achieved a Mean Intersection over Union (MIoU) of 89.34% on the test set. In addition to that, we evaluated the generalizability of our method on the dataset from Xinhua Hospital, the promising results verify the superiority of our method.

Manual segmentation of the inferior alveolar canal (IAC) in panoramic images requires considerable time and labor even for dental experts having extensive experience. The objective of this study was to evaluate the performance of automatic segmentation of IAC with ambiguity classification in panoramic images using a deep learning method.

Among 1366 panoramic images, 1000 were selected as the training dataset and the remaining 336 were assigned to the testing dataset. The radiologists divided the testing dataset into four groups according to the quality of the visible segments of IAC. The segmentation time, dice similarity coefficient (DSC), precision, and recall rate were calculated to evaluate the efficiency and segmentation performance of deep learning-based automatic segmentation.

Automatic segmentation achieved a DSC of 85.7% (95% confidence interval [CI] 75.4%-90.3%), precision of 84.1% (95% CI 78.4%-89.3%), and recall of 87.7% (95% CI 77.7%-93.4%). Compared with manual annotation (5.9s per image), automatic segmentation significantly increased the efficiency of IAC segmentation (33 ms per image). The DSC and precision values of group 4 (most visible) were significantly better than those of group 1 (least visible). The recall values of groups 3 and 4 were significantly better than those of group 1.

The deep learning-based method achieved high performance for IAC segmentation in panoramic images under different visibilities and was positively correlated with IAC image clarity.