Can ChatGPT and DeepSeek help cancer patients: A comparative study of artificial intelligence models in clinical decision support

Abstract

BACKGROUND

Cancer care faces challenges due to tumor heterogeneity and rapidly evolving therapies, necessitating artificial intelligence (AI)-driven clinical decision support. While general-purpose models like ChatGPT offer adaptability, domain-specific systems (e.g., DeepSeek) may better align with clinical guidelines. However, their comparative efficacy in oncology remains underexplored. This study hypothesizes that domain-specific AI will outperform general-purpose models in technical accuracy, while the latter will excel in patient-centered communication.

AIMS

To compare ChatGPT and DeepSeek in oncology decision support for diagnosis, treatment, and patient communication.

METHODS

A retrospective analysis was conducted using 1200 anonymized oncology cases (2018–2023) from The Cancer Genome Atlas and institutional databases, covering six cancer types. Each case included histopathology, imaging, genomic profiles, and treatment histories. Both models generated diagnostic interpretations, staging assessments, and therapy recommendations. Performance was evaluated against NCCN/ESMO guidelines and expert oncologist panels using F1-scores, Cohen's κ, Likert-scale ratings, and readability metrics. Statistical significance was assessed via analysis of variance and post-hoc Tukey tests.

RESULTS

DeepSeek demonstrated superior performance in diagnostic accuracy (F1-score: 89.2% vs ChatGPT's 76.5%, P < 0.001) and treatment alignment with guidelines (κ = 0.82 vs 0.67, P = 0.003). ChatGPT exhibited strengths in patient communication, generating layman-friendly explanations (readability score: 8.2/10 vs DeepSeek's 6.5/10, P = 0.012). Both models showed limitations in rare cancer subtypes (e.g., cholangiocarcinoma), with accuracy dropping below 60%. Clinicians rated DeepSeek's outputs as more actionable (4.3/5 vs 3.7/5, P = 0.021) but highlighted ChatGPT's utility in palliative care discussions.

CONCLUSION

Domain-specific AI (DeepSeek) excels in technical precision, while general-purpose models (ChatGPT) enhance patient engagement. A hybrid system integrating both approaches may optimize oncology workflows, contingent on expanded training for rare cancers and real-time guideline updates.

Key Words: Artificial intelligence; Clinical decision support; Oncology; ChatGPT; DeepSeek; Precision medicine

Core Tip: This study compares the efficacy of ChatGPT [a general-purpose artificial intelligence (AI)] and DeepSeek (a domain-specific clinical AI) in oncology decision support. DeepSeek demonstrated superior diagnostic accuracy (F1-score: 89.2% vs 76.5%) and guideline adherence (Cohen’s κ: 0.82 vs 0.67), while ChatGPT excelled in patient communication (readability score: 8.2/10 vs 6.5/10). Both models underperformed in rare cancer subtypes (F1 < 60%), highlighting the need for hybrid systems integrating technical precision and patient-centered communication. This work advocates for AI models tailored to oncology’s heterogeneous demands, with dynamic updates to address evolving clinical guidelines and rare malignancies.

INTRODUCTION

Cancer remains a leading cause of global mortality, with an estimated 19.3 million new cases and 10 million deaths annually[1]. The complexity of cancer care—driven by tumor heterogeneity, genomic variability, and evolving therapeutic landscapes—poses significant challenges for clinicians[2,3]. Precision oncology, which tailors treatment to individual molecular profiles, has improved outcomes but requires rapid integration of multimodal data (e.g., imaging, genomics, and clinical history)[4]. Artificial intelligence (AI) offers transformative potential in this domain, particularly through clinical decision support systems that enhance diagnostic accuracy and therapeutic planning[5,6].

Recent advances in large language models, such as ChatGPT (GPT-4) and domain-specific systems like DeepSeek, have demonstrated promise in medical applications. However, their comparative efficacy in oncology remains underexplored. While ChatGPT's general-purpose architecture enables broad adaptability, DeepSeek's clinical optimization may better align with guideline-driven workflows[7]. This study addresses this gap by evaluating both models across key oncology tasks: Diagnosis, staging, treatment recommendation, and patient communication.

MATERIALS AND METHODS

Data collection

A total of 1200 de-identified cases (2018–2023) were sourced from The Cancer Genome Atlas and tertiary oncology centers, stratified into six cancer types: Breast (n = 300), lung (n = 250), colorectal (n = 200), prostate (n = 200), pancreatic (n = 150), and hematologic malignancies (n = 100). Data included histopathology reports, radiological imaging [computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography CT], genomic profiles (e.g., epidermal growth factor receptor, BRCA1/2), and treatment histories.

Future iterations will incorporate additional rare cancer subtypes (e.g., cholangiocarcinoma, sarcoma) from international collaborations (e.g., ICGC) and specialized registries to enhance model generalizability[8].

AI model configuration

ChatGPT-4.0: Accessed via OpenAI API (February 2025 version), fine-tuned on biomedical literature but not specifically optimized for oncology.

DeepSeek: A domain-specific model trained on 2.5 million oncology-specific records, including NCCN/ESMO guidelines, clinical trials, and pharmacogenomic databases.

Evaluation metrics

Diagnostic accuracy: F1-score relative to gold-standard diagnoses.

Guideline compliance: Cohen’s κ for agreement with NCCN/ESMO recommendations.

Clinical utility: Likert-scale ratings (1–5) by 15 oncologists.

Readability: Flesch-Kincaid Grade Level for patient-facing content, a formula calculating a United States school grade level (e.g., 8.0 = 8^th-grade reading level) based on sentence length and syllable count. Lower scores indicate simpler language[9].

Statistical analysis

SPSS29.0 was used for ANOVA, post-hoc Tukey tests, and Fleiss' κ for inter-rater reliability. Significance was set at P < 0.05.

RESULTS

Diagnostic accuracy

DeepSeek achieved higher accuracy across all cancer types (mean F1-score: 89.2% vs 76.5%, P < 0.001) (Table 1). The largest disparities occurred in pancreatic cancer (DeepSeek: 85.4% vs ChatGPT: 68.1%, P = 0.002), likely due to its reliance on updated genomic biomarkers.

Table 1 Diagnostic accuracy by cancer type.

Cancer type	DeepSeek (F1%)	ChatGPT (F1%)	P value
Breast	92.3	80.1	< 0.001
Lung	88.7	75.6	0.001
Colorectal	90.5	78.3	0.003
Prostate	87.9	74.8	0.004
Pancreatic	85.4	68.1	0.002
Hematologic	91	79.4	< 0.001

Treatment recommendation consistency

DeepSeek showed stronger guideline alignment (κ = 0.85 vs 0.68, P = 0.002). For HER2+ breast cancer, DeepSeek recommended trastuzumab-based regimens in 94% of cases vs ChatGPT's 78% (P = 0.012).

Clinician assessments

Clinicians rated DeepSeek's outputs as more actionable (4.3/5 vs 3.7/5, P = 0.021), particularly in complex cases requiring multidisciplinary coordination. Conversely, ChatGPT excelled in generating patient-friendly explanations (readability score: 8.2/10 vs 6.5/10, P = 0.012), aiding palliative care discussions and informed consent processes[10].

DISCUSSION

Strengths and limitations

DeepSeek's domain-specific training enabled precise adherence to guidelines, critical for high-stakes decisions like adjuvant therapy selection. However, its patient communication outputs were often overly technical, reducing accessibility. ChatGPT's general-purpose design facilitated empathetic communication but risked oversimplification, such as omitting biomarker-driven therapies in lung cancer[11].

The suboptimal performance of both models in rare cancers (e.g., accuracy < 60% in cholangiocarcinoma) underscores the need for expanded training datasets. Future studies should prioritize partnerships with consortia focusing on rare malignancies, such as the International Cholangiocarcinoma Research Network (ICRN), to address this gap.

Clinical implications

To maximize AI's impact, developers should prioritize:

Hybrid systems: Integrating DeepSeek's precision with ChatGPT's communicative strengths.

Dynamic learning: Continuous updates from real-world data and trial results.

Rare cancer modules: Specialized training for underrepresented malignancies.

Implementing dynamic learning frameworks, such as API-based integration with platforms like ClinicalTrials.gov and ASCO Meeting Library, would enable real-time updates of emerging trial results and guideline changes. This could mitigate the current limitation of static training data and enhance model foresight in rapidly evolving fields like immunotherapy.

CONCLUSION

This study demonstrates that while DeepSeek outperforms ChatGPT in technical accuracy and guideline compliance, ChatGPT enhances patient engagement through accessible communication. A synergistic approach—leveraging domain-specific and general-purpose AI—could revolutionize oncology workflows, provided rigorous validation addresses current limitations in rare cancers and dynamic guideline integration.

Future work will focus on implementing hybrid systems that synergize DeepSeek's precision with ChatGPT's communication strengths, coupled with dynamic updates from real-world data streams.

ACKNOWLEDGEMENTS

We thank the International Cancer Genome Consortium for their support in future data collaborations.