Repairing the human with artificial intelligence in oncology

Abstract

Artificial intelligence is a groundbreaking tool to learn and analyse higher features extracted from any dataset at large scale. This ability makes it ideal to facing any complex problem that may generally arise in the biomedical domain or oncology in particular. In this work, we envisage to provide a global vision of this mathematical discipline outgrowth by linking some other related subdomains such as transfer, reinforcement or federated learning. Complementary, we also introduce the recently popular method of topological data analysis that improves the performance of learning models.

Key Words: Cancer research; Data analysis; Feature classification; Artificial intelligence; Machine learning; Healthcare systems

Core Tip: In this review, we explore powerful artificial intelligence based models enabling the comprehensive analysis of related problems on oncology. To this end, we described an asserted set of machine learning architectures that goes from the most classical multiple perceptron or neural networks to the novel federated and reinforcement learning designs. Overall, we point out the outgrowth of this mathematical discipline in cancer research and how computational biology and topological features can boost the general performances of these learning models.

INTRODUCTION

The flourishing proliferation of artificial intelligence (AI) worldwide over the last decade has disrupted the way oncologists face cancer. More and more every day, the contribution of AI-based models to different axes of cancer research is not only improving their ability to stratify patients early on or discover new drugs but also influences its fundamentals. By integrating novel structures of data organisation, exploitation, and sharing of clinical data among health institutions, AI is achieving in the short-term to successfully accelerate cancer research. Medical practitioners are becoming familiar with some few mathematical concepts, such as machine learning (ML) or (un/semi) supervised learning. The former is a collection of data-driven techniques with the goal of building predictive models from high-dimensional datasets[1,2], while the latter refers to the grade of human intervention that these models require to make predictions.

These methods are being successfully used in cancer at many levels by simply analysing clinical data, biological indicators, or whole slide images[3-5]. Their application has revealed themselves as an effective way to tackle multiple clinical questions, from diagnosis to prediction of treatment outcomes. For instance, in Morilla et al[3], a minimal signature composed of seven miRNAs and two biological indicators was identified using general linear models trained at the base of a deep learning model to predict treatment outcomes in gastrointestinal cancer. In Schmauch et al[4], 2020, the authors predicted the RNA-Seq expression of tumours from whole slide images using a deep learning model as well.

Indeed, in this particular discipline, ML algorithms have evolved faster. Several approaches have succeeded in the classification of cancer subtypes using medical imaging[6-8]. Mammography and digital breast tomosynthesis have enabled a robust breast cancer detection by means of annotation-efficient deep learning approaches[9]. Epigenetic patterns of chromatin opening across the stem and differentiated cells across the immune system have also been predicted by deep neural networks in ATAC-seq analysis. In Maslova et al[10], solely from the DNA sequence of regulatory regions, the authors discovered ab initio binding motifs for known and unknown master regulators, along with their combinatorial operation.

Another domain where the application of AI-based models has largely been used is single-cell RNA sequencing (sc-RNAseq) analysis. In Lotfollahi et al[11] (2020), a new method based on transfer learning (TL) and parameter optimisation is introduced to enable efficient, decentralised, iterative reference building, and the contextualization of new datasets with existing single-cell references without sharing raw data. In addition, few methods have emerged around genetic perturbations of outcomes at the single-cell level in cancer treatments[12,13].

Finally, some computational topology techniques grouped under the heading of “topological data analysis” (TDA) have also been successfully proven as efficient tools in some cancer subtype classifications[14].

Thus, AI has turned the oncologists and co-workers’ lives around providing them with a new perspective, which was once developed by only a bulk of specialists and is rapidly becoming a reference in the domain. This work revisits, then, most of those techniques and provides a quick overview of their applications in cancer research.

AI OR ML

ML or AI models, sometimes a philosophical matter, is a branch of mathematics concerned by numerically mimicking the human brain reasoning as it resolves a given problem. There are many examples of this practice; from those most classic techniques of regression or classification of dataset[15] to the current ground-breaking algorithms as “Deep-Mind, Alpha Fold” for protein-folding prediction[16]. In any case, all of these methods share a common objective: the ML problem. This problem can be mathematically expressed as: $$\hat{C}=\underset{C\in\mathcal{M}}{argmin}\math-bb{E}_{x,y\in\mathcal{X}\times\mathcal{Y}} [\mathcal{B}_{l} (C(x),y)]$$.

For example, if we select the particular loss function binary cross entropy, –B_l–, this equation describes the parameter misapplication of the neural network C by diminishing the expected value of the loss function between the output of this network C(x) and the true label y.

INTERPRETABLE AI MODELS

Frequently, the intricate design of models based on any ML technique (i.e., neural networks) makes them more difficult to interpret than simpler traditional models. Hence, if we want to fully exploit the potential of these models, a deeper understanding of their predictions would be advisable in practice. Thus, the predicted efficacy of a personal therapy on a cancer must be well explained, since its decisions directly influence human health. From a methodological point of view, we need to ensure model development with proper interpretations of their partial outputs in order to prevent undesirable effects of the models[17,18]. The two main streams of this discipline are the so-called “feature attribution” and “feature interaction” methods. The former[19-22] individually rewards input features depending on its local causal effect in the model output, whereas the latter examines those features with large second-order derivatives at the input or weight matrices of feed-forward and convolutional architectures[23,24]. However, the robustness of all these approaches may be compromised by the presence of specific types of architecture.

DEEP LEARNING

One class of ML models broadly used in current computational cancer research is deep neural networks. Overall, they have succeeded over other non-linear models[25] in the analysis of pathologic image recognition and later patient stratification based on the learned models[26,27]. In brief, deep neural models work in a large number of layers of information that is progressively passing by from one layer to another (i.e., the backpropagation algorithms) to extract relevant features from the original data according to a non-linear model, which is associated with the selected optimisation problem. Their designs can encompass a wide range of algorithms from the classic multiple perceptron networks[28-30] and convolutional neural networks[31-36] to the most recently established long short-term memory (LSTM) recurrent neural networks (RNNs) that are put into the spotlight in the next section[37,38].

RNNs: A different and convenient design other than the more classical neural networks in which the information flows forward are the RNNs. These are computationally more complex models with the skill of capturing hidden behaviours other methods in cancer studies cannot do[39-41]. Recurrent models exhibit an intrinsic representation of the data that allows the exploitation of context information. Specifically, a recurrent network is designed to maintain information about earlier iterations for a period that depends only on the weights and input data at the model’s entrance[42]. In particular, the network’s activation layers take advantage of inputs that come from chains of information provided by previous iterations. This influences the current prediction and enables the gathering of network flops that can retain contextual information on a long-term scale. Thus, by following this reasoning, RNNs can dynamically exploit a contextual interval over the input training history[43].

LSTM: An improvement in of RNNs is the construction of LSTM networks. LSTMs can learn to sort the interexchange between dependencies in the predictive problems addressed by batches. These models have had a major impact on the biomedical domain, particularly in cancer research[44-48]. LSTMs have been successfully proven in analysis where the intrinsic technical drawbacks associated with RNNs have prevented a fair performance of the model[49]. There are two main optimisation problems that must be avoided during the training stage when applying LSTM to solve a problem, namely: (1) vanishing gradients; and (2) exploding gradients[50]. In this sense, LSTM specifically provides an inner structural amelioration concerning the units leveraged in the learning model[51]. However, there is an improvement in the LSTM network calibration that is increasingly used in biomedical research: LSTM bidirectional networks. In these architectures, a bidirectional recurrent neural lattice is applied in order to be able to separately pass by two forward and backward recurrent nets sharing the same output layer during the training task[51].

TL

Recycling is always a significant issue! In ML, we can also reuse a model that was originally envisaged for solving a different task other than the problem that we might be currently facing, but both share a similar structural behaviour. This practice is called TL in ML. Its usage has been progressively increasing in problems whose architecture can consume huge amounts of time and computational resources. In these cases, pre-trained networks are applied as a starting learning point, which largely boosts the performance of new models to approach related problems. Then, TL should ameliorate the current model in another setting if such a model is available for learning features from the first problem in a general way[52,53]. Regarding its benefits in oncology, we can outstand its usage in large datasets of piled images to be recognised for patient stratification, as previously described in the following works[54-61].

REINFORCEMENT LEARNING

Reinforcement learning (RL) is one of the latest ML extensions that ameliorates the global performance of learning models when making decisions. In RL, a model learns a given objective in an a priori fixed uncertainty by means of trial and error computations until a solution is obtained. Then, to guide the model, the AI algorithm associates rewards or penalties with the local performance of the model. The final goal was to maximise the amount of rewards obtained. Remarkably, the ML architecture provides no clues on how to find the final solution, even if it rules the reward conditions. Thus, the model must smooth the optimisation problem from a totally random scenario to a complex universe of possibilities. However, if the learning algorithm is launched into a sufficiently powerful computational environment, the ML model will be able to store thousands of trials to effectively achieve the given goal. Nevertheless, a major inconvenience is that the simulation environment is highly dependent on the problem to be computed.

To sum it up, although RL should not be taken as the definitive algorithm, it promises to blow up the current concept of deep learning in oncology[62-64]. An example with no precedents is the DeepMind algorithm very famous nowadays by performing alpha protein folding[16] predictions at a scale ever done before.

FEDERATED LEARNING

A simple description of federated learning (FL) could be a decentralised approach to ML. Thus, FL boosts and accelerates medical discoveries on partnerships with many contributors while protecting patient privacy. In FL, we only improve and calibrate the results and not the data. Thus, what FL really promises it is a new era in secured AI in oncology: Training, testing, or ensuring privacy that way of learning is an efficient method of using data from a comprehensive network of resources belonging each time to a node of many interconnected hospital institutions[65-68].

TOPOLOGICAL ML

Topological ML (TML) is an interaction that has been recently established between TDA and ML. Owing to new advances in computational algorithms, the extraction of complex topological features, such as persistence homology or Betti curves, has become progressively feasible in large datasets. In particular, TDA is commonly referred to as capturing the shape of the data. This method fixes their topological invariants as hotspot to look up relevant structural and categorical information. Indeed, TDA provides ideal completeness in terms of multi-scalability and globalisation missed from the rigidness of their geometric characteristics. In that sense, the use of this tool has been growing in cancer research until it is considered as contextually informative in the analysis of massive biomedical data[69-74]. Multiple studies have exploited the complementary information that emerged from different prisms to gain new insights into the datasets. Its association with ML has enhanced both classical ML methods and deep learning models[75,76].

CONCLUSION

In this work, we summarise the conclusions of some major references of AI in cancer research (Figure 1). Overall, we wanted to point out the rapid AI outgrowth in the biomedical domain and how AI has systematically become familiar to anyone in the domain, expert, or not. This is possibly due to recent advances in learning-oriented algorithms, which have enabled the transformation of data analysis to any scale and complexity provided a suitable environment is available. We have provided many examples of a varied set of learning models (Multi-layer perceptron, convolutional neural networks, RNNs, etc.) that have been successfully proven for related cancer problems such as patient stratification, image-based classification, or recording-device optimisation[77,78]. We have compared different approaches to solve similar questions, and we have introduced novel concepts such as TL, FL, or RL that prevent some of the most classical constraints regarding network architectures or information privacy on high dimensional datasets. Finally, the combination of TDA and ML has also been shown to be a promising discipline where to exploit extra topological features extracted at a higher level. Such tandem promises to contribute to the improvement of the AI algorithm’s performance from a totally different perspective. Although data-driven based AI models have the potential to change the world of unsupervised learning, some limitations could endanger a promising future. The three major issues that hamper a better optimisation and general performance in AI models are related to: (1) the high dependency of the model on the data scale; (2) choice of a proper computational environment, and (3) practical problems of time or computational cost should be assumed. Thus, the future challenges in this discipline begin by smoothing such obstacles as much as possible, which will ultimately end up with AI as the tool of reference in healthcare institutions for a much broader analysis in oncology.

Figure 1 Relational overview of the artificial intelligence-based models introduced in this work. To solve any given complex problem in cancer research by means of machine learning models we can use many deep layers. Then, depending on the particular structures of data, we can empower the performance of the selected architecture, i.e., multilayer perceptron, convolutional or recurrent networks by adding learning strategies such as transfer, federated or topological learning. These strategies are interchangeable (double banded black arrows). As well, we can directly go directly from the selected architecture to the problem’s solution using reinforcement learning. AI: Artificial intelligence; MLPs: Multi-layer perceptrons; CNNs: Convolutional neural networks; RNNs: Recurrent neural networks.

ACKNOWLEDGEMENTS

We acknowledge the financial support from the Institut National de la Sant e et de la Recherche Medicale (INSERM), and the Investissements d’Avenir programme ANR-10-LABX-0017, Sorbonne Paris Nord, Laboratoire d’excellence INFLAMEX.