Postoperative accurate pain assessment of children and artificial intelligence: A medical hypothesis and planned study

Abstract

Although the pediatric perioperative pain management has been improved in recent years, the valid and reliable pain assessment tool in perioperative period of children remains a challenging task. Pediatric perioperative pain management is intractable not only because children cannot express their emotions accurately and objectively due to their inability to describe physiological characteristics of feeling which are different from those of adults, but also because there is a lack of effective and specific assessment tool for children. In addition, exposure to repeated painful stimuli early in life is known to have short and long-term adverse sequelae. The short-term sequelae can induce a series of neurological, endocrine, cardiovascular system stress related to psychological trauma, while long-term sequelae may alter brain maturation process, which can lead to impair neurodevelopmental, behavioral, and cognitive function. Children’s facial expressions largely reflect the degree of pain, which has led to the developing of a number of pain scoring tools that will help improve the quality of pain management in children if they are continually studied in depth. The artificial intelligence (AI) technology represented by machine learning has reached an unprecedented level in image processing of deep facial models through deep convolutional neural networks, which can effectively identify and systematically analyze various subtle features of children’s facial expressions. Based on the construction of a large database of images of facial expressions in children with perioperative pain, this study proposes to develop and apply automatic facial pain expression recognition software using AI technology. The study aims to improve the postoperative pain management for pediatric population and the short-term and long-term quality of life for pediatric patients after operational event.

Key Words: Pediatric; Perioperative pain; Assessment tool; Facial expression; Machine learning; Artificial intelligence

Core Tip: Valid and reliable pain assessment tools in perioperative period of children remain a challenging task by far. The artificial intelligence (AI) technology has reached an unprecedented level in image processing of deep facial models, which can effectively identify and systematically analyze various features of children’s facial expression. Based on the construction of a large database of images of facial expressions in children, we aim to develop an AI tool for pain assessment in order to improve the management of perioperative pain.

INTRODUCTION

Children are exposed to a high number of painful procedures after surgery, which are related to adverse outcomes[1]. A 10-year multicenter randomized controlled trial of children’s perioperative pain showed that children with perioperative hospitalization underwent an average of 7.5 to 17.3 times pain experiences per day, yet painful stimuli cannot be effectively controlled and relieved[2]. Children with experiences of pain can induce post-traumatic stress disorder and chronic pain that affects patient prognosis and neurological development, leading to impaired brain function and cognitive, motor, and behavioral dysfunction[3-6]. Although many tertiary children’s hospitals and medical centers have established medical teams who are specializing in the management of pediatric patient’s pain, the valid and reliable pain assessment tool for pediatric patients have yet to be standardized[7], which is more prevalent than previously recognized.

Previous guidance from the one of American Academy of Pediatrics and the Canadian Pediatric Society addressed the need to assess pediatric pain, especially during and after therapeutic procedures[8]. Despite progress, pain remains underestimated and undertreated in children and the main obstacles is the lack of available assessment tools for children[9]. The self-assessment scale and the behavioral pain scale (BPS) such as visual analogue score (VAS), numerical rating scale (NRS) and faces, legs, activity, cry, and consolability (FLACC) tools are used in children with different clinical conditions[7,10,11]. VAS is the gold standard and is the most validated tool and the vertical version is more suitable for children older than age 6 years. Child must know how to count and be able to transcribe the intensity of the pain in numbers to use NRS. FLACC scale was recently validated in children age 6 mo to 5 years with acute or procedural pain in different settings which would likely require training[12]. The different pain scales and therapeutic threshold according to age are shown in Table 1[6,9]. Flaws in the design (patient self-evaluation and observer evaluation), subjectivity of scoring tools, and different levels of perception of the implementers of the evaluation result in the existence of often positive and negative bias views on the recommendation of the application of the rating tools[13].

Table 1 Postoperative Pain scales and therapeutic threshold according to age.

Clinical context	Age		Pain scale	Therapeutic threshold
Postoperative pain	Preterm and term newborns	0 to 5 yr	Children and infants postoperative pain scale	4/10
		0 to 7 yr	EVENDOL	4/15
	Infants to children < 6 yr	0 to 5 yr	Children and infants postoperative pain scale	4/10
		1 to 7 mo	Amiel tison scale	5/20
		0 to 7 yr	EVENDOL	4/15
		2 mo to 7 yr	FLACC	3/10
		1 to 6 yr	Child facial coding system	1/4
		1 to 7 yr	Behavioral observational pain scale	2/6
		1 to 5 yr	Toddler preschooler postoperative pain scale	Not communicated
		1 to 4 yr	Pain observation scale for young children	3/7
		8 mo to 13 yr	Objective pain scale	3/10
		2 to 12 yr	Postoperative pain measure for parents	6/15
	Children > 6 yr		VAS	3/10
			NRS	3/10
			VRS	Moderate
			Faces pain scale revised	4/10

EVENDOL: Evaluation enfant douleur; FLACC: Faces, legs, activity, cry, and consolability; VAS: Visual analogue score; NRS: Numerical rating scale; VRS: Verbal rating scales.

Pain perception is the result of a complex interplay of sensory, cognitive, social and emotional drivers that vary between individuals[14]. By far, pain-related behavior observation is still one of the main method for grading a patient’s extent of suffering, but its interpretation can be influenced by numerous factors, such as age, dementia, healthcare professional’s cognitive bias and patient motivation[6,9]. The automatic detection of pain is a subject of high interest in the health domain as an important indicator for medical diagnosis after surgery[15]. Facial expression exemplifies one of these pain-related behaviors and automatic recognition is being explored as a method to provide an objective measure of pain perception[16]. Facial expression recognition based on action units resulting from muscle activity is one of the technologies and current trends of automatic pain assessment using machine learning and artificial intelligence[17]. Machine learning is an algorithm that may offer a solution to the vexing challenge of predicting postoperative pain, and the algorithm uses a variety of mathematical approaches and is often more computationally efficient and accurate[18]. Although technology designed to detect facial expression pain has been applied to investigative researches, accuracy in assessing pain intensity remains limited currently. In summary, it is far from ready for objective pain assessment in clinical use, but the potential benefits for pain management, combined with the advances of technology and artificial intelligence (AI), make this field worthy for continued research[16].

With machine learning as AI technology can integrate data from “reasoning”-“knowledge”-“learning” to obtain patterns and then make objective predictions about the unknown subject[19]. Image processing of face models using convolutional neural networks (CNN), which is one of the machine learning techniques also plays an important role in the study of facial expression recognition of pain in patients, including infants and consciousness disorders[20-22]. With the advantages of image recognition and deep analysis, deep transfer learning combines the functions of deep learning and transfer learning, which can transform relevant examples or features from massive data sets to micro data sets, thus improving the generalization ability of the model and reducing the dependence on the database sample size[23]. Since these technological advances can provide mature methodological support for this hypothesis, thus, an important study is recommended: Based on a large database of images of facial expressions of children after procedure associated with pain, the development and application of software for automatic pain degree recognition is worth to study using an AI technology for deep learning and analysis of image data.

WHAT IS THE HYPOTHESIS AND RATIONALE FOR THE STUDY?

In view of aforementioned, we hypothesize that image database construction and software development for automatic evaluation of facial expressions with introduction of AI technology can be used in the assessment of children acute postoperative pain to ameliorate pediatric perioperative pain management.

Pain is a reaction of the body to a noxious stimulus[24]. There is no specific “pain center” in the brain, but rather six areas of the brain including the thalamus and anterior cingulate cortex consistently respond to acute pain[25]. Early exposure to pain is associated with abnormal functional connectivity between thalami and bilateral somatosensory cortex, and between the right insular cortex and ipsilateral amygdala and hippocampal regions, with a more evident effect in children undergoing more invasive operation[26].

Some results suggested that the thalamus may play a key role underlying the association between pain and poor neurodevelopmental outcomes in these children[5,27]. In addition, another study showed that subcortical structures in children may be most vulnerable to procedural pain, inducing cognitive and motor disfunction[28].

A number of pain scales can be categorized into three types, self-report (as the gold standard)[29], behavioral, and physiological parameters. For example, the FLACC scale used for children do often exhibit behaviour indicating pain. Both reviews and published clinical guidelines suggest the FLACC scale for procedural pain assessment, the validity of which has been more recently challenged in a systematic review of the scale’s psychometric properties[30]. A cross-sectional study of children aged 4 to 17 years support the use of NRS for most children aged 6 years and older, but not for those aged 4 and 5 years[31]. Another study show that it is not possible at this time to confidently accept the modified behavioral pain scale as suitable for assessing all procedural pain in young children[32]. Thus, it is of significant limitations of each assessment tool. An ideal pain assessment tool should be a composite one, including self-report and one or more of other parameters (age and gender, development and communication level, different personalities and temperaments, individual clinical condition)[33], and only those tools with adequate reliability and validity can be considered to meet the criteria of accurate pain assessment.

Pain assessment by child facial activity coding is interesting and technical progress has provided a foundation for objective pain evaluation using facial expression. The aim of this study is to establish an automatic facial expression evaluation test software for children reflecting true degree of pain, with introduction of AI. CNN is an effective method to recognize facial emotions, and the application is to extract the image features layer by layer to obtain structural features, which can be used to express the high level semantics of a single image[34]. The accurate final model is obtained by integrating and regressing between visual geometry group (VGG) and residual network 50 (ResNet50), which are the mainstream models. The team use the University of Northern British Columbia-McMaster Shoulder Pain Expression Archive Database to develop the source model in the early stage, with the following parameter settings: The CNN architecture batch size input repair is 165 × 165, the batch size is 128. And the training and parameter fine-tuning are carried out by the VGG, ResNet50 and VGG + ResNet50 models respectively. Finally, the source model is obtained for weight vectors that can be treated as feature vectors.

WHAT IS THE DESIGN OF THE STUDY BASED ON THE HYPOTHESIS?

In the situation of experimental laboratory settings, this hypothesis aims to construct a beta version of the AI-assisted evaluation system for acute pain in perioperative children under the successful establishment of the peri-Operation Acute Pain Children Facial Expression Image Database (OAPC-FEI database), to further establish a prospective cohort study of pediatric patients with acute pain in perioperative period and to validate and continuously optimize the developed software product. The design flow is shown in Figure 1.

Figure 1 Development process based on the hypothesis. UNBC-McMaster: University of Northern British Columbia-McMaster; VGG: Visual geometry group; ResNet 50: Residual Network 50; VAS: Visual analogue score.

Pre-study preparation

We plan to establish a prospective cohort study of pediatric with acute pain in perioperative period under a beta version of the AI-assisted evaluation system, with a normal healthy pediatric population as the control group and pediatric patients with acute pain in perioperative period as the experimental group. The study is conducted on children under the age of 14 years who will undergo abdominal or orthopedic surgery lasting more than 2 h within 6 to 48 h after the end of surgery (ethics support by the hospital ethics committee and informed consent by the guardians of all children will be carried out). The exclusion criteria were: Premature birth, preexisting neurological, muscular or psychiatric diseases, intraoperative cardiac arrest, admission to intensive care unit (ICU) after surgery and non-cooperation of the patient or guardian with the study.

Image and related data collection

Establishment of the OAPC-FEI database: We plan to recruit 2500 pediatric patients with acute pain in perioperative period who will give facial frontal photographs (3 high-resolution images each time, with a pixel size of more than 20000000) and 10 s video capture per group, with image data collection at the time points of pain, relieved, unrelieved or worsened, and not in pain (pain assessment will be defined by 3 professionals with patient feedback, and the corresponding clinical assessment scores were assigned). The clinical data collection will be also conducted.

Pre-processing of images: All images are pre-processed with filtering and denoising, geometric normalization, gray balance, and mean square scaling. Simultaneously each image is randomly transformed (randomly rotated, cropped, flipped, etc.) to enhance the amount of data and the reliability of the results.

Video clips: After acquisition and screening, the original dataset can be obtained. However, the original dataset is far from the standard facial expression database due to the existence of invalid captured frames, small percentage of neonatal facial regions in the captured frames, and different duration of sample videos.

We perform manual video screen editing of the initial dataset to standardize children’s facial expression features, and manually process the original dataset using the beaver nest all-in-one video screen converter with the following editing requirements such as selecting representative clips of typical expression features of children in each captured video screen and eliminating invalid filming frames (video screens that block the facial expressions of children to change sides). In this paper, we use the method of running scripts to control the length of the video screen to solve the problem of different length of the sample video, after the manual video screen editing. In this paper, self-written script file can set the length of the video screen to s = 10 (for a video screen is edited less than 10 s also be retained).

Normalization: When the same sample is collected, the images taken may be various as different imaging conditions. Therefore, the samples are normalized so that the images of the same subject captured are consistent. Geometric normalization allows for consistency in image size and correction of face rotation as angular tilt, while grayscale normalization aims to reduce and compensate for the effects of different lighting conditions on face images. We mainly use scale normalization in the study. We transform the video frame sequence at a frame rate of 25 frames/s after editing all sample videos. However, as the effect of video editing, the resolution of each video frame sequence varies to such an extent that it cannot meet the input requirements of the experiment, so it is normalized to a video frame sequence of size 256 × 256 pixels.

Data augmentation: Data augmentation is to improve the accuracy of prediction by training data as close as possible to the test data. In addition, data augmentation can increase the robustness characteristics of the training model, which results in excellent generalization ability of the model. Each image is randomly transformed (randomly rotated, cropped, flipped, etc.) to enhance the amount of data and the reliability of the results. Use generative adversarial network to generate new positive facial samples from random noise, learn to generate new facial expression samples, increase the diversity of the data and improve the generalization ability of the model. Use open computer vision library to do image facial recognition.

Building a deep transfer learning network model on top of the pre-developed source model

After data is preprocessed, it is used to train a CNN to perform the pain level recognition task. This is achieved by fine-tunning a VGG-16 CNN pre-trained with faces. This section utilizes the VGG model with convolutional layers (with 5 maximum pooling layers), 2 fully connected layers, the softmax output layer, and the ResNet50 model with four residual modules to provide the most accurate reference model for the automatic evaluation method of facial expressions by activating the rectified linear units function, boundary zeroing, and maximum pooling. The specific model is shown with VGG-16 as the basic building block: The following is the pain recognition model used in this study, including: (1) The convolutional and down sampling layers, migrated from the VGG-16 network and loaded with parameters already trained on the corresponding VGG-16 network; and (2) the full connectivity layer: From the pre-trained full connectivity layer.

Model validation and optimization

Eighty percent (2000 cases) of the case images from OAPC-FEI were selected and entered into the source model for self-training, validation, optimization, and fine-tuning to obtain the new prediction model. Finally, the remaining 500 images were imported into a new prediction model for validation, and specificity and sensitivity were calculated, analyzed, and compared with the BPS, which is the currently accepted pain scale.

Development of automatic evaluation test software for facial expression recognition in children with acute pain

To establish a prospective cohort study of 500 perioperative children with acute pain, to validate and continuously optimize the developed evaluation methods, finally to develop a facial expression recognition AI-assisted evaluation system for pediatric patients with acute pain in perioperative period. In this paper, we compare our model in a binary setting by using the area under the curve plotted with sensitivity as the ordinate and 1-specificity as the abscissa and we also test it against other state-of-the-art continuous prediction models with intraclass correlation coefficient, Pearson correlation coefficient, mean square error, and mean absolute error[35].

CONSEQUENCES AND DISCUSSION OF THE HYPOTHESIS

In combination with adverse stimuli such as surgical trauma, pain as a sixth vital sign in pediatric patients has an important place in the management of perioperative patients. While numerous pain evaluation tools for children have emerged, few are comprehensive and reliably accurate in assessing pain, and each has its own certain limitations, so that the current onerous pain evaluation system still has much room for improvement. Indeed, pain assessment could be highly benefited from automatic tools and this goal has been already addressed several times in the past. In addition, pain detection is also an important task of computer vision, since it is a clear step toward an automatic detector of spontaneous face expressions[35-37]. Therefore, the development and application of evaluation models for automatic pain recognition and analysis applicable to pediatric patients will be of great clinical value. Through our experiment for this hypothesis, three results may be achieved. Firstly, the database of facial expression images of children with acute perioperative pain provides direction and evaluation means for effectively enhancing and improving children’s postoperative pain in China in the future. Secondly, the introduction of AI (deep transfer learning) techniques to evaluate facial pain expressions provides a new approach for acute pain assessment and contributes to the development of novel software for automatic pain assessment in children. Finally, the development of the new automatic software may provide pediatric patients a timely and safe pain assessment after surgery, and can promote and establish a wide pain evaluation capabilities for professionals.

CONCLUSION

Our hypothesis shows a new approach to pain assessment based on AI with data from images of pediatric patient’s facial expressions in postoperative period. It may provide us another domain in accuracy of pain assessment for pediatric patients, enhancing the quality of management of acute pain.

ACKNOWLEDGEMENTS

We would like to thank Dr. Wu MJ for his guidance on the protocol and computer technology.