Application experience and research progress of different emerging technologies in plastic surgery

Abstract

In the diagnosis and treatment of plastic surgery, there are structural processing problems, such as positioning, moving, and reconstructing complex three-dimensional structures. Doctors operate according to their own experience, and the inability to accurately locate these structures is an important problem in plastic surgery. Emerging digital technologies such as virtual reality, augmented reality, and three-dimensional printing are widely used in the medical field, particularly in plastic surgery. This article reviews the development of these three technical concepts, introduces the technical elements and specific applications required in plastic surgery, summarizes the application status of the three technologies in plastic surgery, and summarizes prospects for future development.

Key Words: Virtual reality; Augmented reality; Three-dimensional printing; Plastic surgery; Progress

Core Tip: As an emerging discipline, the needs of patients undergoing plastic surgery differ greatly from those of patients in other disciplines. People seeking cosmetic surgery choose plastic surgery because they seek a higher standard of beauty, and their expectations of the procedure are high. Therefore, plastic surgery, unlike other disciplines, requires a high degree of accuracy. Patients usually lack a concrete sense of the outcomes of a procedure before surgery. Physicians’ clinical practice, including training and teaching, also relies heavily on crude personal experience and lacks accurate numerical standards, and young physicians rarely have opportunities to operate. Digital technologies such as virtual reality, augmented reality, and three-dimensional printing have been widely used for the preoperative design, intraoperative navigation, and postoperative evaluation of plastic surgery.

INTRODUCTION

With improvements in material life, the demand for beauty has also increased, and plastic surgery is on the rise in China. As an emerging discipline, the needs of patients undergoing plastic surgery differ greatly from those of patients in other disciplines. People seeking cosmetic surgery choose plastic surgery because they seek a higher standard of beauty, and their expectations of the procedure are high. Therefore, plastic surgery, unlike other disciplines, requires a high degree of accuracy[1]. Patients usually lack a concrete sense of the outcomes of a procedure before surgery. Physicians’ clinical practice, including training and teaching, also relies heavily on crude personal experience and lacks accurate numerical standards, and young physicians rarely have opportunities to operate. Digital technologies such as virtual reality, augmented reality, and three-dimensional (3D) printing have been widely used for the preoperative design, intraoperative navigation, and postoperative evaluation of plastic surgery[2]. Mixed reality is an emerging technology in which reality is combined with the digital environment, including virtual and augmented reality. 3D printing is an emerging technology based on computer digital imaging and continuous multilayer printing. It is expected to solve a range of engineering problems because of its ability to digitize and print complex 3D structures into physical objects[3]. From small teacups to large aircrafts, 3D printing can be used for streamlined production. This technology impacts people’s lives, and different technologies are often used simultaneously in plastic surgery[4]. To provide the reader with a better understanding of the current status and problems in the application of these technologies in plastic surgery, and to promote their wider application and generate more innovative technologies, this paper provides a comprehensive review of the application of different emerging technologies in plastic surgery.

RELEVANT CONCEPTS OF DIFFERENT EMERGING TECHNOLOGIES

Virtual reality concept

Virtual reality uses computers as its core and combines other technologies to generate virtual environments that are highly similar to a real environment. Users can use related devices to interact with objects in a virtual environment to produce a real-environment experience. In the middle of the last century, Dr. Sutherland, the founder of computer graphics, proposed a new-display technology in which the observer can be immersed in a virtual environment as if it were in the real world. By the 21^st century, with the continuous development of key virtual reality technologies and related hardware and software, virtual reality has been increasingly used in the military, medical, and other fields[5]. Clerc et al[6] proposed that virtual reality could be used to reduce pain and anxiety during various medical procedures. The authors sought to explore the effects of virtual reality in common walking minor orthopedic procedures that children may find uncomfortable. The use of virtual reality for awake pediatric orthopedic procedures reduced the procedure time but did not reduce pain or anxiety compared to standard care for children aged 6-16 years. Virtual reality is safe and popular and should be considered as an additional tool. Increased efficiency may allow more cases to be performed. To compare the effectiveness of 3D printing, virtual reality and conventional normal physical (NP) models on the morphology of craniovertebral junction (CVJ) deformities in clinical education were used. The objective and subjective results suggest that 3D printing is a more effective teaching tool than NP models for learning the pathological morphology of CVJ deformities. Virtual reality models also show great efficacy, second only to 3D printing, in improving participants’ understanding of CVJ deformities.

Augmented reality concept

Unlike virtual reality, augmented reality emphasizes the ability to integrate virtual information into the real world, without isolating it from the real world. Augmented reality can be defined in two ways[7]. The first was proposed by Skarbez et al[8] to describe how to classify combined virtual objects and real environments, as shown in Figure 1, where reality and virtual reality are at the two ends of the spectrum and augmented and virtual reality make up mixed reality. Augmented reality is the superimposition of images in a real scene to enhance the display. From this definition, mixed reality is the axis, and motion from left to right is a process from “augmented reality” to “augmented virtual.” The second, proposed by van den Heuvel et al[9], considers augmented reality to have three characteristics: (1) Virtual combined with reality; (2) Three-dimensional registration; and (3) Real-time interaction. Caruso et al[10] examined interactive augmented reality during small otorhinolaryngological procedures. Although virtual reality has been successfully used for pediatric vascular access, keeping children from their comforting parents in the real world can lead to patient anxiety. Augmented reality offers the potential advantage of using distracting holograms while patients maintain eye contact with their parents. The primary objective of this study was to determine the effect of augmented reality on fear during pediatric otolaryngological procedures. Secondary objectives included assessing pain; procedural compliance; and patient, parent, and physician attitudes toward augmented reality, as well as assessing the feasibility of adding augmented reality to a busy outpatient ear nose and throat (ENT) practice (Figure 1).

Figure 1 Reality-virtual continuum.

3D printing concept

Research on 3D printing in plastic surgery dates to the end of the last century. Stoker et al[11] first used this technology for the preoperative simulation of craniofacial surgery. Since the introduction of 3D printing in 2005, the Ninth People’s Hospital of Shanghai Jiao Tong University School of Medicine has improved the system from modeling to intraoperative application and established an additional manufacturing laboratory to continuously develop the potential of 3D printing in plastic surgery[12]. Currently, 3D printing has been successfully used in 2000 procedures, mainly for rhinoplasty, chin augmentation prostheses, cranial defects, and jaw reconstruction. The development of 3D printing has led to innovations in the field of plastic surgery. Based on imaging data, 3D printing can be used to quickly and accurately generate fully simulated biological models for aiding diagnosis and complex surgical formulations[13]. Kuma et al[14] reported that digital technology has been combined with temporomandibular joint reconstruction surgery (TRS) in recent years. Therefore, a systematic study was conducted to evaluate the impact of digital technology on TRS. In addition, the CAD/CAM and 3D printing techniques were briefly compared. TRS takes full advantage of the flexibility and accuracy of assisted surgery. In the future, tiTRS could successfully replace cTRS if these drawbacks are addressed, and the technology will be made accessible to the public.

ELEMENTS OF DIFFERENT EMERGING TECHNOLOGIES

Virtual reality elements

The elements required to implement virtual reality are model data acquisition, modeling, representation techniques, and interactive devices (Figure 2).

Figure 2 Virtual reality elements.

Model data acquisition: First, 3D scanners, cameras, and imaging equipment are used to collect the actual model data. Second, in manual construction, manually-generated data are used to construct the model.

Modeling: First, scene appearance modeling is used to represent a virtual object structure. Next, physics-based modeling is conducted, which is rigid, flexible, indeterminate, and human-motion modeling.

Performance technology: This involves presenting various virtual scene models in the digital space through different presentation methods and algorithms on presentation devices, and presenting users with visual, auditory, and haptic immersive realistic feelings.

Interactive devices: These are divided into: (1) Display devices (helmet display, desktop display, including desktop display, and stereo glasses), projection displays (including computers, large screens, and projectors), handheld displays, free stereo displays, or no need to wear equipment; (2) Haptic interaction devices, such as force feedback joysticks; and (3) Tracking and positioning devices for active and passive tracking and positioning, including electromagnetic, acoustic, and electromechanical types (Figure 2).

Augmented reality elements

The implementation of augmented reality requires sub-model data acquisition, modeling and representation, registration tracking, and augmented image display technologies (Figure 3).

Figure 3 Augmented reality elements.

Registration tracking: (1) Sensor-based registration tracking: The relative positions of objects are calculated based on the data obtained from the sensors; (2) Computer vision-based registration tracking: The observer’s motion trajectory is reversed based on a real scene to determine the virtual information location; and (3) Integrated vision and sensor-based registration tracking: Locates the user’s head.

Head-mounted display: (1) Video and optical see-through helmet display; (2) Computer-screen display; (3) Handheld mobile displays; and (4) Projection display.

3D printing elements

3D printing is an additional manufacturing technology that generates 3D solids by adding layer-by-layer stacks of materials. It not only overcomes the problem of loss caused by the reduction of traditional materials for manufacturing but also makes product manufacturing more accurate. Particularly in the high-end manufacturing industry, 3D printing has several advantages[15]. With the development of high-end manufacturing, 3D printing has attracted considerable attention and is an important technology for driving the third industrial revolution. It consists of three main elements. First, the product needs to be accurately designed in 3D and used with auxiliary design tools for precise positioning. Second, powerful molding equipment is required, and third, the performance and molding material requirements must be met[16]. Because 3D printing has revolutionized traditional manufacturing and is a subversion of traditional manufacturing, printing materials have constrained the development of 3D printing. The only way to expand its application is to develop new materials that can eventually be used in high-end fields. Owing to material limitations, the most mature materials are mainly resin plastics, but they cannot fully meet the requirements of high-end fields; therefore, they must be continuously improved[17]. Plastic materials can be melted. In the molten state, they are extruded from the nozzle and eventually form a product upon curing. Plastic materials are rapidly being developed in the field of 3D printing owing to their good thermal fluidity, fast cooling, and strength. Resin plastics are used in ceramics, glass, and inorganic gels and have become the basic materials for printing. 3D printing bioplastics mainly include polylactic acid, polyethylene terephthalate-1,4-cyclohexanedimethanol ester, and polyhydroxybutyrate, which have good biodegradability. The properties of the various bioplastics are listed in Table 1.

Table 1 Various three-dimensional printed bioplastics and their properties.

Category	Characteristics
PLA	It comes in a variety of translucent colors and glossy textures. As an environmentally friendly plastic, it can biodegrade into active compost. Derived from renewable resources - corn starch and sugar cane. 3D technology is used to form biodegradable polymer materials that can produce tissue engineering scaffolds with high porosity and the ability to grow
PETG	Short thermoforming cycle time, low temperature and high yield. Low shrinkage and good hydrophobicity of the material, thus eliminating the need for storage in confined spaces
PHB	Low melting point biodegradable polyester. As with most biomaterials, it is often used as sutures and has shape memory. Due to the low melting point, high printing temperatures are not required, thus saving energy

PLA: Polylactic acid; PETG: Polyethylene terephthalate-1,4-cyclohexanedimethanol ester; PHB: Polyhydroxybutyrate; 3D: Three-dimensional.

APPLICATIONS OF DIFFERENT EMERGING TECHNOLOGIES IN PLASTIC SURGERY

Virtual reality applications

Preoperative design: The virtual surgical design system included stereoscopic glasses, haptic feedback devices, monitors, translucent mirrors, and other devices for preoperative flap design. The camera tracks the position of the operator wearing the stereoscope to ensure that the image could be viewed from different directions. The surgeon designs the flap and adjusts it using a feedback device to determine the best solution. This system can also be designed for soft tissues and blood vessels[18]. However, this system is subject to errors and requires additional time for preoperative design. The entire process is illustrated in Figure 4.

Figure 4 Preoperative design process. 3D: Three-dimensional.

Intraoperative navigation: Although the accuracy of virtual reality is perfectly suited to clinical needs, the operator always switches back and forth between the operating area and display, which can cause operator discomfort and increase operating time. Therefore, an intraoperative navigation system was developed in the present study. The operator can look directly at the operating area and observe the virtual image superimposed on the real environment. Although augmented reality is more accurate, it requires a longer preparation time and cost and cannot be used in emergency situations, as it is a two-dimensional marker attached to the mandible[19]. Currently, markers fixed to the bone tissue are the most accurate method of registration, but they are usually not accepted by physicians and patients due to their invasive nature in practice. The aim of the study by Pietruski et al[20] was to compare the accuracy of simulated mandibular osteotomies performed using a cutting guide and two different intraoperative navigation systems based on simple and navigated augmented reality technologies. These results suggest that virtual reality-based intraoperative navigation systems can be used in daily clinical practice in the future. Although virtual reality technology still needs to be improved, it can already be used for the presentation of digital navigation data to enhance the surgeons’ awareness and hand-eye coordination during mandibular resection and reconstruction.

Augmented reality applications

Augmented reality is widely used in craniomaxillofacial surgery; however, it remains in the exploratory stage of soft-tissue plastic surgery. A foreign study used the bone tissue anchoring registration method, but the error was high, which may be mainly related to soft tissue deformation, though the error was still within the acceptable range. Shi et al[21] showed that mandibular plastic surgery has an important place in traditional plastic surgery. However, its success depends on the surgeon’s experience. An augmented reality-based mandibular orthopedic surgery-assisting robot was developed to improve surgical outcomes and reduce the burden on the surgeon. The system achieves precise positioning and automatic drilling, solves problems encountered during drilling, and provides a method for future plastic surgery. Although the term augmented reality is increasingly appearing in published studies, real-time, image-guided (so-called “hands-free” and “heads-up”) surgical techniques are often confused with other virtual imaging procedures. To classify augmented reality applications in the field of maxillofacial surgery, three medical databases from 1995 to 2018 were searched for publications containing the terms “augmented reality,” “mixed reality,” and “surgery.” Thirteen publications containing fully available data were identified for a comparative analysis of the methods used and results obtained. Five of the thirteen described a method based on hands-free and flat-vision augmented reality methods using smart glasses or headsets in combination with tracking. Most publications reported a minimum error of less than 1 mm between the virtual model and patient. Augmented reality during surgery can be divided into four categories: Head-up-guided surgery (Type I) with tracking (Ia) or without tracking (Ib); guided surgery using a translucent screen (Type II); guided surgery based on the digital projection of images onto the patient (Type III); and guided surgery based on the transfer of digital data to a monitor display (Type IV).

3D printing

While conventional 3D printing is used to solve complex 3D structural and morphological problems, the newly developed “bioprinting technology” takes advantage of 3D printing in the construction of tissues and organs to solve the problem of microenvironment simulation during the construction process. The resulting 3D structure is not only a morphological framework for the tissue but also contains biological information about the coordination of cellular and inter-tissue functions. The printing of biomaterials, collagen and other scaffolds, and cellular localization printing can simulate tissue structure well, thus perfecting the construction and achieving the purpose of repairing tissue organs through culture and implantation[22,23]. With the development of 3D printing, it is expected to successfully print bionic tissue organs with the same function as normal tissue organs, completely solving the limitations of autologous transplantation and developing the application of 3D printing to a new level. The effect of technical applications is shown in Figure 5. Plastic surgery is a broad field that includes maxillofacial surgery, flap transplantation, liposuction, body contouring, breast surgery, and facial cosmetic surgery. Owing to the material safety requirements of plastic surgery, biomaterials are widely used owing to their good biocompatibility and biodegradability. Currently, a wide variety of biomaterials is used clinically in plastic surgery, and their applications are diverse. In addition, with the rise of 3D printing technology in recent years, good progress has been made in macroscopically precise and personalized microporous-structured bioprosthesis materials, which are expected to bring new developments in biomaterials[24]. Daemen et al[25] used 3D TOE to reconstruct a fully dynamic 3D view of a diseased valve. The reconstruction was cropped at the valve level and captured at mid-systole to evaluate seam defects. The reconstruction was then exported as a surface mesh. To ensure a waterproof and noise-reduced model, the mesh was processed using a computer modeling program, and the valve was 3D printed. For simulation purposes, deformable models were created based on negative-mold fabrication and cast in silicone to simulate the tissue. Model validation was performed by intraoperative evaluation of valve disease and repair strategies. The feasibility of creating rigid plastic and tissue-mimicking silicone mitral valve replicas has been demonstrated. These models can be used in the future to enhance surgical anatomy interpretation, facilitate the planning and simulation of complex procedures, and for training purposes.

Figure 5 Completed plastic surgery: Full nose reconstruction by three-dimensional printing design.

CONCLUSION

Currently, with the development of digital technology and increased requirements of doctors and patients, the three technologies mentioned above will play an important role in plastic surgery, providing strong support for optimizing surgical procedures, improving results, and reducing complications. However, their current application is not common because of high prices, low accuracy, delayed tracking, and insufficient display. As the price of head-mounted devices decreases and image-processing software improves, these three technologies will be promoted in plastic surgery with broad application prospects, requiring in-depth cooperation between physicians, computers, and engineering professionals to promote more accurate and effective applications of the three technologies.