Artificial Intelligence for Developing Tools and Technologies in Vascular and Cardiac Surgery Applications: A Survey

Introduction

Novel computing and information technologies have led to the generation of large quantities of data, whose efficient analysis constitutes a major challenge. Humans do not have the capacity to analyze and identify patterns across vast quantities of data, thus necessitating technological solutions. Artificial intelligence (AI) contains a robust armamentarium of diverse methods and tools that can be utilized for these purposes, based on the requirements of the specific application. Processing of multimodal and datarich information, ranging from physiological signals, pathological information, patient-specific anatomy and vascular network morphology, all potentially varying in real time is critical in the decision-making during interventions for cardiovascular patients, which makes the use of AI technologies a natural choice for the aforementioned applications.

From a technical point of view, artificial intelligence may be defined as follows [1]: “AI = knowledge representation + solution search”, and its’ goal is the development of intelligent systems that act and think rationally. Overall, artificial intelligence has been identified to consist of six components as represented in (Figure 1).

Figure 1: Artificial Intelligence components (Adapted from (1)).

Notably, one of the most important components of artificial intelligence is “machine learning”, which encompasses models that utilize available data to evaluate their parameters (e.g., regression models for prediction making) or to recognize patterns (e.g., clusters within data) [2]. With the rapid development of data technologies, machine learning has evolved to become the most widely used and most recognizable component of AI, with increasing translation to practical applications [3]. Machine learning tools are developed on the basis of several parameters, called weights, which are estimated during the training phase. The training phase is a process that utilizes pre-existing data to identify and optimize values for the weights. Usually, training takes the form of an optimization problem that aims at minimizing an objective function based on a set of known data. The training data are organized as combinations of known inputs-known outputs for the system under investigation.

In this paper, we present the implementation of artificial intelligence towards the analysis cardiovascular physiology and more specifically the relevance of AI in cardiovascular diagnostic and therapeutic operations. We will describe existing methodologies and the current state-of-the-art in applied AI, as well as trends for future applications in complex cardiac and aortic procedures. In the next sections, the AI applications are presented in technical domain groups, followed by an overview and a summary of the key highlights.

Artificial Intelligence in Operation Parameter Prediction

Utilization of simulations and models is essential to estimate the outcome of operations and therefore based on these predictions to formulate appropriate decisions. Predictive models have found use in cardiac and vascular surgery, exhibiting high potential as prediction tools. Machine learning offers a variety of tools that learn from existing data (patterns) and provide prediction over future values of specific parameters. Approaches in this direction include the development of a deep neural network adopted for simulating the hemodynamics of the thoracic aorta and predict the steady state distributions of pressure and flow velocity [4]. After training using computational fluid dynamics (CFD) data and geometry analysis, the deep neural network was able to predict both distributions with error ranging between 1.5 and 2%. A deep neural network is a machine learning unit comprised of more than 3 layers that mimics the way human brain functions; (Figure 2) depicts the basic architecture of a neural network [5]. A simple neural network for developing a predictive model for cardiovascular flows has been previously developed [6] and avoids the use of conventional simulators. The neural network is instead trained on noisy and scattered clinical data of flow and vessel wall displacement, from 4-dimensional flow Magnetic Resonance Imaging (4D-MRI) and provided predictions for velocity, pressure and wall displacement pulse wave propagation that were found to be consistent with physical models.

Figure 2: Artificial Intelligence components (Adapted from (1)).

Furthermore, an ensemble of a neural network, a Bayesian network (i.e., a directed graphical model that expresses causal relationships between the system variables (2)), and a support vector machine (abbreviated as SVM - a decision model that is based on identifying the maximum margin between sets of known datapoints that denote different decisions (2)) were developed [7] to provide predictions of post-operative morbidity following endovascular aneurysm repair (EVAR). The ensemble model was developed using a training set of pre-operative patient data such as their co-morbidities and post-operative outcome variables. The performance of the ensemble model was compared against real patient outcomes and was found to be satisfactory [7]. Similarly, another methodology involved a neural network adopted for classifying the sizes of the aortic deformation using 3-Dimensional (3D) data from Computerized Tomography (CT) Scans and subsequently predicted the survival probability of patients with Abdominal Aortic Aneurysms (AAAs) [8]. Results from such an approach provided a high prediction accuracy independent of the number of weights used to model the neural network [8].

Image Analysis and Segmentation

AI based approaches have already made significant contributions in the field of image analysis for diagnostic inference making. Early applications for these technologies were implemented in vascular surgery research, such as autonomous imaging analysis diagnostic models, for peripheral arterial disease and myopathy classification [9]. In this approach, image segmentation was performed on skeletal muscle biopsy specimens and binary image reconstructions where utilized for autonomous discriminant analysis and patient classification [9]. An essential part of such image analysis is the autonomous process of segmentation and detection (the process of partitioning an image into multiple sets of pixels, known as image objects and boundaries). In the medical field, where CT scans are used to support cardiovascular and aortic operations, accurate image segmentation is of paramount importance. AI techniques, mainly taken from the machine learning library, have been applied for image segmentation of CT scan Digital Imaging and Communications (DICOM) file formats. A new method that utilizes support vector machines (SVM) has recently been used as the basis for segmenting an image and subsequently detecting the thin boundaries of silver-stained endothelial cells [10]. Results exhibited detection accuracy of 93% for this approach. Furthermore, an automatic method for image segmentation for the lumen of the thoracic aorta and its main branches, from 3D cardiovascular magnetic resonance (3D-CMR) has been reported in [11]. The presented automatic method was based on machine learning algorithms and more specifically, principal component analysis and classifiers to characterize the aortic anatomy.

A more sophisticated method that has recently been proposed for aortic images analysis utilized a combination of artificial intelligence tools [12]. Segmentation is often further complicated by the high variance of adjacent structures along the aorta in the thoracic and abdominal regions (specifically, it is challenging to discriminate the presence of luminal thrombus from surrounding tissue at the aortic wall). Therefore, a combination of machine learning methods and additional features were utilized in concert for the automated detection of the aortic outer wall, which included both the lumen and thrombus, in order to achieve more accurate diagnostic measurements. The proposed method implemented a support vector machine classifier [2] whose parameters were being adjusted by means of an evolutionary algorithm. Evolutionary algorithms are a set of artificial intelligence algorithms that mimic the natural evolution of species; as depicted in (Figure 3) [13].

Figure 3: Basic steps of an evolutionary algorithm, which mimics the natural evolution processes.

Deep learning neural networks have also been successfully applied and tested for image segmentation from optical coherence tomography, such as to identify the aortic intimal layer in the rat [14]. Convolutional neural networks (CNN), a type of neural networks that do not change even when certain input data undergo numerical transformations [2], have been also utilized for segmentation of images, focused on applications in predicting AAA outcomes [15]. The ability to generalize the use of a CNN trained using synthetic images was tested on a set of real medical images, and the CNN was found to provide high detection accuracy [15]. Another approach for image segmentation has been developed based on identifying clusters within the images [16]. The intelligent algorithms used for such an approach worked for aortic root analysis in preparation for aortic valve implant procedures, were the K-means clustering algorithms exhibited high efficacy. K-means is a widely used machine learning algorithm which identifies clusters within available datasets; where clusters represent groups of data with similar properties [2].

A comprehensive review of the literature detailing the use of AI techniques exclusively for AAA image segmentation underlines not only the value of using AI algorithms to predict rupture or postsurgical outcomes, but also the inherent benefit that AI approaches improve with use and data availability since the training/learning dataset grows larger and the predictive accuracy improves [17]. Lastly, a new approach that adopts a set of deep convolutional networks (i.e., a convolution neural network with at least four layers) has been proposed for identifying features in CT images from 3D reconstructions of the aortic anatomy and autonomous measurements pertinent to pre-operative surgical planning. Tests demonstrated that the isolation and localization of clinically significant features on CT scans was feasible through CNNs even in non-optimized images [18]. In particular, the proposed technique employed a set of three deep convolutional networks (i.e., a single deep CNN for each dimension) to identify regions of interest, namely, the heart, aortic arch and descending aorta within the CT images, has the potential to improve image analysis and accurate detection of organs in images, at rates as high as 98% as shown in [18].

Monitoring and Diagnosis Systems

AI has also been used for developing medical prognosis, diagnosis and monitoring systems. Specifically, for the diagnosis of cardiovascular anomalies, a novel method utilizes data from impedance cardiography signals (ICG) (which represent aortic impedance variation during the heart activity cycle), the Fisher information (which denotes the expected value of the observed information (2)) and the Mahalanobis distance (a measure of the distance between a single datapoint and distribution of data (2)) to differentiate values between known anomalies and measurements [19]. Results exhibit that the proposed intelligent method provided accuracy equal to 96.4% of correct clinical diagnoses. Another approach for monitoring patients after aortic arch repair procedures, introduces a quantitative solution for continuous evaluation of quality of life [20]. In this approach, deep learning, which is utilized for data fusion and reasoning, outputs a numerical value that expresses a quality of life index for the patient under surveillance.

Furthermore, interesting approaches have been implemented to serve as intelligent diagnostic systems [21]. In particular, the synergy between machine learning algorithms used for 3D CT image reconstruction and predictive modeling of aortic deformations have been leveraged for the diagnosis and treatment of aortic valve pathologies. Ultimately the proposed work could replace the highly resource-intensive manual geometric reconstruction and finite element analyses, with a fully automated predictive system. An advanced technique for diagnosis of AAA, has already demonstrated the ability to autonomously analyze computational fluid dynamic data and calculate wall shear stresses [22]. The algorithm is based on data mining methods and more specifically on the Ant Colony optimization algorithm (ACO); which is an evolutionary algorithm and probabilistic technique for computations, inspired by the motion of ants in fields in nature [23]. The results reported designate the ant colony optimization-based system as a highly accurate system for the presence of AAA.

Since the applicability of algorithms is highly dependent on the input data source, new methods for performing clinical diagnosis have also looked at alternate sources for relevant information, including the use of ultrasound signals [24]. The proposed method adopts a temporal convolutional neural network to process a sequence of ultrasonic signals (measuring the vascular diameter of the fetal abdominal aorta) and output a diagnosis in real time [24]. Tests on experimentally obtained signals demonstrate the efficiency of the method as a substantial improvement in the accuracy of diagnosis is observed when compared to the current state-of-theart. Similarly, AI systems have also been developed for diagnosis of cardiac disease using phonocardiogram signals [25]. The AI system is implemented in two forms: the first adopts a Bayesian neural network, while the second adopts a radial basis function network [25]. The comparison between the two types of neural networks highlighted the superiority of the radial basis network over the Bayesian; both demonstrating high relative accuracy at 98% and 90% respectively. AI was also used to implement a monitoring system that conducted prognosis of postsurgical complications [26], for patients who had undergone coronary artery bypass grafting (CABG). The AI tool used in this particular instance was a single hidden layer artificial neural network [26]. Prognosis algorithms of post-cardiac surgery complications were benchmarked for performance accuracy in a recent review of the field [27]. In that work, several intelligent algorithms such as Random Forest (that is a multitude of decision trees which designate the best decision), Adaboost (a technique for combining multiple classifier models and then fuse their decision into a single classifier (2) using their weighted sum), gradient boosting (similar to Adaboost but with individual weights computed using a differentiable loss function (2)) were compared, with gradient boosting having the highest accuracy.

Surgical Performance Support and Assistive Systems

Technological advancements in AI have also made possible the development of systems that support and assist surgeons in their procedural decision making or that simplify several complex but predictable processes that human operators find difficult to perform. Such assistive systems often involve a method for controlling assistive robotics in order to conduct minimally invasive surgeries [28]. The main premise is to introduce the technology platforms for an improved human -robot interface and adaptation of the surgical robotic systems to the individual operator’s dexterity needs. Intelligent systems have been developed for controlling robotics that support the operator during cardiac motion [29], where the system is based on the locally weighted regression method (i.e., regression that is conducted around a specific point and utilizes a training set comprised of data “local” (close) to that specific point). The proposed technology compensates for physiologic cardiac motion, in order to create more stable contact during the operators’ hand motion. An expert system that supports decision making involved in the treatment for cardiac intensive care patients, postoperative management and prevention of medical errors has also been outlined [30]. Expert systems are a subset of artificial intelligence that model the accumulated expertise of a human expert, in the form of IF-THEN rules. The basic architecture of an expert system is depicted in (Figure 4). An evaluation of various visual assistive systems for transcatheter implantation of aortic valve prosthesis has been previously reported [31]. The set of systems evaluated are based on image processing, machine learning and deep learning algorithms [31]. The idea is to be able to support the operator after learning the operative process, utilizing angiographic images etc. For the same type of operations, the development of an expert system has also been presented [32], focusing on supporting decision-making with regard to appropriate implant selection, based on patient anatomical data and manufacturing parameters. Lastly, a clinical decision support system has been proposed to help physicians provide a better understanding of the AAA behavior and prognosis [33]. The decision support system utilizes a set of machine learning algorithms to predict mortality rate of patients who have undergone open repair of AAAs. The AI tool used in this work was a multilayer perceptron (a type of neural network) that achieved 95% accuracy [33]. A more advanced application in assistive systems, is the navigation control for endovascular robotic interventions implementing a fuzzy-logic control model that may support aortic aneurysms invasive surgery [34]. The use of fuzzylogic – in the form of fuzzy variables and fuzzy rules - facilitates the dynamic processing of input coordinate data, handles data uncertainty, and allows fast navigation decision making. In this work, a human aortic model was analyzed for ideal endovascular anatomical navigation of a robotic system, by processing coordinate parameters in an (x,y,z) fashion and subsequently control the robotic movement for endo-aortic navigation. The overall work virtually described a fundamental AI surgical assistive model that mimics the vascular surgery expertise.

Figure 4: Architecture of an Expert System.

Results

intelligence offers a wide variety of methods that allow the development of automated tools in various domains. In this paper, the use of artificial intelligence in the medical domain is presented. Several AI tools have been developed and proposed to support cardiovascular diagnostics, monitoring, patient management and operations. The reviewed works were grouped based on the system functionality they implement. Specifically, AI tools have been used for i) parameter prediction, ii) image analysis and segmentation, iii) monitoring and diagnosis systems, and iv) operation support and assistive systems. (Figure 5) depicts the AI tools and their functionality as presented in the current paper. In addition, the fundamentals of some of the most popular AI tools are summarized, namely, neural networks, evolutionary algorithms, and expert systems.

Figure 5: Summary of AI tools and their functionality as presented in the current paper.

Discussion

Artificial intelligence has found use in cardiovascular and aortic applications but wide-spread application is still low. Thus, there is room for more advanced and sophisticated tools and technologies that will push the envelope in the state-of-the-art in AI applications for cardiac, and vascular procedures. AI technologies have the potential to support physicians in various capacities. However, development of intelligent technologies and tools requires collaboration between people from various fields - i.e., medical, engineering and computer science to ensure both high accuracy and translation to practice.

References

• Russell S, Norvig P (2002) Artificial intelligence: a modern approach. Pearson.
• Bishop CM (2006) Pattern recognition and machine learning. Springer.
• Gopal M (2018) Applied Machine Learning. McGraw-Hill Education.
• Liang L, Mao W, Sun W (2020) A feasibility study of deep learning for predicting hemodynamics of human thoracic aorta. J Biomech. 99: 109544.
• Tsoukalas LH, Uhrig RE (1997) Fuzzy and neural approaches in engineering.
• Kissas G, Yang Y, Hwuang E, Walter R Witschey, John A Detre, et al. (2020) Machine learning in cardiovascular flows modeling: Predicting arterial blood pressure from non-invasive 4D flow MRI data using physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering. 358: 112623.
• Hsieh N, Chen J, Lee K, Tsai H (2011) Using Intelligence Techniques to Predict Postoperative Morbidity of Endovascular Aneurys Repair. Springer 197-206.
• Maiora J, Graña M (2011) A Hybrid System for Survival Analysis after EVAR Treatment of AAA. Springer 344-351.
• Cluff K, Miserlis D, Naganathan GK, Iraklis I Pipinos, Panagiotis Koutakis, et al. (2013) Morphometric analysis of gastrocnemius muscle biopsies from patients with peripheral arterial disease: objective grading of muscle degeneration. Am J Physiol Regul Integr Comp Physiol 305(3): R291-R299.
• Iftikhar S, Bond AR, Wagan AI, Peter D Weinberg, Anil A Bharath (2011) Segmentation of Endothelial Cell Boundaries of Rabbit Aortic Images Using a Machine Learning Approach. International Journal of Biomedical Imaging.
• Vitanovski D, Ralovich K, Ionasec R, Yefeng Zheng, Michael Suehling, et al. (2012) Personalized learning-based segmentation of thoracic aorta and main branches for diagnosis and treatment planning. Proceedings International Symposium on Biomedical Imaging pp. 836-839.
• Morariu CA, Thomas M, Pauli J, Daniel Sebastian Dohle, Konstantinos Tsagakis (2016) Sequential vs. batch machine-learning with evolutionary hyperparameter optimization for segmenting aortic dissection thrombus. 2016 23rd International Conference on Pattern Recognition (ICPR) pp. 1189-1194.
• Alamaniotis M, Tsoukalas LH, Fevgas A, Panayota Tsompanopoulou, Panayiotis Bozanis (2015) Multiobjective Unfolding of Shared Power Consumption Pattern Using Genetic Algorithm for Estimating Individual Usage in Smart Cities. IEEE 27th International Conference on Tools with Artificial Intelligence (ICTAI).
• Chueh KM, Kao HL, Chen HH, Chia-Tung Shun, Manuel Calderon-Delgado, et al. (2019) Deep Feature Learning for Contour Segmentation of Aorta's Intima by Using Sub-Micron-Resolution OCT. IEEE International Conference on BioPhotonics (BioPhotonics) pp. 1-2.
• López-Linares K, Stephens M, García I, Iván Macía, Miguel Ángel González Ballester, et al. (2019) Abdominal Aortic Aneurysm Segmentation Using Convolutional Neural Networks Trained with Images Generated with a Synthetic Shape Model. Springer International Publishing 11794: 167-174.
• Elattar M, van Kesteren F, Wiegerinck E, Ed van Bavel, Jan Baan, et al. (2015) Automatic planning of minimally invasive aortic valve replacement Surgery. Springer 9164: 536-540.
• Raffort J, Adam C, Carrier M, Ali Ballaith, Raphael Coscas, et al. (2020) Artificial intelligence in abdominal aortic aneurysm. J Vasc Surg 72(1): 321-333.
• Vos BDd, Wolterink JM, Jong PAd, Viergever, Max A, Išgum, Ivan (2016) 2D image classification for 3D anatomy localization: employing deep convolutional neural networks. Medical Imaging and Image Processing. International Society for Optics and Photonics 9784.
• AlHadidi T, Salah RB (2015) A new intelligent method for the automatic diagnosis of cardiovascular anomalies. 17th International Conference on E-health Networking pp. 126-132.
• de Dios M, Mendes D, Cantarino SG, Sim-Sim, Margarida (2018) Making the invisible visible: Intelligent Recovery Monitoring of Aortic Arch Repair Surgery Proposal. Springer pp. 173-184.
• Liang L, Kong F, Martin C, Thuy Pham, Qian Wang, et al. (2017) Machine learning-based 3-D geometry reconstruction and modeling of aortic valve deformation using 3-D computed tomography images. Int J Numer Method Biomed Eng 33(5): 10.
• Kumar AD, Nidhya R, Hanah Ayisha V, Vigneshwar Manokar, Nandhini Vigneshwar Manokar (2014) Identification of Abdominal Aorta AneurysmUsing Ant Colony Optimization Algorithm. Cham: Springer International Publishing 1: 49-57.
• Engelbrecht AP (2007) Computational intelligence: an introduction. John Wiley & Sons.
• Savioli N, Visentin S, Cosmi E, Enrico Grisan, Pablo Lamata, et al. (2018) Temporal Convolution Networks for Real-Time Abdominal Fetal Aorta Analysis with Ultrasound. Springer 11140: 148-157.
• Abdel-Motaleb I, Akula R (2012) Artificial intelligence algorithm for heart disease diagnosis using Phonocardiogram signals. IEEE International Conference on Electro/Information Technology.
• Souza C, Pizzolato E, Mendes R, Audrey Borghi-Silva, Maurício Machado, et al. (2009) Artificial neural networks prognostic evaluation of post-surgery complications in patients underwent to coronary artery bypass graft surgery. International Conference on Machine Learning and Applications IEEE.
• Lapp L, Bouamrane MM, Kavanagh K, Marc Roper, David Young, et al. (2019) Evaluation of Random Forest and Ensemble Methods at Predicting Complications Following Cardiac Surgery. Springer International Publishing 11526: 376-385.
• Yang GZ, Mylonas GP, Kwok K-W, Adrian Chung (2008) Perceptual docking for robotic Control. Springer 5128: 21-30.
• Florez JM, Bellot D, Szewczyk J, Guillaume Morel (2011) Serial Comanipulation in Beating Heart Surgery Using a LWPR-Model Based Predictive Force Control Approach. Springer 7102: 389-400.
• Marcos Martínez-Romero, José M Vázquez-Naya, Javier Pereira, Alejandro Pazos, Miguel Pereira, et al. (2012) An Ontology-based Expert System for Decision Support in Cardiac Intensive Care Environments. Frontiers in Artificial Intelligence and Applications 243: 1360 -1369.
• Geidarov NA, Klyshnikov KY, Ovcharenko EA (2020) Use of Neural Networks in Visual Assistance Systems for Transcatheter Implantation of Aortic Valve Prostheses. Biomed Eng 53(6): 440-446.
• Rösler ÁM, Fraportti J, Nectoux P, Gabriel Constantin, Sílvio Cazella, et al. (2018) Development and Application of a System Based on Artificial Intelligence for Transcatheter Aortic Prosthesis Selection. Brazilian journal of cardiovascular surgery 33(4): 391-397.
• Monsalve-Torra A, Ruiz-Fernandez D, Marin-Alonso O, Antonio, Soriano-Payá, Jaime Camacho-Mackenzie, et al. (2016) Using machine learning methods for predicting in hospital mortality in patients undergoing open repair of abdominal aortic aneurysm. Journal of biomedical informatics 62: 195-201.
• Miserlis D, Jafari A, Guda T, Alamaniotis A (2020) Fuzzy Logic Navigation System for Autonomous Endovascular Operations. IEEE 20th International Conference on Bioinformatics and Bioengineering (BIBE).