Skip to main content
Free AccessImaging patients with stable chest pain special feature: Review Article

Development and application of artificial intelligence in cardiac imaging

Published Online:


In this review, we describe the technical aspects of artificial intelligence (AI) in cardiac imaging, starting with radiomics, basic algorithms of deep learning and application tasks of algorithms, until recently the availability of the public database. Subsequently, we conducted a systematic literature search for recently published clinically relevant studies on AI in cardiac imaging. As a result, 24 and 14 studies using CT and MRI, respectively, were included and summarized. From these studies, it can be concluded that AI is widely applied in cardiac applications in the clinic, including coronary calcium scoring, coronary CT angiography, fractional flow reserve CT, plaque analysis, left ventricular myocardium analysis, diagnosis of myocardial infarction, prognosis of coronary artery disease, assessment of cardiac function, and diagnosis and prognosis of cardiomyopathy. These advancements show that AI has a promising prospect in cardiac imaging.


Cardiovascular diseases are the most common cause of mortality worldwide.1 The recent development of algorithms and methodologies for artificial intelligence (AI) accelerates quantitative automated imaging technology towards the diagnosis of diseases and personalized treatment strategies. Many algorithms have been developed for image detection, segmentation and classification, which significantly improved the clinical application in cardiac imaging. The AI-based clinical application has been developed, such as diagnosis of coronary artery disease, assessment of cardiac function, improvement of image quality. Since AI is currently revolutionizing the technical development and clinical application of cardiac imaging, in this review, we aim to give a broad overview of the development of AI in cardiac imaging, including CT and MRI. First, we narratively described the technical developments of AI. Then, we show the results of a systematic literature search for recent clinically relevant studies and discuss the clinical application of AI and its prospect.

Technical aspects


Radiomics is a process designed to extract a large number of quantitative image features using data-characterization algorithms.2,3 Radiomics allows data mining and statistical classifiers to determine the relevant features of an image to the target task and to build a prediction model, that is helpful to diagnose disorders in medical imaging. The radiomic features generally include size and shape based-features, intensity histogram, image voxel relationships, and filtered features and fractal features.4 Recently, radiomics showed to be able to differentiate hypertrophic cardiomyopathy from hypertensive heart disease; the integration of six texture and histogram features achieved an accuracy of 85.5%, outperforming the accuracy of conventional T1 weighted imaging of 64%.5 Radiomic texture analysis of late iodine enhancement on CT images reflects left ventricle remodeling and systolic–diastolic function, and may help to identify different patterns of structure remodeling.6 Coronary plaques are small and have a limited number of voxels, and are therefore very challenging for image analysis. Kolossvary et al demonstrated that the voxels of a coronary plaque were sufficient to perform a radiomic analysis, and found that 21% of radiomic parameters were significantly different between plaques with and without the napkin-ring sign and that radiomic parameters had a higher area under curve (AUC) than conventional parameters (0.92 vs 0.75).7 Kolossvary et al also performed a radiomic approach to identify advanced atherosclerotic lesions ex vivo, and showed a better AUC than visual assessment (0.73 vs 0.65).8

Basic algorithms of deep learning

In contrast to classical machine learning, which uses pre-computed features to build a predictive model for a diagnostic or prognostic task, deep learning simultaneously learns relevant features and builds up a predictive model from input images to the desired outcome.9 Convolutional neural network (CNN) is the most commonly used architecture in cardiac image analysis, which uses a convolution operation in the neural network. At present, CNN and CNN-derived networks have been widely used in detection, segmentation, and classification in cardiac imaging.10 Recurrent neural network (RNN) is another type of neural network, which is often used for processing sequential data. The commonly used RNN structures are long short-term memory (LSTM)11 and gated recurrent unit (GRU).12 There are many sequential data structures in cardiac imaging, such as cine MRI. RNN is especially useful to process the cardiac images for patients with arrhythmia or patients who have problems in breath-holding and even advanced techniques cannot generate good images.13–15

Application tasks of algorithms

The application of AI algorithms on cardiac imaging mainly consists of three parts: detection, segmentation, and classification.

In an image detection task, an object, such as a coronary plaque, has to be identified for further processing. For detection, CNN uses a bounding box to search for the target structure in the input two-dimensional or three-dimensional images. Subimages containing suspected structures are obtained and applied to another CNN model to discriminate between true or false target structures. Finally, the coordinates of detected target structures are obtained as the final output. Network architects like RNN and fully convolutional CNN (FCN) are also used for detection purposes. Xu et al used RNN to detect myocardial infarction areas in cardiac MR sequences and yielded an overall accuracy of 94.35%.16 Guo et al combined a multitask FCN to compute a local centerline distance map of a coronary artery tree, and to detect branch end points with minimal path extractor.17 They reduced missing coronary artery branches and improved the patient-level success rate of centerline extraction from 54.3 to 88.8%.

Image segmentation is the process of partitioning the cardiac image into different regions in order to separate certain structures or lesions, which is often used as a preprocessing step for feature extraction and classification. CNN and CNN-derived networks have been applied to segment left ventricle,18,19 right ventricle,20 coronary artery,21 epicardial and thoracic adipose tissue,22 and coronary calcified plaque.23,24Tran et al proposed to use FCN to segment left and right ventricle in cardiac MR25 FCN is an encoder–decoder-based CNN architecture that is widely used for image segmentation. The FCN model is trained end-to-end in a single learning stage and is easy to optimize. The segmentation task can be divided into several stages. Avendi et al used a segmentation pipeline in three stages.26 First, they used CNN to locate the left ventricle and obtained the location information as a bounding box. Second, a stacked autoencoder was applied to infer the shape of the left ventricle. Finally, a deformable model was proposed to do the segmentation with the inferred shape incorporated.

Image classification is used to predict a label for a given image and is often used to determine the presence or absence of an abnormality in cardiac imaging. For this purpose, CNN accepts the two-dimensional or three-dimensional images as input, forwards process and finalizes the fully connected layer and outputs a category label. Recently, unsupervised learning and ensemble learning have been applied to cardiac imaging.27,28 Zreik et al used an unsupervised convolutional autoencoder of the left ventricle myocardium and applied encoding statistics to classify patients into those with and without functionally significant coronary artery stenosis.27 Wu et al proposed a deep-learning architecture by combining multilayer perceptron (MLP) and a bidirectional tree-structural LSTM for anatomical classification or labeling of coronary arteries.28 The AUCs of labeling four main coronary branches and major side branches were 97 and 90%, respectively.

Deep learning can also resolve other imaging-related tasks, like image registration, generation, and enhancement. In the applications which incorporate the whole process of diagnosis, the algorithms are usually very complex and may consist of several different modules. Zhang et al introduced a method to diagnose chronic myocardial infarction on nonenhanced cardiac cine MRI. They firstly detected the left ventricle region with a localization deep learning network, then extracted motion features with LSTM, and finally used a fully connected discriminative network to distinguish myocardial infarction from normal tissues.19

Public database

The availability of high-quality image database is critical for the development of AI algorithms. The scope and quality of the data set determines the accuracy, generalizability, and robustness of the algorithm.29 The recent availability of public databases encourages researchers to pursue new AI algorithms. Public databases of lung nodules, e.g. the lung image database consortium (LIDC),30 greatly accelerated the development and application of deep learning in lung nodule detection. However, creating an image database is notoriously difficult, patient privacy must be protected and health information must be hidden.31 For developing deep learning algorithms, the database size and the coverage of the patient population should be ensured, which is critical for the generalizability. Meanwhile, labeling of a large number of medical images requires the expertise of experienced radiologists and great labor. In cardiology, several efforts are underway to create large databases from electronic health records like a cardiac arrhythmia database32 and a heart attack prediction database.33 These databases contain hundreds of variables, which can be used to develop new AI systems. There have been some initiatives to develop cardiac image databases. In 2016, Kaggle organized the second Data Science Bowl for left ventricular volume assessment, in which cardiac MR images from over 1000 subjects were provided by the National Institutes of Health and Children's National Medical Center.34 The goal of this competition was to develop algorithms to automatically measure cardiac volumes and to calculate ejection fractions. None of the images in the training and validation set were annotated. In 2017, the Medical Image Computing and Computer Assisted Intervention Society organized the Automated Cardiac Diagnosis Challenge for automatic segmentation of the left ventricular endocardium and epicardium, and a training set of 100 subjects was provided with manual annotation as ground truth.35 However, a major drawback is that those databases suffer from heterogeneity and low quality which narrows down the applicability of cardiac AI outcomes. Recently, a data set search engine has become available,36 listing data sets of multiple organized databases and prior competitions, potentially improved discoverability of existing data sources.

Clinical aspects

Scope and search strategy

A PubMed database search was performed using the terms related to AI, imaging methods and the heart. The searching strategy is shown in Table 1. Articles were included if they: (1) were clinically relevant studies and focused on patients; (2) used AI algorithms; (3) were published in the last 5 years (2015–2019); (4) investigated the detection, diagnosis, prognosis and quality control. Articles were excluded if they: (1) were reviews, abstracts, case reports, letters or conference papers; (2) did not use CT or MRI; (3) did not focus on the heart; (4) were technical studies; (5) did not investigate humans, but for experimental animals or phantom; (6) had a sample size <50, or did not mention the sample size.

Table 1. Literature search strategy

Search terms used to identify relevant citations
#1: (“tomography, X-ray computed”[MeSH] OR “Magnetic Resonance Imaging”[Mesh] OR “comput* tomography”[tiab] OR “radiography”[tiab] OR CT[tiab] OR MDCT[tiab] OR MR [tiab] OR MRI[tiab] OR CMR[tiab])
#2: (“Artificial Intelligence”[Mesh] OR “machine learning”[tiab] OR “deep learning”[tiab] OR “neural network*“[tiab] OR “convolutional neural network*“[tiab] OR “generative network*"[tiab] OR CNN*[tiab] OR AI[tiab])
#3: (“Heart”[Mesh] OR cardiac[tiab] OR cardi*[tiab])
Grammar in advanced search: #1 AND #2 AND #3

The flowchart of the literature review and selection is shown in Figure 1. The categories of AI tasks in clinical cardiac radiology are shown in Figure 2. The included literature are listed in Table 2.

Figure 1.
Figure 1.

Flowchart of literature review and selection. AI, artificial intelligence.

Figure 2.
Figure 2.

The four major categories (with example applications) of artificial intelligence tasks in clinical cardiac radiology. CAC, coronary artery calcium; CAD, coronary artery disease; CCTA, coronary computedtomographic angiography; EAT, epicardial adipose tissue; FFR-CT, fractional flow reserve-CT; PVAT, perivascular adipose tissue.

Table 2. Summary of clinical applications using artificial intelligence on cardiac radiology

ReferenceScan methodApplicationAI methodPatient, nPerformance
Tourassi et al. (2016)37Chest CTAutomatic CAC scoringDLCNN1,028Accuracy: 0.844
Wolterink et al. (2016)24CCTAAutomatic CAC scoringDLCNN250Accuracy: 0.83
CC: 0.944 (correlation with cardiac calcium scoring CT)
Wang et al. (2019)38CTAutomatic CAC scoringDL530CC: 0.77 (correlation with manual analysis)
Cano-Espinosa et al. (2018)39CTAutomatic CAC scoringDLCNN5,973Accuracy: 0.726
CC: 0.93 (correlation with manual reference)
Commandeur et al. (2018)22CTAutomatic EAT quanificationDLCNN250DC: 0.823
Oikonomou et al. (2019)40CCTAAutomatic PVAT analysisML1,575A new AI-powered imaging biomarker leads to a striking improvement of cardiac risk prediction
Zreik et al. (2019)41CCTADetection and classification of plaque and stenosisDLRNN163Accuracy:
Plaque: 0.77
Stenosis: 0.80
Zreik et al. (2019)42CCTADetection of functional stenosisDLconvolutional autoencoders187AUC: 0.87
Kumamaru et al. (2019)43CCTAAutomated estimation of CT-FFRDLCNN and GAN1,052Accuracy: 0.76
AUC: 0.78
Itu et al. (2016)44CCTADetection of functional stenosis by CT-FFRML87Accuracy: 0.832
CC: 0.994 (correlation with CFD predictions)
Coenen et al. (2018)45CCTADetection of functional stenosis by CT-FFRML351Accuracy: 0.85
CC: 0.997 (correlation with CFD predictions)
Dey et al. (2018)46CCTADetection of functional stenosis by plaque quantificationML80AUC: 0.84
Gaur et al. (2016)47CCTADetection of functional stenosis by CT-FFR and plaque quantificationML254AUC: 0.90
von Knebel Doeberitz et al. (2019)48CCTADetection of functional stenosis by CT-FFR and plaque quantificationML84AUC: 0.93
Han et al. (2018)49CCTADetection of functional stenosis by CTPML252
Zreik et al. (2018)27CCTADetection of functional stenosis by myocardium analysisDLCNN166AUC: 0.74
DC: 0.91
van Hamersvelt et al. (2019)50CCTADetection of functional stenosis by myocardium analysisDLCNN126AUC: 0.76
Baessler et al. (2018)51Cine MRIDiagnosis of subacute and chronic MIML120AUC:
Subacute MI: 0.93
Chronic MI: 0.92
Larroza et al. (2018)52LGE and cine MRIDiagnosis of chronic MIML50AUC: 0.849
Zhang et al. (2019)19Cine MRIDiagnosis of chronic MIDL212AUC: 0.94
Mannil et al. (2018)53CTDiagnosis of subacute and chronic MIML57AUC: 0.78
Motwani et al. (2017)54CCTAPrediction of all-cause mortality of CADMLInformation gain ranking10,030AUC: 0.79
Nakanishi et al. (2018)55CTPrognosis of CADML6,814AUC: 0.765
Johnson et al. (2019)56CCTAPrognosis of CADML6,892AUC: 0.72–0.85
van Assen et al. (2019)57CCTAPrognosis of CAD by plaque analysisML45AUC: 0.94
von Knebel Doeberitz et al. (2019)58CCTAPrognosis of CAD by CT-FFR and plaque quantificationML82AUC: 0.94
Tao et al. (2019)59Cine MRIAutomatic left ventricular function analysisDLCNN400Correlation with manual analysis (r = 0.98)
Ruijsink et al. (2019)60Cine MRIAutomatic cardiac function analysisDL2,029Correlation with manual analysis (r = 0.89–0.95)
Baessler et al. (2018)61MRIDiagnosis of hypertrophic cardiomyopathyML62AUC: 0.95
Fahmy et al. (2019)62MRIAutomated myocardial scar quantification in hypertrophic cardiomyopathyDLCNN1,073Accuracy: 0.98
Alis et al. (2019)63LGE MRIAssessment of ventricular tachyarrhythmia in hypertrophic cardiomyopathyML64Accuracy: 0.941
Neisius et al. (2019)5MRIDiscriminates between hypertensive heart disease and hypertrophic cardiomyopathyMLSVM232Accuracy: 0.800
Shao et al. (2018)64MRIDiagnosis of dilated cardiomyopathyMLSVM50Accuracy: 0.85
Gopalakrishnan et al. (2015)65MRIDiagnosis of pediatric cardiomyopathiesMLBayesian rule learning83Accuracy: 0.807
AUC: 0.796
Samad et al. (2018)66MRIPrognosis of ventricular function after repairing tetralogy of FallotMLSVM153AUC: 0.82
Fries et al. (2019)67MRIClassification of aortic valve malformationsMLweak supervision4,000Enable classification of the aortic valve by weak supervision
Wolterink et al. (2017)68CTImage denoisingDLGAN284Cost:<10 s / case
Kustner et al. (2019)69MRIImage deblurringDLGAN18Structural similarity index >0.8
Normalized mutual information >0.9

AI, artificial intelligence; CT computed tomography; MRI, magnetic resonance imaging; DL, deep learning; ML, machine learning; CNN, Convolutional neural networks; AUC, area under curve; DC, dice similarity index; CC, correlation coefficient; EAT, epicardial adipose tissue; PVAT, perivascular adipose tissue; CHD, congenital heart disease; LGE, late Gadolinium-enhanced; SVM, support vector machine; GAN, generative adversarial network; CTP, computed tomography perfusion

Automatic coronary artery calcium scoring

Coronary artery calcium (CAC) score on cardiac CT has been recognized as an independent predictor of cardiovascular events and mortality.70 Manual CAC measurement is time-consuming and its accuracy is heavily affected by motion artifacts, image noise, or blooming artifacts caused by multiple or large calcifications. The utilization of deep learning can fully automate this task, resulting in significant time savings and improved accuracy.24,37–39 Previous studies using a dedicated electrocardiogram (ECG)-triggered cardiac CT to acquire cardiac images to calculate CAC. Recently, it has been demonstrated that CAC scoring is also feasible on conventional non-ECG-triggered chest CT scans, although hampered by a substantial underestimation of the cardiac event risk. Tourassi et al showed that low-dose chest CT could achieve automatic CAC scoring.37 Using 1028 non-enhanced and non-ECG-triggered low-dose chest CT scans, CNN was trained to estimate cardiac bounding boxes, from which voxels ≥130 HU were extracted as candidates for CAC, and subsequently, Agatston scores were automatically calculated for a standard five-class cardiovascular risk classification, with an accuracy of 0.844. In another study, Wolterink et al showed that CAC quantification could be automatically derived from coronary CT angiography (CCTA).24 Based on a CNN trained with 250 patients who had both CCTA and non-enhanced cardiac CT scans, CCTA yielded results highly consistent with non-enhanced cardiac CT with an intraclass correlation coefficient of 0.944. This allows patients to have an automated CAC scoring while being screened for lung cancer by low-dose CT or undergoing CCTA scans, without the need for another dedicated cardiac CT scan for the coronary score. Therefore, the clinical workflow can be simplified and the radiation dose of CT for patients is reduced.

Automatic epicardial and perivascular adipose tissue analysis for improving cardiac risk prediction

Epicardial adipose tissue (EAT) is metabolically active fat in the pericardium directly surrounding the coronary arteries. EAT associates with diseases in the coronary arteries and the heart, such as arteriosclerosis, coronary calcification, and ventricular fibrillation, as well as adverse cardiovascular events. Commandeur et al developed a fully automated deep learning framework to quantify the EAT and thoracic adipose tissue of 250 asymptomatic individuals on CT scans, with a good correlation between automatic and expert manual quantification (r = 0.924–0.945), and a dice coefficient (DC) of 0.823 and 0.905, respectively.22 This rapid and automatic procedure has the potential to improve cardiovascular risk stratification in patients undergoing routine CT scans.

In addition, coronary artery inflammation can cause dynamic changes in the lipid balance of perivascular adipose tissue (PVAT), which can be detected by the perivascular fat attenuation index on CCTA. Oikonomou et al found that radiotranscriptomic analysis of coronary PVAT on CCTA could reveal coronary inflammation and structural remodeling. In their study, testing on 1575 patients, the analysis significantly improved the prediction of cardiac risk with respect to a traditional risk factor stratification (p < 0.001).40

Coronary CT angiography for coronary artery stenosis

CCTA plays a key role in the non-invasive assessment of coronary artery disease (CAD). With up to 99% negative predictive value, CCTA can effectively exclude obstructive CAD. CCTA can also be used to determine the degree of stenosis and quantitatively evaluate the arteriosclerotic plaque features, including high-risk plaque markers such as low attenuation, positive remodeling, spotty calcification, and nap-ring signs.71,72 But such measurements require complex post-processing by an experienced observer. Therefore, deep learning has been used to optimize the information extracted from CCTA, especially to develop algorithms that can perform plaque analysis in an automated, accurate, and objective manner. Zreik et al used recurrent CNN to automatically detect and classify the coronary artery plaque and diagnose the degree of coronary artery stenosis in 163 patients on CCTA, with an accuracy of 0.77 and 0.80, respectively.41 Most recently, AI has been shown to automatically segment coronary arteries, extract center lines and generate post-processed CCTA images (Figure 3), i.e. volume rendering, maximal intensity projection and curved multiplanar reconstructions. These procedures used automatic detection and measurement of stenosis, and finally generated structural or free-text reports. Although a multicenter study is necessary to validate these techniques, cardiac AI has shown a great potential to improve not only clinical workflow for CCTA but also to improve the accuracy of stenosis detection and clinical decision-making.

Figure 3.
Figure 3.

Schematic diagram of Shukun coronary artificial intelligence algorithm workflow. (A) axial image with annotations of aorta and coronary arteries (green); (B) VR image of initial segmentation including vessel image rupture (red) and coronary vein (blue); (C) VR image of corrected segmentation; (D) extracted centerline of coronary arteries; (E1) cMPR image with a detected stenosis (red); (E2) straightened rendering of the curved path shows a detected stenosis (red). cMPR, curved multiplanar reformation; VR, volume rendering.

Fractional flow reserve CT and CCTA plaque analysis for significant functional stenosis

CCTA mainly provides anatomical information of the coronary arteries, but there are still limitations in the diagnosis of functional coronary stenosis.73 Invasive fractional flow reserve (FFR) is regarded as the gold-standard for evaluating the significance of coronary hemodynamics, although recently, it has been shown in a non-inferior design study that non-invasive MR myocardial perfusion can replace invasive FFR.74 But FFR measurements with a pressure wire are invasive, relatively expensive and in a small number of patients FFR causes adverse events. The recently introduced FFR-CT, based on conventional CCTA images, can assess coronary hemodynamics and is highly consistent with invasive FFR,75 although in the critical stenosis category FFR-CT produces indeterminate outcomes. Furthermore, FFR-CT requires complex computer fluid dynamics (CFD) computations, that are time-consuming and heavily affected by the image quality. The utilization of AI on FFR-CT can replace the intensive computing of CFD, and can significantly reduce analyzing time.44,45,47,48 Itu et al trained 12,000 anatomic coronary artery structures using an AI algorithm.44 Their model can predict FFR values on each point on the central line of the coronary artery tree. The accuracy was 83.2% in 87 patients compared to invasive FFR as a gold-standard, and the correlation with the predictions based on the CFD model was excellent (r = 0.9994). The mean analyzing time was reduced from 196.4 s (CFD model) to 2.4 s (AI). In a multicenter study, FFR-CT was performed on 351 patients using a deep learning algorithm to detect functionally obstructive CAD,45 which ran on an ordinary computer. The diagnostic accuracy was significantly improved from 71% for pure visual CCTA to 85%, and the AUC from 0.69 to 0.84.

Some studies have found that quantitative plaque analysis can also predict lesion-specific ischemia. In a multicenter trial of 254 patients, Dey et al used quantitative plaque analysis to predict lesion-specific ischemia by quantifying stenosis, plaque volumes (non-calcified plaque, low-density plaque, and calcified plaque), contrast density difference and plaque length, and finally achieved an AUC up to 0.84.46 Some studies combined the two approaches, using machine learning method of FFR-CT and CCTA plaque analysis, and showed that the prediction of lesion-specific ischemia was improved to 0.90 by Gaur et al,47 and 0.93 by Von Knebel Doeberitz et al.48

Automatic left ventricular myocardium analysis for significant functional stenosis

Because an obstruction in a coronary artery may lead to left ventricular myocardial ischemia, for patients with moderate to severe coronary artery stenosis, a myocardial analysis can be performed in addition to direct analysis of the coronary artery in order to determine its functional significance. Zreik et al27 analyzed the left ventricular myocardium of 166 patients on CCTA by using a deep learning algorithm to automatically identify the significant functional coronary artery stenosis. They showed that left ventricular myocardial segmentation DC and AUC was 0.91 and 0.74, respectively. A single CCTA scan can automatically analyze the left ventricular myocardium to identify significant functional stenosis, which may strengthen the clinical evidence of non-invasive CCTA procedure, thus reduce the false positive rate for further invasive coronary angiography.

Diagnosis of myocardial infarction

The diagnosis of myocardial infarction (MI) by machine learning is mainly realized in cardiac MR (CMR).19,51,52,76 The most common imaging technique in CMR is cine sequence which can provide serial images and are mainly used to assess the contractile function of the heart. Late gadolinium enhancement (LGE) is a well-established sequence for quantifying the extent of tissue abnormalities in or around the cardiac muscle and can be used to detect and validate MI.77 Larroza et al analyzed 50 patients on LGE and cine CMR with chronic myocardial infarction to identify the infarcted non-viable, viable, and remote segments with an AUC of 0.849.52 Baessler et al used radiomic approach to diagnose subacute and chronic MI on non-contrast-enhanced cine CMR, with an AUC of 0.93 and 0.92, respectively.51 Zhang et al used deep learning to detect chronic myocardial infarction on non-enhanced cine CMR in 212 patients and reached an AUC of 0.94.19 Several studies have shown that a non-enhanced cine CMR can diagnose myocardial infarction with the similar efficiency as LGE CMR. Therefore, AI may have the potential to reduce the use of gadolinium contrast administration. MI can also be diagnosed with CT. Mannil et al used 6 commonly used machine learning methods to diagnose 27 patients with acute MI and 30 patients with chronic MI on low-dose CT, with an AUC of 0.78.53

Prognosis of coronary artery disease and adverse cardiovascular events

CCTA shows pathological information, that may be associated with prognosis, but the optimal way to extract the data is still under investigation. Machine learning can be used to model vascular features to better predict CAD and adverse cardiovascular events.54–57 The prediction of 5 year all-cause mortality in 10,030 patients in CCTA was studied by Motwani et al, where 25 clinical factors and 44 CCTA features were evaluated, and the AUC was 0.79.54 This was significantly better than the existing clinical or CCTA parameters, such as the Framingham risk with an AUC of only 0.61. Johnson et al used machine learning to assess the prognosis of 6892 patients on CCTA.56 The AUC for all-cause death, CAD death, coronary heart disease death, and non-fatal MI was 0.77, 0.72, 0.85, and 0.79, respectively. Van Assen et al reported an accuracy of 0.77 to predict adverse cardiovascular events in 45 patients with suspected CAD by plaque analysis, and the accuracy improved to 0.87 in combination with traditional clinical risk factors.57 Machine learning can better mine the prognosis information in CCTA and provide improved prognosis evaluation than with existing parameters.

Cardiac magnetic resonance imaging for cardiac function

Most cardiovascular imaging diagnoses are based on anatomical segmentation of the heart cavities, valves, coronary arteries, and precise measurements of various cardiac function parameters, including ejection fraction or perfusion defects. CMR is a non-invasive imaging technique that produces high-quality images with good tissue contrast between different soft tissues without exposure to ionizing radiation. CMR can obtain a specific anatomical plane in any direction, and there are many imaging methods such as cine, flow, tagged, LGE, and fusion CMR. Machine learning/deep learning based on CMR can now perform rapid and automatic segmentation of heart chambers such as left ventricle, right ventricle, bi-ventricles, and analysis of heart function. Tao et al used CNN to perform a multicenter quantitative study on 400 cine CMR patients,59 and obtained an excellent correlation with a manual expert analysis (r = 0.98). Deep learning is used to train highly variable data sets, which can be used for automatic and accurate cine CMR analysis of multivendor and multicenter data. Ruijsink et al used a deep learning framework to perform automatic cardiac analysis of 2029 patients,60 and applied comprehensive image quality control before and after the analysis. The results of the automatic analysis highly correlated with a manual analysis (r = 0.89–0.95), with a sensitivity of 0.95. Therefore, deep learning can be used to perform a fully automated, quality-controlled CMR analysis without the need for clinician supervision.

Diagnosis and prognosis of cardiomyopathy

CMR plays an increasingly important role in the diagnosis, treatment planning, and prognosis of cardiomyopathy.5,35,61–66 Baessler et al used radiomic analysis and machine learning to detect hypertrophied cardiomyopathy on non-enhanced CMR of 62 patients.61 Four radiomic features were identified with 100% sensitivity and 90% specificity. Changes in heart tissue can be measured with extremely high accuracy without enhancement. Shao et al used a support vector machine learning algorithm to perform a texture analysis of T1 weighted CMR images in 74 patients with an accuracy of 0.85.64 Gopalakrishnan et al analyzed 83 children's CMR using machine learning algorithms to diagnose pediatric cardiomyopathy with an accuracy of 0.807 and an AUC of 0.796.65 Using a machine learning algorithm, Samad et al were able to predict the risk of deterioration after repair surgery of tetralogy of Fallot.66 Two CMR examinations performed at least 6 months apart in 153 patients were analyzed. The AUC for differentiating two categories (deterioration and non-deterioration) was 0.82, and for three categories (major-, minor- and non-deterioration) was 0.77.

Image quality control and improvement

Generative adversarial network (GAN) uses deep learning‐based generative algorithms, which have shown to be able to solve generation and transformation problems in image processing, including image denoising and image deblurring.78 Recently, GAN methods have been applied to medical images. Wolterink et al used GAN to reduce noise on low-dose CT,68 as well as non-enhanced cardiac CT. After GAN training, the noise of low-dose CT was reduced, and the image quality was similar to conventional-dose CT. Similarly, GAN can also be used on MRI. Kustner et al performed retrospective motion correction via GAN to correct artifacts caused by respiration or cardiac motion with high evaluation metrics (normalized mean squared error 0.08, structural similarity index 0.8, normalized mutual information 0.9).69 These explorations are limited to the research environment, that would help to accomplish further practical application.

Prospect of clinical application

Many applications of cardiac AI have been used to make human work more accurate and faster in cardiac imaging. But most studies emphasized on the use of single-purpose algorithms or tools at the proof-of-concept stage. Although there is a long way between technological developments and a routine implementation in clinical practice, several specific AI applications have already demonstrated expert-level performance, and it is likely that these applications will be integrated into the imaging workflow of the radiologist.

One important future prospect of AI is the ability to perform beyond human perception and capability. The first example thereof is CNN powered FFR-CT. FFR is the golden standard for coronary hemodynamics. Traditional CCTA provides anatomic and pathological vision. Cardiac FFR-CT AI provides coronary hemodynamic information, that is not visible to the human eye and outperforms fluid dynamic analysis. The second example is the use of non-contrast cine cardiac MRI data to detect the presence of myocardial infarction which cannot be clearly identified by the human eyes and has been validated by using machine learning, radiomics and deep learning.19,51,52 These approaches may have the potential to reduce the use of gadolinium contrast and largely shorten scan duration. Another example is the use of CNN to improve image quality or to correct motion artifact, which can be the main barrier for a clear cardiac image and is beyond human capability. Zhang et al did an experimental study showing that CNN trained by motion artifacts can largely reduce the CAC scoring variation, and to correct CAC scores of blurred images.79

Another future application field is AI-driven prognosis assessment. Traditionally, clinicians choose treatment methods according to their knowledge and existing evidence. Some studies have demonstrated optimal cardiac AI results for the prediction of prognosis, which will potentially impact therapy planning.54,56 Nevertheless, more well-designed clinical trials are needed to confirm these early results and to open the perspective of clinical implementation of cardiac AI.


Cardiac AI algorithms have been developed for image detection, segmentation, and classification, as well as recently available advanced algorithms, which can significantly improve the clinical application in cardiac imaging. In the past few years, numerous studies have shown the great potential of AI in the diagnosis of CAD and cardiomyopathy, and the assessment of cardiac function, as well as the prediction of prognosis. Recently, cardiac AI exhibits its ability to outperform human perception in several specific cardiac fields. These developments in cardiac imaging improved the daily practice of cardiac imaging and will greatly show its impact in the near future.

Acknowledgment We thank Dr. Marcel J. W. Greuter for his kindly assistance with language editing.

Competing interests Dr. Ning Guo is an employee of Shukun (Beijing) Technology Co., Ltd.

Funding National Natural Science Foundation of China (project no. 81471662 and 81971612), Ministry of Science and Technology of China (2016YFE0103000), Shanghai Municipal Education Commission – Gaofeng Clinical Medicine Grant Support (20181814), Shanghai Jiao Tong University (ZH2018ZDB10), and Clinical Research Innovation Plan of Shanghai General Hospital (CTCCR-2018B04, CTCCR-2019D05). The funders played no role in the study design, data collection, and analysis, decision to publish, or preparation of the manuscript.


Volume 93, Issue 1113September 2020

© 2020 The Authors. Published by the British Institute of Radiology


  • ReceivedSeptember 24,2019
  • RevisedJanuary 06,2020
  • AcceptedJanuary 28,2020
  • Published onlineFebruary 06,2020