As an innovative imaging processing mode, radiomics can extract microscopic information from images for quantitative analysis. The selected features and machine learning model can provide valuable data for clinical decisions in heart disease. Up till now, several studies have demonstrated the role of radiomics in the accurate diagnosis and discrimination of heart disease as well as in the prognosis assessment of the patient with heart disease. Cardiac Magnetic Resonance (CMR) displays a wide range of advantages, such as multi-parameter, multi-sequence, multi-plane, and no radiation. CMR has advantages in noninvasive assessment of structural and functional heart disease. This paper reviews the workflow and related studies on common heart disease based on CMR images in radiomics.
Radiomics is an emerging field for image analysis. It can extract many invisible features by high-throughput computing and convert them into exploitable high-dimensional data. The research subject of radiomics can be any pictures, and the research process is orderly and controllable. Different steps are required to reach that aim: image collection, image segmentation, feature extraction, and modeling. The identified features based on the histogram as first order feature, followed by shape feature, texture features and high order features can be combined in clinic application and provide disease diagnosis, adverse event prediction, survival analysis, and prognosis assessment . Over the years, radiomics has attracted scholars worldwide. Because radiomics analysis can acquire a lot of microscopic information, and it also can be carried out simultaneously with other clinical procedures, improving ipso facto the health care system by shortening the study time of images by radiologists.
At present, radiomics has been successfully used in tumors: considering lung cancer studies by extracting features from ground glass nodules resulting in a confident classification of diseases [2, 3]; also involved in the studies of hepatocellular carcinoma (HCC) with promising advances in the evaluation of hepatic diseases, determining immunologic index, treatment and prognostication [4, 5, 6]. Lately, it has been illustrated that microvascular invasion (MVI) represented an independent factor for early recurrence and poor prognosis of HCC [7, 8]. Many reports have covered that the models with MVI imaging characteristics were superior to the single clinical index model [9, 10]. Those successful applications in tumors are the starting point to an extension of radiomics to cardiac disease studies. Though in early years, echocardiography based on radiomics was used to differentiate diseases, such as amyloidosis . Due to the poor repeatability of echocardiography, the relevant research failed to make a breakthrough. Therefore, the application of radiomics in heart disease may need to be based on other examination methods.
CMR is a no-radiation examination and can offer images with multi-parameter, multi-sequence, and multi-plane. So it becomes one of the most critical methods for noninvasive assessment of structural and functional heart disease. For instance, cine sequence is useful for evaluating the cardiac function, and the late gadolinium enhancement (LGE) sequence helps to determine the existence of myocardial fibrosis. The first pass perfusion can assess myocardial filling defect, while the T2 weighted sequence detects the formation of myocardial edema. CMR assists in the diagnosis and evaluates the prognosis of heart disease. It has been shown in some studies that HCM patients with delayed enhancement on LGE sequence had a 3.4-fold increased risk of sudden cardiac death and a 1.8-fold increase in all-cause mortality . It was reported that delayed enhancement on the LGE sequence may be a potential matrix for ventricular arrhythmia in the DCM population . Kunze et al.  reported that in patients with revascularization after myocardial infarction, the natural T1 signal value was correlated with peripheral monocyte count (P = 0.024); so T1 signal value was related to systemic inflammatory activity.
Recently, scholars have developed a strong interest in radiomics based on CMR images (Table 1). Their studies have achieved good results in myocardial infarction (MI), hypertrophic cardiomyopathy (HCM), dilated cardiomyopathy (DCM), and myocarditis. Therefore, the aim of this paper is to review the workflow and related studies on common heart disease based on CMR images in radiomics.
|Heart disease||Year||Publication||Image substrate||Sample size||Important feature||Performance|
|Myocardial infarction||2018||Gibbs et al. ||LGE||76||Kurtosis and skewness||AUC = 0.73|
|2019||Androulakis et al. ||LGE||154||Entropy||HR = 3.20|
|2018||Baessler et al. ||Cine||180||Teta1, Perc.01, Var-iance, WavEnHH.s-3, and S(5,5) SumEntrp||AUC = 0.93|
|Hypertrophic||2019||Neisius et al. ||T1 mapping||232||Run-length non-enrichment [RLN]-135
||AUC = 0.89|
|Cardiomyopathy (Myocardial hypertrophy)||2018||Schofield et al. ||Cine||216||mean value, standard deviation, entropy, kurtosis, average positive pixel, and skewness||AUC = 0.89|
|2018||Amano et al. ||LGE||23||entropy LL||AUC = 0.72|
|2018||Cheng et al. ||LGE||67||X0_GLRLM_energy, X0_H_skewness and X0_GLCM_cluster_tendency||HR = 0.78|
|2020||Alis et al. ||LGE||64||GlcmV1SumVarnc, GlcmN3DifVarnc,||AUC = 0.92|
|GlcmV5AngScMom, and Gab8H4Mag|
|2018||Baebler et al. ||T1-weighted||62||GLevNonU, WavEnLL, and Fraction||AUC = 0.95|
|2020||Neisius et al. ||T1 mapping/LGE||217||LRLGE-45
||C-index = 0.75|
|Dilated cardiomyopathy||2018||Shao et al. ||T1 mapping||74||P10, P25, P50, P75, P90, Mode , SD, Maximum, Mean, homogeneity, entropy and contrast||Accuracy = 0.85|
|2019||Muthalaly et al. ||LGE||130||entropy||HR = 3.5|
|Myocarditis||2018||Baessler et al. ||T1 and T2 mapping||39||T2-rluni, T2 sum entropy, and T2-GLevNonU||AUC = 0.88|
|2019||Baessler et al. ||T1 and T2 mapping||31||average T2 time and T2-GLNU||AUC = 0.76|
|LGE, late gadolinium enhancement; AUC, Area Under Curve; HR, hazard ratio; Local binary patterns; LBP (The gray value of the center pixel is taken as the threshold, and the corresponding binary code is obtained to represent the local texture feature); X0 features (They are extracted from the original image).|
The primary process of radiomics is meant to draw the region of interest (ROIs), obtain a large number of features, and then filter the data to receive robust and informative optimal features. Finally, model training and testing are carried out (Fig. 1). The specific steps are designed as follows:
The basic process of radiomics and the factors influence the results.
Images acquired during traditional CMR (including short and long axis, T1/T2
weighted, initial perfusion pass, and late gadolinium enhancing sequences) are
selected for radiomics analysis. Recent studies tend to preferentially select
late gadolinium enhancement and cine sequences images [15, 16], while the use of
the remaining sequences is gradually increasing [17, 18]. Image preprocessing
methods include image resampling and data normalization. Image resampling is to
change the image voxel resolution to 1 mm
Image segmentation is used to outline ROIs on images for further feature selection and model building, and segmentation methods could be divided into three types consisting of manual, semi-automatic, and fully automatic segmentation algorithms . Manual segmentation is time-costing but could promise accurate and reliable outcomes. The automatic algorithm could reduce the contour work for doctors dramatically with high operation repeatability. However, the automatic segmentation algorithm is not as popularized as the manual algorithm, and the accuracy of automatic segmentation is still to be improved. In CMR, the endocardium and epicardium of the left ventricle and endocardium of the right ventricle are often delineated on short-axis images, which define the boundaries of the left ventricular myocardium and the two ventricular blood pools. After the selection of ROIs, data can be extracted; data changes as ROIs change. Therefore, most studies are carried out based on the automatize selection and manually adjusting of ROIs, with better chances of repeatability  (Fig. 2).
On LGE images, we obtained the segmentation results of scar and myocardium of four different patients. The epicardial boundary was blue, the endocardial boundary was red and the scar was yellow. There were two segmentation methods: manual segmentation (previous line) and automatic segmentation (next row). Automatic image segmentation had better repeatability. Cited in .
Feature extraction is a crucial step for radiomics research. Routine images are converted into plenty of quantitative data. It is these quantitative features that combine image information with clinical practice. Radiomics features represent the geometry of ROIs, the distribution and the spatial correlation of signal strength of voxels in ROIs. These features can be divided into four categories:
a) first order features: derived from the ventricular contour to define voxel intensity distribution in ROIs (the principle is shown later), such as energy, kurtosis, skewness, and entropy (Table 2). In 1948, Shannon et al.  initially proposed that entropy could describe the quantitative information relative to the communication system. In CMR protocol, entropy is used to evaluate the heterogeneity of pixel signal strength composed by a mixture of fibrotic tissue and normal myocardium. Based on this application, many studies have reported the implication of entropy in the diagnosis and prognosis of common heart disease .
|Standard Deviation||Interquartile Range||90th percentile|
|Root Mean Squared (RMS)||Mean Absolute Deviation (MAD)||Robust Mean Absolute Deviation (rMAD)|
b) shape features: determining the size and shape of ROIs in two or three dimensions. They include not only the conventional features like length, volume, and the surface of ROIs, but also features describing the overall shape of ROIs, such as compactness, sphericity, and elongation (Table 3). Shape features are independent of the gray intensity of the image, so they are only acquired through the original image .
|Mesh Surface||Mesh Volume|
|Pixel Surface||Voxel Volume|
|Perimeter to Surface ratio||Surface Area to Volume ratio|
|Maximum 2D diameter||Spherical Disproportion|
|Major Axis Length||Maximum 3D diameter|
|Minor Axis Length||Major Axis Length|
|Elongation||Minor Axis Length|
|Least Axis Length|
c) texture features: describing the distribution and the interaction of voxels in ROIs and using digital quantization eigenvalues to describe the image information (Table 4). Quantitative texture features can be relatively objective parameters for diagnosis or prognosis prediction in the clinic, compared with conventional qualitative judgment based on a medical picture. The first step of texture feature analysis is to build the signal strength matrix. Each voxel within the ROIs is assigned to a value accordingly to the different intensity, and then a signal strength form is constructed to form a signal strength matrix. Next, we can directly calculate the signal histogram of the signal strength matrix, which forms the first order features mentioned above. However, the first order features represent the overall signal distribution within ROIs. More complex algorithms are needed to describe the correlation between each adjacent voxel. Through the complex calculation and reconstruction of the original signal strength matrix, a new matrix can be formed, such as gray level co-occurrence matrix (GLCM), gray level run length matrix (GLRLM), and gray level size zone matrix (GLSZM). GLCM is basically constructed according to the frequency of different signal strength pairs in the original signal strength matrix. Corresponding measures such as contrast (local variation) and entropy (confusion) can then be calculated; these features reflect the gray level changes of signal strength pairs and the degree of confusion in ROIs . GLRLM is constructed by the number of consecutive runs along the specified direction in the signal strength matrix. It can evaluate the spatial correlation of any number of voxels (not just pairs). For example, run entropy reflects the degree of confusion in the gray distribution of the stroke length (higher value is more chaotic); high gray level run emphasis measures the distribution of higher gray values (higher value results in a greater concentration of higher gray values). GLSZM is reconstructed by quantifying the number of voxels within the same signal strength in the image, such as gray level non uniformity and gray level variance. Many scholars have confirmed the value of Gray level non uniformity (GLN) in the diagnosis and differential diagnosis of myocarditis .
|Gray Level Co-occurrence Matrix (GLCM)||Gray Level Size Zone Matrix (GLSZM)||Gray Level Run Length Matrix (GLRLM)||Gray Level Dependence Matrix (GLDM)||Neighbouring Gray Tone Difference Matrix (NGTDM)|
|Autocorrelation||Emphasis||Run Emphasis||Small Dependence Emphasis (SDE)||Coarseness|
|Average||Gray Level Non-Uniformity (GLN)||Gray Level Non-Uniformity (GLN)||Large Dependence Emphasis (LDE)||Contrast|
|Cluster||Gray Level Non-Uniformity Normalized (GLNN)||Gray Level Non-Uniformity Normalized (GLNN)||Gray Level Non-Uniformity (GLN)||Busyness|
|Contrast||Size-Zone Non-Uniformity (SZN)||Run Length Non-Uniformity (RLN)||Dependence Non-Uniformity (DN)||Complexity|
|Entropy||Zone Percentage (ZP)||Run Length Non-Uniformity Normalized (RLNN)||Dependence Non-Uniformity Normalized (DNN)||Strength|
|Variance||Gray Level Variance (GLV)||Run Percentage (RP)||Gray Level Variance (GLV)|
|Energy||Zone Variance (ZV)||Gray Level Variance (GLV)||Dependence Variance (DV)|
|Informational Measure of Correlation (IMC)||Zone Entropy (ZE)||Run Variance (RV)||Dependence Entropy (DE)|
|Inverse Difference Moment (IDM)||Low Gray Level Zone Emphasis (LGLZE)||Run Entropy (RE)||Low Gray Level Emphasis (LGLE)|
|Maximal Correlation Coefficient (MCC)||High Gray Level Zone Emphasis (HGLZE)||Low Gray Level Run Emphasis (LGLRE)||High Gray Level Emphasis (HGLE)|
|Maximum Probability||Small Area Low Gray Level Emphasis (SALGLE)||High Gray Level Run Emphasis (HGLRE)||Small Dependence Low Gray Level Emphasis (SDLGLE)|
|Sum of Squares||Small Area High Gray Level Emphasis (SAHGLE)||Short Run Low Gray Level Emphasis (SRLGLE)||Small Dependence High Gray Level Emphasis (SDHGLE)|
|Large Area Low Gray Level Emphasis (LALGLE)||Short Run High Gray Level Emphasis (SRHGLE)||Large Dependence Low Gray Level Emphasis (LDLGLE)|
|Large Area High Gray Level Emphasis (LAHGLE)||Long Run Low Gray Level Emphasis (LRLGLE)||Large Dependence High Gray Level Emphasis (LDHGLE)|
|Long Run High Gray Level Emphasis (LRHGLE)|
d) high order feature: filter or high order image parameters are added to first order or texture features, such as Wavelet, Laplace of Gaussian, and autoregressive model. There may be redundant features or features not closely related to the results .
The number of extracted features can reach thousands. So feature selection (such as principal component analysis or clustering) will identify and delete redundant and unstable image features . It will avoid the overfitting of the model.
The purpose of radiomics is to develop a model for diagnosis, prognosis, and treatment evaluation of medical diseases. A radiomics model is developed using specific classifiers for model training. There are kinds of classifiers, including random forest (RF), support vector machine (SVM), naive Bayes, and k-nearest neighbor method (K-NN) . RF is composed of many decision trees; each decision tree comprises random samples and random features. The final judgment will be made through the summary of all the decision trees’ results . Its advantage is that there will be no overfitting phenomenon if there are enough decision trees, while too many trees will slow down the algorithm speed. SVM uses a separation hyperplane to partition the training set correctly. It can tolerate some wrong samples in the boundaries, thus improving the robustness and generalization of the model . Naive Bayes is based on probability theory and assumes that each feature is independent and suitable for small sample classification. The principle of K-NN is to take the k training samples nearest to the test samples. The category with the most frequent occurrence in these k samples is the classification result. The disadvantage of K-NN is that it has higher requirements for the balance of sample categories.
All these classifiers can be used for modeling, but the classifier which shows the best performance is preferable. Training sets, verification sets and testing sets are very important in the process of radiomics analysis. Generally, the data is distributed into several groups: training sets used to fit the model, and verification sets used to adjust parameters of the model but also for preliminary evaluation of the model, while the testing sets used to evaluate the final generalization effect of the model. For instance, K-NN was used to identify whether HCM patients occur ventricular arrhythmia (the Area Under Curve (AUC) = 0.92), but the research lacked testing sets. Hence, it was impossible to evaluate the generalization effect of the model, and further research was needed . The model is representative only if the sample size is large enough; experts propose k-fold cross-validation for the defect that there are insufficient clinical research samples . The training sets are split into k samples, and one of them is randomly selected as the verification sample, and the remaining k-1 is the training sample. The process is repeated k times to obtain the best performance. Usually, k is acceptable for sampling when its value is between 5 and 10. If k is less than 5, the underfitting problem could occur .
MI is one of the most serious cardiovascular diseases threatening human health, and the main pathophysiological mechanism is myocardial ischemia and necrosis due to coronary artery stenosis and occlusion . Patients have the risk of ventricular arrhythmia and poor left ventricular remodeling . Baessler et al.  studied the efficacy of texture features in diagnosing subacute and chronic myocardial infarction on CMR cine images. The results showed that the five texture features Teta1, Perc.01, Variance, Wavenhh.s-3, and S (5,5) SumEntrp had statistical significance in differentiating infarcted myocardium from normal myocardium. In the multiple Logistic regression models, diagnostic accuracy of MI was excellent (AUC = 0.93). Gibbs et al.  explored the relationship between the features based on the Laplacian of a Gaussian filter and scar heterogeneity and arrhythmia events. The results showed that the kurtosis (P¼0.005) and skewness (P¼0.046) of MI patients with arrhythmia events were significantly higher than those without arrhythmia events. Androulakis et al.  used the entropy based on CMR-LGE images to measure scar heterogeneity parameters in patients with myocardial infarction and the entire left ventricular myocardium. In the multivariate analysis results, the risk of ventricular arrhythmia increased by 1.9-fold for every unit of entropy increase in scar tissue (hazard ratio: 1.9; 95% confidence interval: 1.0-3.5; P = 0.042), indicating that entropy may be related to the fibrous scar causing the arrhythmia. For each unit increase in entropy throughout the left ventricle, the risk of death increased 3.2 times (hazard ratio: 3.2; 95% confidence interval: 1.1-9.9; P = 0.038), suggesting that the entropy of the entire left ventricle may reflect the fibrosis pattern associated with poor cardiac remodeling. However, there was no statistically significant difference in the patient’s entropy with or without ventricular arrhythmia (P = 0.07) in this study. Some scholars analyzed that it may be due to the loss of the spatial relationship between pixels and original pixels when the intensity signal value was input. So entropy could not be directly applied to image data analysis .
HCM is the most common primary cardiomyopathy, with a prevalence of around
1/500. The diagnostic criteria for HCM are end-diastolic left ventricular wall
Most patients with HCM have a good prognosis, but a small number of patients
have adverse cardiac events, such as adverse cardiac remodeling, malignant
arrhythmia, sudden cardiac death. Some scholars believe that a key mechanism of
adverse events is myocardial fibrosis. Since Choudhury et al.  first
reported the evaluation of HCM fibrosis by delayed enhancement of LGE sequence in
2002, more and more studies have confirmed that the existence and degree of
delayed enhancement were related to the occurrence of adverse cardiac events. In
recent years, many studies have begun to explore the relationship between CMR-LGE
texture features and cardiac adverse events. As far as we know, Amano et
al.  reported, for the first time, the correlation between CMR-LGE image
texture features of HCM and ventricular arrhythmia. The results showed that among
the four texture features, the entropy LL of patients with arrhythmia was
significantly smaller than that of patients without arrhythmia (P =
0.0058), the area under the curve of entropy LL was 0.72, and the area under the
curve of delayed enhancement was 0.96. Although entropy LL was weaker than
delayed enhancement in distinguishing HCM from ventricular tachycardia in this
study, entropy LL was indeed a discriminating factor for HCM with or without
ventricular tachycardia. Unlike previous scholars, Cheng et al. 
magnified the focus from several specific texture features to 90 features
extracted from images and studied the correlation between LGE-CMR texture
features and poor prognosis of HCM patients with systolic dysfunction. The
results showed that the higher X0_GLRLM_energy, lower X0_H_skewness and lower
X0_GLCM_cluster_tendency were positively correlated with adverse events
HCM is usually found in youth, so some patients need to have multiple CMR
examinations during long-term follow-up. Gadolinium-based contrast agents are the
most common contrast mediums used in CMR examination. However, gadolinium is a
contraindication for patients with renal failure, and it will cause nephrogenic
systemic fibrosis. This is because after receiving gadolinium-based contrast
agents in patients with renal function impairment, the clearance rate of contrast
medium in vivo is slowed down, and Gd
The etiology of DCM can be either primary or secondary. This disease is characterized by left ventricular or right ventricular dilatation with contractile dysfunction. The diagnosis of DCM remains a major clinical challenge because of the exclusion of ischemic cardiomyopathy and other non-ischemic cardiomyopathies that may produce a pattern similar to the left ventricular remodeling. Shao et al.  studied whether texture analysis on the T1-mapping image of SVM was helpful to DCM diagnosis. The results showed that the SVM classifier’s diagnostic accuracy based on a meaningful histogram of the T1-mapping MRI image and GLCM was 85%. This method may provide an auxiliary tool for objective quantitative evaluation of clinical DCM diagnosis. Accurate risk stratification of sudden cardiac death is essential for the prognosis of patients with DCM.
The accuracy of left ventricular ejection fraction (LVEF) used to be a risk
stratification index is limited and is greatly affected by the operator.
Therefore, scholars hope to explore a new parameter to assist the risk
stratification assessment of DCM. Myocardial fibrosis in patients with DCM
provides a material basis for ventricular arrhythmia . Myocardial
heterogeneity is formed by the coexistence of different types and degrees of
fibrous tissue with viable myocardium. Muthalaly et al.  used
radiomics methods and hypothesized that the left ventricular myocardium
heterogeneity was considerable in DCM patients requiring primary prevention by
CMR assessment; that is, the entropy value was enormous, and the risk of
ventricular arrhythmia was also greater. It showed that when judging whether DCM
patients had an arrhythmia, left ventricular entropy was statistically
significant in univariate factor analysis and multivariate factor analysis
The viral infection is the most common cause of myocarditis, with an incidence
rate of 20/10000 and a mortality rate of 4.8/100000 [52, 53]. The disease
pathological changes comprise myocardial edema, degeneration, necrosis, and
lymphocyte infiltration [54, 55]. Myocarditis is asymptomatic in mild cases, but
severe arrhythmia, acute cardiac insufficiency, and sudden cardiac death have
been noticed in some cases. The “gold standard” for diagnosing myocarditis is
the endomyocardial biopsy, but the examination is invasive and has low
sensitivity, so it is not widely used in clinical practice [56, 57]. CMR provides
specific value for noninvasive diagnosis of myocarditis. Noninvasive diagnosis of
myocarditis is still one of the challenging problems in clinical practice.
Baessler et al.  analyzed the T1 and T2 mapping images of patients
with acute infarct myocarditis similar to acute infarct myocarditis using
radiomics. They found that these three texture features were statistically
different between patients with positive and negative endocardial biopsy
In summary, radiomics based on CMR has made certain research progress in common heart disease such as MI, HCM, DCM, and myocarditis. However, it is traditional parameters that are still used as references in clinical decision-making, not radiomics. So the application value of radiomics is still not well defined. We may need to solve the following problems: Firstly, CMR scan time is long, and the time resolution is not as good as ultrasound; CMR is not suitable for some special populations, such as claustrophobia and elderly and frail patients. Secondly, it is difficult to obtain enough heart disease in a single-center, suggesting that radiomics research may require multicenter cooperation to study heart disease. Finally, for the sake of repeatability and comparability of radiomics research, it is essential to use standardized imaging protocols. However, radiomics is still in the embryonic stage and mainly focuses on texture features in the study of heart disease. At present, studies on feature extraction, feature selection and classifier training of heart disease are not enough. Therefore, to achieve better clinical decision-making, we should overcome these challenges and perform further studies to prove the role of radiomics in heart disease.
Design and initial draft: Jing-Le Fei.
Critical Review and revisions: Cai-Ling Pu, Fang-Yi Xu, Yan Wu, Hong-Jie Hu.
Final approval of manuscript: Jing-Le Fei, Cai-Ling Pu, Fang-Yi Xu, Yan Wu, Hong-Jie Hu.
Thanks a lot for Umba Mabombo Pierre’s English support.
National Natural Science Foundation of China (NSFC) under Grant No. 81873908.
The authors declare no conflicts of interest statement.