Composition and physicochemical features (188D) can extract 188 features containing composition, transform and distribution information [53]. This method includes three parts: composition, transform and distribution [54, 55]. The first section is the amino acid (AA) composition. By calculating the frequencies of amino acids in a protein sequence, sequences are converted into 20D vectors [53]:

(1)$${(v_1,v_2,v_3,\mathrm{\cdots },v_{20})}^T=(\frac{n_1}{L},\frac{n_2}{L},\mathrm{\cdots },\frac{n_{20}}{L})$$

where $n_i$ represents the quantity of an AA in the protein sequence.

The second section is transform. According to the physicochemical properties of a protein, 20 AAs can be divided into 3 different groups. The proportion of each group in the protein sequences is calculated. We use the secondary structure as an example:

(2)$$C_j=\frac{{\mathrm{count}}_{D_i}}{L}(i=1,2,3;\mathrm{j}=1,2,\mathrm{\cdots },8)$$

where $D_i$represents the number of each kind of amino acids and L is the length of the sequence. In this study, we considered [56] eight physicochemical properties. For each physicochemical property, there are three types of amino acids. Therefore, we can obtain 24D features.

The third section is distribution [55, 57]. We calculate the distribution at five positions: at the beginning, 25%, 50%, 75% and the end. We obtained 120D features in this section. Then, we considered the number of different amino acid dipeptides. According to this step, we obtained the 24D features. The formulas are as follows.

(3)$T_{i,j}={\displaystyle \frac{D_i{D}_j\mathrm{or}{D}_j{D}_i}{L-1}}$ $T_{i,j}$
$i,j\in \{(i=1,j=2),(i=2,j=3),(i=3,j=1)\}$

(4)$\mathrm{D}={\displaystyle \frac{H_{ij}}{L}}$
$(j=\text{begining},25\%,50\%,75\%,\text{ending};i=1,2,3)$

where the chain length is measured as $H_{ij}$ at the beginning, 25%, 50%, 75% and end of AAs at which a particular property is located.

This method divides amino acids into three types according to different physical and chemical properties, and considers the position information of three different types of amino acids under different physical and chemical properties. This method comprehensively considers the location information and physical and chemical properties of amino acids. The method is simple and easy to understand. Although the method uses physical and chemical properties, it still mainly focuses on the position information of amino acids, and the physical and chemical properties are not further reflected.

G-gap dipeptide composition is a method used to describe the information about the composition of dipeptides in protein sequences. In this study, we used an enhanced method proposed by Huan et al. [56], who added a pseudo amino acid composition to the g-gap. Thus, each protein sequence is converted into $400+n\lambda$ vectors [56, 58, 59].

(5)$$F{V}_{g-gap}={[x_1,x_2,\mathrm{\cdots },x_{400},x_{400+1},\mathrm{\cdots },x_{400+n\lambda }]}^T$$

(6)$$x_u=\{\begin{array}{c}\hfill \frac{f_u}{{\sum }_{i=1}^{400}{f}_i+\omega {\sum }_{j=1}^{n\lambda }{\tau }_i}(1\le u\le 400)\hfill \\ \hfill \frac{\omega {\tau }_u}{{\sum }_{i=1}^{400}{f}_i+\omega {\sum }_{j=1}^{n\lambda }{\tau }_i}(400+1\le u\le 400+n\lambda )\hfill \end{array}$$

(7)$$f_u=\frac{n_i^g}{L-g-1}$$

where the number of occurrences of i-th dipeptide appearances is denoted by $n_i^g$ and $\omega$ is the weight. $\lambda$ is the number of total counted ranks or tires of the correlations along a protein sequence, and n is the number of physicochemical properties used in this study. ${\tau }_i$ is the i-th tier correlation factor, which reflects the sequence-order correlation between all the i-th most contiguous dipeptides along a protein sequence. In this study, n is 9 and g is 2. Finally, we obtained the 670D features.

This feature extraction method is based on two amino acids. This method first considers the position of each amino acid in the sequence, and fuses the physical and chemical properties of the amino acid through the pseudo amino acid composition.

Adaptive Skip Gram Features (hereafter referred to as 400D) can extract 400 features. This method extracts features according to the distance between amino acids [55, 60, 61]. We assume a given protein sequence P. P is expressed as follows.

(8)$$A_1{A}_2{A}_3\mathrm{\cdots }{A}_n$$

$\mathrm{DT}(A_i,A_j)$ is the distance between amino acids.

(9)$$\mathrm{DT}(A_i,A_j)=j-i-1$$

where i and j represent the position of amino acids. According to the definition of amino acid distance, if two amino acids are adjacent, the distance is 0. The maximum distance between amino acids is L-2. The k-skip-n-gram algorithm counts the frequency of occurrence of any n amino acid sequences in the sequence, expressed as follows:

(10)$${\mathrm{FV}}_{\text{skipgram}}=\left\{\frac{\mathrm{N}\left({\mathrm{a}}_{{\mathrm{m}}_1}{\mathrm{a}}_{{\mathrm{m}}_2}\mathrm{\cdots }{\mathrm{a}}_{{\mathrm{m}}_{\mathrm{n}}}\right)}{\mathrm{N}\left({\mathrm{T}}_{\text{skipgram}}\right)}\right|1\le {\mathrm{m}}_1\le 20,\mathrm{\cdots },1\le {\mathrm{m}}_{\mathrm{n}}\le 20\}$$

(11)$$T_{\text{skipgram}}=\{{\bigcup }_{a=0}^{k}Skip(DT=a)|a=0,1,2,\mathrm{\cdots },k;k\le \frac{L^{\min }}{n-1}$$

$T_{\text{skipgram}}$ represents subsequences, which are composed of n amino acids in the sequence. $N\left(T_{skipgram}\right)$ represents the number of elements in $T_{skipgram}$. $N\left(a_{m_1}{a}_{m_2}\mathrm{\cdots }{a}_{m_n}\right)$ represents the number of occurrences of all n amino acid component sequences in $T_{skipgram}$. The number of $F{V}_{skipgram}$ is ${20}^n$. Due to the differential value of k, Wei et al. [61] proposed the adaptive skip-n-gram model. This model cancels the limitation of k. The values of k are adaptive according to the length of the sample sequence, which makes the features contain more distance information and makes the k-skip-n-gram model have no parameters, avoiding the overfitting problem. In this study, n is 2, so we obtained 400D features using this method.

This method is derived from the n-gram model in natural language processing. It mainly considers whether each amino acid or each polypeptide appears in the sequence and how often it appears. So, this method take into account the distance between amino acids.

Logistic regression [62, 63] is a logarithmic model, and its form is a parametric logistic distribution represented by the conditional probability distribution $P(Y|X)$.

(12)$$\mathrm{P}(Y=1\mid x)=\frac{\exp (\omega \cdot x)}{1+\exp (\omega \cdot x)}$$

(13)$$\mathrm{P}(Y=0\mid x)=\frac{1}{1+\exp (\omega \cdot x)}$$

where $x\in R^n$ is the feature, $Y\in \{0,1\}$ is the class, and $\omega$ is the weight vector. For a given sample, we should calculate both probabilities.

In this study, we used two kinds of logistic regression models. One is shown above. The other is kernel logistic regression. LR is a linear classifier. Therefore, kernel logistic regression (KLR) [64] was proposed, which can be used to classify nonlinear data. KLR uses kernel functions to project the features into high-dimensional space.

Two kinds of decision trees are used in this study. One is C4.5 [12, 65], and the other is PART [66], which is based on C4.5.

(1) C4.5. This model uses the tree structure to describe the process of classification. C4.5 contains nodes and directed edges. There are two kinds of nodes, internal nodes and leaf nodes. Internal nodes indicate attributes, which are the conditional basis of classification. Leaf nodes are classes, which are the labels of samples. The process of C4.5 classifying instances is that C4.5 arranges samples from the root node to a leaf node. Feature selection is based on the information gain ratio.

(2) PART. PART is based on C4.5 and the partial decision tree, proposed by Eibe Frank et al. [67] in 1998. PART extracts rules from a dataset according to an incomplete decision tee. The original principle of the algorithm comes from the separate-and-conquer strategy. This method creates a rule and then removes the instance that is covered by the rule. The method builds rules for the remaining instances until there are no existing instances, recursively.

Naïve Bayes [53, 23] is a classifier based on Bayes’ theorem and features condition independent hypotheses. In this algorithm, first, the input’s and output’s joint probability distribution are studied, which is based on the independent hypothesis of feature conditions; then, for a given input x, we used Bayes’ theorem to calculate the maximum posterior probability of output y. Naive Bayes is a common classification method, which is easy to implement [23, 66].

K-Nearest Neighbor (KNN) [68, 69] is a basic classification and regression method. In this study, we only discuss the application of KNN in classification problems. KNN contains three important factors: the selection of k values, distance functions and decision rules. The algorithm for KNN is as follows:

(1) According to the determined distance function, we can derive k points that are closed to instance x in the training set. The neighborhood of x that covers these k points is called $N_k(x)$;

(2) In $N_k(x)$, the class of x is determined according to the classification decision rule.

(14)$\mathrm{y}=\arg \underset{c_j}{\max }{\displaystyle \sum _{x_i\in N_k(x)}}I(y_i=c_j),$
$i=1,2,3,\mathrm{\dots },N;j=1,2,3,\mathrm{\dots },K$

where $x_i$ is the feature vector and $y_i=\{c_1,c_2,\mathrm{\dots },c_K\}$ is the label. $I(*)$ is the indicator function.

We used the 188D, g-gap and 400D to extract features from sequences. These methods have different emphases. 188D contains the amino acid composition and physical and chemical properties. G-gap contains the dipeptide composition, which can indicate the importance of the peptide chain. Since proteins are produced by the distortion and folding of peptide chains, dipeptides can better recognize proteins. G-gap adds nine kinds of physical and chemical properties to improve its accuracy. 400D features take into account the distance between amino acids. Therefore, it is meaningful to combine these three methods to more comprehensively extract features and improve accuracy.

In this study, we used four combination methods. First, we combined 188D and g-gap and obtained 858D features. 188D divides amino acids into three groups and studies the physical and chemical properties of each type of amino acid. However, g-gap considers the properties of each amino acid.

Second, we combined 188D and 400D and obtained 588D features. Since 400D does not consider the physicochemical properties of amino acids, the combination of 188D and 400D extracts features based on the AA composition and physicochemical properties.

Third, we combined g-gap and 400D and obtained 1070D features. 400D is different from g-gap. The dipeptides in g-gap are adjacent, while in 400D, two amino acids can be separated by several amino acids, that is, the two amino acids considered are not adjacent. G-gap focuses on the composition of dipeptides in the protein sequence, and 400D focuses on the relative position of the amino acids.

Fourth, we combined 188D, g-gap and 400D and obtained 1258D features. 188D considers the frequency of occurrence of a single amino acid and the amino acid’s position according to its physical and chemical properties, which are different from the other two methods. G-gap focuses on the composition of dipeptides in the protein sequence, which is different from the other two methods. 400D focuses on the distance between two amino acids, which is different from the other methods.

Ensemble feature extraction methods can make up for the shortcomings between different methods, and then can construct a set of comprehensive features. By combining feature extraction methods, we can get four different sets of feature vectors.

Ensemble classifiers can organically combine multiple traditional learning models to obtain more stable and accurate results. There are three common ensemble learning algorithms, including bagging, boosting and stacking.

In this study, we used random forest and vote as the ensemble learning methods to classify proteins.

(1) Random Forest (RF) [78, 79, 80, 81, 82]. Random forest is an extension of bagging. Leo Breiman proposed RF. RF is composed of many decision trees, with no correlation between different decision trees. When we need to classify a sample, each decision tree in the forest makes a judgment and classification. The final result is the class with the highest number of votes.

(2) Vote. This method classifies the sample according to the seven classifiers. First, we use LR, KLR, NB, DT, PART, RF and KNN to classify protein sequences. We obtain seven results, and we use majority voting to obtain the final result. If a label receives more than half the votes, the prediction is that label; otherwise, the prediction is rejected.

(15)

where $h_i^j(x)$ is the output of classifier $h_i$ on the label $c_j$. T represents the number of base classifiers, and N represents the number of labels.

In this section, we compare the joint features with single ones. We combined the three feature extraction methods, obtaining four joint features: 588D (combining 188D with 400D), 858D (combining 188D with g-gap), 1070D (combining 400D and g-gap) and 1258D (combining 188D with 1070D). In this section, we also use six classifiers for prediction.

First, we conducted the classification experiment on 588D. To make the comparison of the results clearer, we used the DT, NB and PART experimental results to create a histogram, which is shown in Fig. 4. When the classifier is DT, 188D has the best result. NB and PART were selected for similar reasons. 400D had the best result with NB, and 588D had the best result with PART. The experimental results of the other classifiers are shown in Appendix Table 6. According to Fig. 4, 588D is the best feature extraction method among the three methods, except for DT. 588D improves accuracy most of the time, but it is slightly worse than 188D when the classifiers are DT and LR.

Similar to the above method, we created histograms of the remaining three combined methods, which are shown in Fig. 5. According to Fig. 5, we found the accuracy was generally improved, but the improvement rate was not large. Specifically, 1258D had little improvement compared to 588D and 1070D. However, when the classifier was NB, the performance of 1070D was worse than 400D, potentially because the ensemble method increases the correlation between features and reduces feature independence. We used Max-revelation-Max-Distance (MRMD) to reduce the dimensionality, which may improve the accuracy. This method used the Pearson correlation coefficient (PCC) to calculate the relevance and the Euclidean distance to identify instances of redundancy. The results are shown in Appendix Table 7. Compared to the results without dimensionality reduction, the effect of the classifiers increased and decreased. When the classifiers are LR, DT and NB, the results improved. Compared to the single method, overall accuracy was improved. Therefore, using the ensemble feature method is better than the single feature extraction method.

In this section, we compare the ensemble classifiers with traditional classifiers. We use RF and VOTE as the ensemble classifiers. RF is an ensemble classifier based on the bagging algorithm, using DT as the base learner. RF combines all the classification results for voting and designates the label with the most votes as the final result. We proposed the vote method. In this method, we used seven classifiers to classify the samples, obtaining the final result according to majority voting. The seven classifiers are DT, PART, KLR, LR, KNN, RF and NB, which were used in the previous sections. The results are shown in Fig. 6. There are three ROC curves in each subgraph. Two of them are RF and VOTE, and the other is the traditional classifier with the best performance. From Fig. 6, we observe that the results using ensemble classifiers are better than the results using traditional classifiers. The best AUC was 0.90, for which the classifier was RF and the feature extraction method was 188D. Moreover, the worst AUC in the section was 0.70, for which the classifier was PART and the feature extraction method was g-gap. According to Fig. 6, RF is superior to VOTE. The reason the ensemble method is better than the traditional classification method is that the ensemble method avoids the accidental errors of single methods by comprehensively considering multiple classifiers. All experimental results are shown in Table 2.

According to the results, we found ensemble classifiers are better than single classifier. Ensemble classifiers are beneficial in three ways. First, since the learning task has a large hypothesis space, there may be many hypotheses in the training set to achieve the same performance. Therefore, the single classifier may choose the hypothesis space by mistake, resulting in poor generalization. Ensemble classifiers can reduce this risk. Second, ensemble classifiers can reduce the risk of falling into a terrible local minimum. Third, by combining multiple classifiers, the corresponding hypothesis space will expand, making it possible to learn the best approximation.