M2D-QPCA: An Improved Quaternion Principal Component Analysis Method for Color Face Recognition 1

Principal component analysis (PCA) is one of the successful dimensionality reduction approaches for color face recognition. For various PCA methods, the experiments show that the contribution of eigenvectors is different and different weights of eigenvectors can cause different effects. Based on this, a modified and simplified color two-dimensional quaternion principal component analysis (M2D-QPCA) method is proposed along the framework of the color two-dimensional quaternion principal component analysis (2D-QPCA) method and the improved two-dimensional quaternion principal component analysis (2D-GQPCA) method. The shortcomings of 2D-QPCA are corrected and the CPU time of 2D-GQPCA is reduced. The experiments on two real face data sets show that the accuracy of M2D-QPCA is better than that of 2D-QPCA and other PCA-like methods and the CPU time of M2D-QPCA is less than that of 2D-GQPCA.


Introduction
Color image recognition has received high attention in recent years. Principal component analysis (PCA) and its various variants have been successfully used for color face recognition [1][2][3][4][5][6][7]. Based on the theory of Kirby and Sirovich [1], Turk and Pentland [3] presented the well-known eigenface method for face recognition. Early various PCA methods were mainly implemented by converting color images into grayscale images and representing an image in a vector. As a result, these methods lose color information and partial spatial information of images. In order to make full use of the spatial information of images, Yang, et al. [6] proposed a novel technique named twodimensional principal component analysis (2DPCA) by representing a grayscale image in a matrix. In Torres, et al. [2], Torres et al. pointed the importance of color information in face recognition and extended traditional PCA to color face recognition by using the R, G, B color channel, respectively. This method ignores the connection between three color channels. In order to overcome this shortcoming, Yang and Liu [5] presented a general discriminant model for color face recognition, which uses a set of color component combination coefficients to convert three color channels into one channel by , but the optimal coefficients and are difficultly obtained. Xiang, et al. [4], presented a color image as a matrix where is the number of color channels (usually 3), and is the number of pixels. The color image is flattened like what the PCA approach does, and each column of the matrix is a color vector that represents one image pixel. Then they proposed a color 2DPCA (C2DPCA) method, which combined the spatial and color information for color face recognition. These methods greatly improve the level of face recognition, and however, are still not generalized to treat color face images directly.
To simultaneously and mathematically deal with three channels of color image, the quaternion was applied to represent the color pixel consisting of three components [8][9][10][11]. Recently, Jia, et al. [12] presented the color twodimensional principal component analysis (2D-QPCA) method based on quaternion model for color face recognition. Based on 2D quaternion matrices rather than 1D quaternion vectors, 2D-QPCA combines the color information and the spatial characteristic for face recognition, and straightly computes the low-dimensional quaternion covariance matrix (QCM) of the training color face images and determines the corresponding eigenvectors by quaternion eigen-decomposition (QED). Xiao and Zhou [13], proposed a novel quaternion ridge regression (QRR) model for two-dimensional QPCA (QRR-2D-QPCA) and mathematically proved that this QRR model is equivalent to the QCM model of 2D-QPCA.
In these PCA-like methods, eigenvectors are treated equally, although eigenvalues are different. But, in the experiments, we found that the recognition rate varies with the modulus of the eigenvectors. Based on this discovery, in this paper, we propose a modified two-dimensional quaternion principal component analysis method (M2D-QPCA) for face recognition by means of the framework of the 2D-QPCA method. At the same time, we also correct and optimize the shortcomings of the 2D-QPCA method. The experiment on a real face data set shows that the accuracy of our method is better than that of the 2D-QPCA method or other PCA-like methods.
Recently, Zhao, et al. [14] presented an improved two-dimensional quaternion principal component analysis (2D-GQPCA) method, which is the generalization of two-directional two-dimensional PCA((2D) PCA) method [15]. Different from the well-known PCA-like methods, the improved 2D-QPCA (2D-GQPCA) method prefers to the components that have larger variances of projected samples and weight them with larger factors. Our M2D-QPCA method simplifies 2D-GQPCA method, greatly reduces the amount of computation and slightly improves accuracy. The paper is organized as follows. In Section 2, we elaborate on the principle of the 2D-QPCA method, state 2D-GQPCA method and propose a modified color 2DPCA method (M2D-QPCA) for face recognition. In Section 3, two experiment is presented and verifies the efficiency of our method. Finally, the conclusion is presented in Section 4.

The Modified Color 2DPCA Method: M2D-QPCA
In this section we briefly introduce the relationship between the quaternion and the color image, elaborate on the principle of the 2D-QPCA method and propose the modified color two-dimensional quaternion principal component analysis (M2D-QPCA) method along the framework of the color two-dimensional quaternion principal component analysis (2D-QPCA) method [12]. We also state the 2D-GQPCA method [14] and compare it with our method.

Quaternion Matix and Color Image
In 1843, William Rowan Hamilton found the quaternion, which has the following form: where are real and are three imaginary units stasfying The set of all quaternions is denoted by . The conjugate of is defined as and the modulus is defined as If the real part is zero we call as the pure quaternion, which can just represent a pixel of the RGB color space and R, G, B stand for the values of Red, Green, Blue components, respectively. So, an color image can be saved as an pure quaternion matrix with the nonnegative integer matrix and .

2.2.2D-QPCA and M2D-QPCA
Now we propose the modified color two-dimensional principal component analysis (M2D-QPCA) method for face recognition using quaternion model, which is different from the 2D-QPCA method presented by Jia, et al. [12].
Let denote an matrix with unitary column vectors. Our idea is to project color image onto by the following linear transformation [16,17]: Thus, we obtain an projected matrix , which is called the projected feature image of image . The total scatter of the projected samples is used to determine a good projection matrix . That is, the following criterion is adopted: where denotes the covariance matrix of the projected feature images of the training samples and denotes the trace of . The physical significance of maximizing the criterion in (2)  We compute the average image and the color image covariance (scatter) matrix (QCM) of training samples by (3) and (4) The aim of 2D-QPCA is to find a set of unitary projection basis vectors , where is often called the eigenface subspace, such that, when projected onto , the projected samples (5) of have the maximal scatter. , which maximizes the trace of the generalized total scatter criterion , meets this requirement. The columns of are the eigenvectors (called eigenfaces) of corresponding to the first largest eigenvalues. The procedures of 2D-QPCA [12] are given in Table 1.  (3) and (4). (vii) Output .
In Algorithm 2D-QPCA and other PCA-like methods, although eigenvalues are different, eigenvectors are treated equally. This has led to some unreasonable phenomena. For example, in Example 1 of Jia, et al. [12], as the dimension of the eigenface subspace increases, the accuracy of face recognition decreases. We believe that the contribution of each eigenvector is different and proportional to its corresponding eigenvalue. As the dimension of the eigenface subspace increases, some unimportant eigenvectors are added equally into the eigenface subspace. As a result, the eigenface subspace becomes more and more inappropriate. Relative examples can be found at the end of the next section.
Let's take another example. We know that the spaces are the same. But when projected onto and , the projected images of the unit circle are the unit circle and the ellipse, respectively. As shown in Figure 1, the unit circle is magnified differently in the same direction. In order to overcome this shortcoming, we modify 2D-QPCA method, weight every eigenvector with the corresponding eigenvalue and derive our M2D-QPCA method, which differ from Algorithm 2D-QPCA in Step (iii)-(v) and is abbreviated as Table 2. Test image for recognition.
Output: such that belongs to class .
(i) Compute the average image and the quaternion covariance matrix of the training set by (3) and (4).
(ii) Perform QED for the eigenface subspace: with .
(iii) For given , take the weighted matrix and the eigenface subspace (iv) Compute the feature matrix for (v) Compute the feature matrix for the test image .
(vi) Find the nearest feature matrix satisfying .
In addition, from (1) and Step (vi), we think that (5) should be modified to (6) the number of operations is reduced and the result is not changed.
Output: such that belongs to class . (ix) Output .
These two methods have the following differences.
(1) 2D-GQPCA method takes two projection directions: column direction and row direction. Our M2D-QPCA method only chooses row direction and removes (i) and (ii) of 2D-GQPCA method. So, the computation of the quaternion covariance matrix in our M2D-QPCA method is simpler than that in 2D-GQPCA method.
(2) The feature matrix in our M2D-QPCA method is different from that in 2D-GQPCA method and does not need to subtract the average image .
(3) Our M2D-QPCA method does not need to compute within-class covariance matrices and their eigendecompositions. So compared with 2D-GQPCA method, our M2D-QPCA method greatly reduces the amount of calculation. The following examples show that the face recognition accuracy of our M2D-QPCA method is not lower than that in 2D-GQPCA method.
So, in this section, we test M2D-QPCA method by the same face database and only compare with 2D-QPCA method and 2D-GQPCA method. Furthermore, for a fair comparison, 2D-GQPCA method also choose one projection direction: row direction. In fact, in the examples in [14], the authors did choose a direction.
All experiments in this section are performed on a personal computer with 3.2 GHz Intel Core i5-6500 and 16 GB 2400 MHz DDR4 using MATLAB-R2018b and Quaternion toolbox for Matlab(QTFM 2.6) [20].   In 2D-QPCA method, when the number of eigenfaces is 1 or 2, the face recognition accuracy is the maximum 92%. In M2D-QPCA method, when the numbers of eigenfaces are from 1 to 4, the face recognition accuracy increases from the minimum 92% to the maximum 95%.
From Figure 2, we can see that our M2DPCA method can reach a higher recognition accuracy than the 2D-QPCA method, and the accuracy doesn't decrease as the dimension of the eigenface subspace increases. In the 2D-QPCA method, when the number of eigenfaces is 1 or 2, the face recognition accuracy is the maximum 92%. In the M2D-QPCA method, when the numbers of eigenfaces are from 1 to 4, the face recognition accuracy increases from the minimum 92% to the maximum 95%.   Table 4 and Table 5, we can also see that the accuracy is significantly improved as the training set increases, and the average recognition time has great advantages when only one eigenface is used.
Example 2: The color Face Recognition Technology database (FERET)(version 2, DVD2, thumbnails) contains 269 persons, 3528 color face images, and each person has various numbers of face images with various back-grounds. The minimal number of face images for one person is 6, and the maximal one is 44. The size of each cropped color face image is pixels. We choose 219 persons with 10 views per individual as samples. Some samples are shown in Figure 4.   Figure 5, we can also see that our M2D-QPCA method can reach a higher recognition accuracy than 2D-GQPCA method.
In this example, the maximal eigenvalue of every within-class covariance matrix reaches and Step (ii) of 2D-GQPCA method overflows. So 2D-GQPCA method cannot work. We must pre-process these eigenvalues to ensure the operation of this method. This phenomenon also tells us that, if the eigenvalues differ greatly, 2D-GQPCA method may have some problems.
The difference in CPU time of 2D-GQPCA method and M2D-QPCA method is mainly in (i)-(iv) of the former and (i)-(ii) of the latter. Table 6 gives the average CPU time obtained from multiple experiments. We can see that our M2D-QPCA method has a big advantage in runtime, which is consistent with the previous analysis.

Choice of Weight
At the end of this section, we discuss the effects of different weighting methods of eigenvectors on the accuracy of face recognition. We know that the importance of the eigenvector can be accurately measured by its corresponding eigenvalue and in the QED of M2D-QPCA method, the diagonal elements of are successively decreasing. So the importance of each column(eigenvector) of is also decreasing. In Figure 6, where is the diagonal matrix with diagonal elements and is the diagonal matrix with diagonal elements , we can see that, when the weight coefficient of the eigenvector in constructing the eigenface space is its corresponding eigenvalue, the accuracy is the best, and when the weight coefficient is decreasing, equal and increasing, the accuracy is getting worse and worse. Furthermore, the faster the weight coefficient increases, the worse the accuracy. Based on these observations, we propose our M2D-QPCA method.