PCA与SVD区别

PCA与SVD的区别

1.1楼

foreseer201

初级会员

注册于: 2006/12/29

发帖数: 95

近期看到这方面的内容，发现两者之间的区别：

PCA对原数据的协方差矩阵（中心化后转置相乘）进行特征值分解，得到一组基底组成的矩阵。然后原数据与之相乘得到在这组新基底下的投影数据，进行后续处理。。。

对协方差矩阵的分解，直观上理解为两个目的：降低相关性、保留方差大的样本数据。

但是SVD直接分解的是原数据。虽然形式上与PCA相似（并且同时得到两个方向的PCA），但是不像PCA有那么直观的意义。仅是从矩阵运算的形式上降维了原数据。

区别在于SVD得到的U阵和V阵是原数据转置相乘后的特征向量组，而PCA却是对其协方差矩阵特值分解。

那么是不是说，SVD分解后的U、V阵靠前的分量可以直接使用？而不必像PCA的需要投影？

刘版那个“禁军教头林书豪”的例子中，就是直接使用的U阵前两个分量。

1 年前回复#回复

2.2楼

肖楠

版主

注册于: 2009/10/31

发帖数: 2,794

如果不考虑数据的均值，SVD 完全等价于PCA。

SVD 对稀疏数据用得更多一些。

1 年前回复#回复

3.3楼

ltx5151

新手上路

注册于: 2012/05/24

发帖数: 6

PCA and SVD are equivalent in certain sense, so no need to worry their differences. Assume X is the standardized data matrix, and the columns are variables, rows are samples. SVD of X is $$X =

UDV^T$$, where the matrix $$UD$$ is just the principal component matrix that is used commonly, and if you just use the first k columns of $$UD$$, then it is the first k principal components. Actually, the implementation of most PCA functions in software is using SVD. PCA is an eigen decomposition of covariance matrix (note that this is the singular value decomposition for the covariance matrix, but this SVD is not the SVD we usually mean. When we say SVD, we normally refer the the SVD of X). PCA has its certain interpretations such as the largest variance directions. So one can argue that PCA focus more on covariance matrix while SVD focus more on the data itself. But there is no fundamental differences between PCA and SVD. From a matrix viewpoint, SVD are best low rank matrix

approximation in Frobenius norm, as well as as the low dimensional hyperplane approximation to the data cloud, which give much wider applications than PCA. As mentioned by nan.xiao, SVD is also widely used for sparse matrix or matrix completion with missing values, which is based on its

interpretation which is NOT reflected by PCA.

1 年前回复#回复

4.4楼

foreseer201

初级会员

注册于: 2006/12/29

发帖数: 95

Thanks for both reply.

"PCA focus more on covariance matrix while SVD focus more on the data itself." It was my confusion between PCA and SVD before.

"From a matrix viewpoint, SVD are best low rank matrix approximation in Frobenius norm, as well as as the low dimensional hyperplane approximation to the data cloud".

Thus to some degree UD(first k columns) are equivalent XP(X is the data itself, P is the eigen

decomposition result of covariance matrix. Also take the first k columns) in the respect of

application...