PCA and SVD TODO
Concept
SVD decomposes an arbitrary real matrix A as A = UΣVᵀ, where U and V are orthogonal matrices and Σ is a diagonal matrix holding the nonnegative singular values in decreasing order. Geometrically, this means any linear transformation can be viewed as the composition of 'rotation (or reflection) → axis-wise scaling → rotation'; the size of the singular values indicates how much each direction is stretched, and the number of singular values close to zero indicates how rank-deficient the matrix is. The truncated SVD keeping only the top k singular values has the property (Eckart–Young) of being the optimal rank-k approximation under the Frobenius norm, which is the theoretical basis for dimensionality reduction and noise removal. PCA is the procedure of centering the data by column means and then finding the eigenvectors of the covariance matrix — in the SVD of the centered data matrix, the columns of V are exactly the principal components, and the squared singular values are proportional to the variance explained by each component. In other words, PCA is the statistical interpretation of SVD, and forgetting to center the data is the most common practical pitfall that makes the two results diverge.
Compressing high-dimensional metrics or extracting the dominant axes of variation from correlated signals comes up repeatedly in anomaly detection, risk decomposition, and feature extraction, and reading condition number and rank deficiency is also a basic tool for diagnosing numerical instability.
Code & Formula
# PCA·SVD — 상관된 2D 합성 데이터에서 주성분(최대 분산 방향)을 SVD로 직접 구한다.
import numpy as np
rng = np.random.default_rng(0)
n = 300
# x, y가 강하게 상관되도록 만든 2D 데이터 (주된 퍼짐 방향이 대략 45도가 되게)
t = rng.normal(0, 3, n)
x = t + rng.normal(0, 0.3, n)
y = t * 0.6 + rng.normal(0, 0.3, n)
X = np.column_stack([x, y]) # shape (n, 2)
X_centered = X - X.mean(axis=0) # PCA는 평균을 원점으로 옮긴 뒤 분산 방향을 찾는다
U, S, Vt = np.linalg.svd(X_centered, full_matrices=False)
print(f"특이값(Σ): {S}")
print("주성분 방향(V의 행, 분산이 큰 순서):\n", Vt)
pc1 = Vt[0]
angle_deg = np.degrees(np.arctan2(pc1[1], pc1[0]))
print(f"\n제1주성분(PC1) 방향 벡터 = {pc1}, x축 대비 각도 ≈ {angle_deg:.1f}도")
# 데이터를 PC1 축 하나에 투영 (2D -> 1D 차원축소)
projected = X_centered @ pc1
explained_var_ratio = (S ** 2) / np.sum(S ** 2)
print(f"\nPC1 하나로 설명되는 분산 비율 = {explained_var_ratio[0]:.4f}")
print(f"투영된 1D 값 예시(앞 5개): {np.round(projected[:5], 3)}")
Exercise
Generate synthetic 2-3 dimensional data with strong correlation, center it and compute the SVD, derive the variance explained by each principal component from the squared-singular-value ratios, and compare how the results differ if you skip centering.
Practical Connection
Arranging multiple markets' price time series as a matrix and extracting principal components can separate a common factor like 'overall market direction' from movements unique to individual markets, which can be used for anomalous price detection and gauging risk concentration.
If you study this on a given day, add a note link and a ✅ to this line in the source curriculum (docs/knowledge/math-50-curriculum.md) and this spot will lead straight to the note body. You can also write directly on this page — but regenerating overwrites it, so it's safer to keep anything you want to save as markdown under docs/algorithms/.