Linear Algebra — 28.1: The Singular Value Decomposition
Every matrix, square or rectangular, factors as A = UΣV^T. This video builds the SVD from a single geometric question: does there exist an orthonormal basis that A maps to another orthonormal basis? Starting from the spectral theorem for symmetric matrices and generalizing to arbitrary linear maps, we derive the SVD, interpret it geometrically as rotation-stretch-rotation, and clarify what singular values really measure.
Key concepts covered:
• Why a generic orthonormal basis fails to stay orthogonal under a matrix A
• The spectral theorem A = QΛQ^T as the symmetric positive definite baseline
• Why eigenvectors break down for non-symmetric or rectangular matrices
• The SVD factorization A = UΣV^T and the role of each factor
• The defining equation Av_i = σ_i u_i linking two orthonormal bases
• Deriving A = UΣV^T from AV = UΣ using V^(-1) = V^T
• How rank, the null space, and zero singular values fit into the decomposition
• Recovery of the spectral theorem as the symmetric positive definite special case
• Geometric interpretation: unit circle to ellipse via rotation, stretch, rotation
• Distinction between singular values and eigenvalues; σ_i as square roots of eigenvalues of A^T A
• Why the SVD is called the final factorization, and its role in PCA, image compression, and the pseudo-inverse
━━━━━━━━━━━━━━━━━━━━━━━━
SOURCE MATERIALS
The source materials for this video are from • 29. Singular Value Decomposition