Linear Algebra — 28.4: SVD and the Four Fundamental Subspaces
Why is the Singular Value Decomposition considered the ideal factorization in linear algebra? This video shows how the SVD selects the unique orthonormal basis of the row space that A maps directly onto an orthonormal basis of the column space — producing orthogonality that survives multiplication by A.
We connect the right and left singular vectors to the four fundamental subspaces, derive why eigenvectors of A transpose A give the perfect input basis, and work through a complete 2x2 example to see U, Sigma, and V emerge.
Key concepts covered:
The four fundamental subspaces: row space, null space, column space, and left null space
How the singular vectors v_i and u_i form orthonormal bases for all four subspaces
The central SVD relation: A v_i = sigma_i u_i for i from 1 to r
Why generic Gram-Schmidt bases fail to stay orthogonal after applying A
Choosing v_i as eigenvectors of A transpose A to preserve orthogonality
Singular values as square roots of eigenvalues of A transpose A
Worked example with A = [[3,0],[4,5]], computing eigenvalues, singular values, and both bases
A = U Sigma V transpose as rotate, scale, rotate
Why PCA, low-rank approximation, the pseudoinverse, and least squares all rely on the SVD
━━━━━━━━━━━━━━━━━━━━━━━━
SOURCE MATERIALS
The source materials for this video are from • 29. Singular Value Decomposition