Linear Algebra
First Principles Question
What does it mean to transform space? And why does that matter for data?
The Core Idea
A vector is a point in space. A matrix is a machine that moves points — it can rotate, scale, shear, or project them. Everything in ML that involves “transforming data” is secretly just this.
Key Concepts to Cover
- Vectors and vector spaces — what does it mean to span a space?
- Matrix multiplication as transformation, not just arithmetic
- Determinant — does the transformation preserve volume, or collapse it?
- Eigenvectors — the special directions a matrix doesn’t rotate, only stretches
- Eigenvalues — how much those directions get stretched
- Dot product — angle and projection between vectors
- Column space, null space — where does the transformation send things?
- Pseudo-inverse — what to do when a matrix can’t be inverted
Why It Matters for ML
- PCA: find eigenvectors of the covariance matrix → principal components
- LDA: same, but optimize a ratio of scatter matrices
- Linear Regression: least squares = pseudo-inverse solution
- CNN: convolution is a matrix operation on local patches
Prerequisites
None. This is Layer 0.
Builds To
Principal Component Analysis · Linear Discriminant Analysis · Linear Regression · Convolutional Neural Network
Content Ideas
Obsidian note: “A matrix is a machine. Here’s what it does to space.”
X post: “A matrix is just a machine that rotates and stretches space. That’s it. Everything in ML follows from that.”
GitHub: linear-algebra-visual — NumPy implementations of each transformation with matplotlib plots