ML First Principles — Map of Content
Personal branding content map for Suryansh.co. Every note here is a topic to be explained from first principles — for Obsidian Publish, with companion X posts and GitHub code.
Dependency Tree
Linear Algebra Calculus Probability Info Theory Statistics
↓ ↓ ↓ ↓ ↓
Matrices, Derivatives, Distributions, Entropy, Bias-Variance,
Eigenvectors Chain Rule Bayes Theorem Info Gain Overfitting
↓ ↓
Optimization Distance &
+ Grad Desc Similarity
↓
PCA LDA LR GBT CNN Naive Bayes DT RF HMM
Foundations
- Linear Algebra — vectors, matrices, eigenvectors
- Calculus — derivatives, chain rule, partial derivatives
- Probability & Distributions — Bayes theorem, Gaussian, CLT
- Information Theory — entropy, information gain, KL divergence
- Statistics & Bias-Variance — overfitting, cross-validation, hypothesis testing
- Optimization & Gradient Descent — loss surfaces, step size, convergence
- Distance & Similarity — Euclidean, cosine, why metrics matter
Models
- Linear Regression — minimize a quadratic, nothing more
- Naive Bayes — pure Bayes theorem + one bold assumption
- Principal Component Analysis — find the axes of maximum variance
- Linear Discriminant Analysis — PCA that knows the class labels
- Decision Tree — which feature reduces uncertainty the most?
- Random Forest — why averaging uncorrelated models works
- Gradient Boosting Tree — gradient descent in function space
- Convolutional Neural Network — local patterns, shared weights, depth
- Hidden Markov Model — hidden states generating observable sequences
Content Pipeline
| Note | Obsidian | X Post | GitHub |
|---|---|---|---|
| Linear Algebra | First principles write-up | ”A matrix is just a machine that rotates and stretches space” | linear-algebra-visual |
| Optimization & Gradient Descent | First principles write-up | ”Why you move opposite to the gradient” | gradient-descent-viz |
| Statistics & Bias-Variance | First principles write-up | ”Why a perfect training fit is a red flag” | included in model repos |
| PCA | First principles write-up | ”PCA doesn’t select features. It creates new ones.” | pca-lda-from-scratch |
| LDA | First principles write-up | Thread: PCA vs LDA | pca-lda-from-scratch |
| Linear Regression | First principles write-up | ”One derivative. That’s the whole algorithm.” | ml-from-scratch |
| Naive Bayes | First principles write-up | ”The most honest model in ML” | ml-from-scratch |
| Decision Tree | First principles write-up | ”Decision trees are just repeated questions about uncertainty” | ml-from-scratch |
| Random Forest | First principles write-up | Meme: averaging bad models → good model | ml-from-scratch |
| GBT | First principles write-up | Thread: boosting as gradient descent | ml-from-scratch |
| CNN | First principles write-up | ”CNNs don’t understand images. They understand patterns.” | sam-building-footprints |
| HMM | First principles write-up | ”What if the world has hidden states?” | wavelet-denoising-emg |