Skip to content

Concept List

This document contains 300 concepts for the Applied Linear Algebra for AI and Machine Learning course.

Part 1: Foundations of Linear Algebra

Chapter 1: Vectors and Vector Spaces

  1. Scalar
  2. Vector
  3. Vector Notation
  4. 2D Vector
  5. 3D Vector
  6. N-Dimensional Vector
  7. Vector Addition
  8. Scalar Multiplication
  9. Vector Subtraction
  10. Dot Product
  11. Cross Product
  12. Vector Magnitude
  13. Unit Vector
  14. Vector Normalization
  15. Euclidean Distance
  16. L1 Norm
  17. L2 Norm
  18. L-Infinity Norm
  19. Linear Combination
  20. Span
  21. Linear Independence
  22. Linear Dependence
  23. Basis Vector
  24. Standard Basis
  25. Coordinate System
  26. Vector Space
  27. Dimension of Space

Chapter 2: Matrices and Matrix Operations

  1. Matrix
  2. Matrix Notation
  3. Matrix Dimensions
  4. Row Vector
  5. Column Vector
  6. Matrix Entry
  7. Matrix Addition
  8. Matrix Scalar Multiply
  9. Matrix-Vector Product
  10. Matrix Multiplication
  11. Matrix Transpose
  12. Symmetric Matrix
  13. Identity Matrix
  14. Diagonal Matrix
  15. Triangular Matrix
  16. Upper Triangular
  17. Lower Triangular
  18. Orthogonal Matrix
  19. Matrix Inverse
  20. Invertible Matrix
  21. Sparse Matrix
  22. Dense Matrix
  23. Block Matrix

Chapter 3: Systems of Linear Equations

  1. Linear Equation
  2. System of Equations
  3. Matrix Equation Form
  4. Augmented Matrix
  5. Gaussian Elimination
  6. Row Operations
  7. Row Swap
  8. Row Scaling
  9. Row Addition
  10. Row Echelon Form
  11. Reduced Row Echelon Form
  12. Pivot Position
  13. Pivot Column
  14. Free Variable
  15. Basic Variable
  16. Solution Set
  17. Unique Solution
  18. Infinite Solutions
  19. No Solution
  20. Homogeneous System
  21. Trivial Solution
  22. Numerical Stability
  23. Back Substitution

Chapter 4: Linear Transformations

  1. Function
  2. Linear Transformation
  3. Transformation Matrix
  4. Domain
  5. Codomain
  6. Image
  7. Rotation Matrix
  8. 2D Rotation
  9. 3D Rotation
  10. Scaling Matrix
  11. Uniform Scaling
  12. Non-Uniform Scaling
  13. Shear Matrix
  14. Reflection Matrix
  15. Projection
  16. Orthogonal Projection
  17. Composition of Transforms
  18. Kernel
  19. Null Space
  20. Range
  21. Column Space
  22. Rank
  23. Nullity
  24. Rank-Nullity Theorem
  25. Invertible Transform
  26. Change of Basis
  27. Basis Transition Matrix

Part 2: Advanced Matrix Theory

Chapter 5: Determinants and Matrix Properties

  1. Determinant
  2. 2x2 Determinant
  3. 3x3 Determinant
  4. Cofactor Expansion
  5. Minor
  6. Cofactor
  7. Determinant Properties
  8. Multiplicative Property
  9. Transpose Determinant
  10. Singular Matrix
  11. Volume Scaling Factor
  12. Signed Area
  13. Cramers Rule

Chapter 6: Eigenvalues and Eigenvectors

  1. Eigenvalue
  2. Eigenvector
  3. Eigen Equation
  4. Characteristic Polynomial
  5. Characteristic Equation
  6. Eigenspace
  7. Algebraic Multiplicity
  8. Geometric Multiplicity
  9. Diagonalization
  10. Diagonal Form
  11. Similar Matrices
  12. Complex Eigenvalue
  13. Spectral Theorem
  14. Symmetric Eigenvalues
  15. Power Iteration
  16. Dominant Eigenvalue
  17. Eigendecomposition

Chapter 7: Matrix Decompositions

  1. Matrix Factorization
  2. LU Decomposition
  3. Partial Pivoting
  4. QR Decomposition
  5. Gram-Schmidt QR
  6. Householder QR
  7. Cholesky Decomposition
  8. Positive Definite Matrix
  9. SVD
  10. Singular Value
  11. Left Singular Vector
  12. Right Singular Vector
  13. Full SVD
  14. Compact SVD
  15. Truncated SVD
  16. Low-Rank Approximation
  17. Matrix Rank
  18. Numerical Rank
  19. Condition Number

Chapter 8: Vector Spaces and Inner Product Spaces

  1. Abstract Vector Space
  2. Subspace
  3. Vector Space Axioms
  4. Inner Product
  5. Inner Product Space
  6. Norm from Inner Product
  7. Cauchy-Schwarz Inequality
  8. Orthogonality
  9. Orthogonal Vectors
  10. Orthonormal Set
  11. Orthonormal Basis
  12. Gram-Schmidt Process
  13. Projection onto Subspace
  14. Least Squares Problem
  15. Normal Equations
  16. Row Space
  17. Left Null Space
  18. Four Subspaces
  19. Pseudoinverse

Part 3: Linear Algebra in Machine Learning

Chapter 9: ML Foundations

  1. Feature Vector
  2. Feature Matrix
  3. Data Matrix
  4. Covariance Matrix
  5. Correlation Matrix
  6. Standardization
  7. PCA
  8. Principal Component
  9. Variance Explained
  10. Scree Plot
  11. Dimensionality Reduction
  12. Linear Regression
  13. Design Matrix
  14. Ridge Regression
  15. Lasso Regression
  16. Regularization
  17. Gradient Vector
  18. Gradient Descent
  19. Batch Gradient Descent
  20. Learning Rate

Chapter 10: Neural Networks and Deep Learning

  1. Perceptron
  2. Neuron Model
  3. Activation Function
  4. ReLU
  5. Sigmoid
  6. Tanh
  7. Softmax
  8. Weight Matrix
  9. Bias Vector
  10. Forward Propagation
  11. Backpropagation
  12. Chain Rule Matrices
  13. Loss Function
  14. Cross-Entropy Loss
  15. Neural Network Layer
  16. Hidden Layer
  17. Deep Network
  18. Convolutional Layer
  19. Convolution Kernel
  20. Stride
  21. Padding
  22. Pooling Layer
  23. Batch Normalization
  24. Layer Normalization
  25. Tensor
  26. Tensor Operations

Chapter 11: Generative AI and Large Language Models

  1. Embedding
  2. Embedding Space
  3. Word Embedding
  4. Semantic Similarity
  5. Cosine Similarity
  6. Attention Mechanism
  7. Self-Attention
  8. Cross-Attention
  9. Query Matrix
  10. Key Matrix
  11. Value Matrix
  12. Attention Score
  13. Attention Weights
  14. Multi-Head Attention
  15. Transformer Architecture
  16. Position Encoding
  17. LoRA
  18. Latent Space
  19. Interpolation

Chapter 12: Optimization and Learning Algorithms

  1. Hessian Matrix
  2. Convexity
  3. Convex Function
  4. Newtons Method
  5. Quasi-Newton Method
  6. BFGS Algorithm
  7. SGD
  8. Mini-Batch SGD
  9. Momentum
  10. Adam Optimizer
  11. RMSprop
  12. Lagrange Multiplier
  13. Constrained Optimization
  14. KKT Conditions

Part 4: Computer Vision and Autonomous Systems

Chapter 13: Image Processing and Computer Vision

  1. Image Matrix
  2. Grayscale Image
  3. RGB Image
  4. Image Tensor
  5. Image Convolution
  6. Image Filter
  7. Blur Filter
  8. Sharpen Filter
  9. Edge Detection
  10. Sobel Operator
  11. Fourier Transform
  12. Frequency Domain
  13. Image Compression
  14. Color Space Transform
  15. Feature Detection
  16. Homography

Chapter 14: 3D Geometry and Transformations

  1. 3D Coordinate System
  2. Euler Angles
  3. Gimbal Lock
  4. Quaternion
  5. Quaternion Rotation
  6. Homogeneous Coordinates
  7. Rigid Body Transform
  8. SE3 Transform
  9. Camera Matrix
  10. Intrinsic Parameters
  11. Extrinsic Parameters
  12. Projection Matrix
  13. Perspective Projection
  14. Stereo Vision
  15. Triangulation
  16. Epipolar Geometry
  17. Point Cloud

Chapter 15: Autonomous Driving and Sensor Fusion

  1. LIDAR Point Cloud
  2. Camera Calibration
  3. Sensor Fusion
  4. Kalman Filter
  5. State Vector
  6. Measurement Vector
  7. Prediction Step
  8. Update Step
  9. Kalman Gain
  10. Extended Kalman Filter
  11. State Estimation
  12. SLAM
  13. Localization
  14. Mapping
  15. Object Detection
  16. Object Tracking
  17. Bounding Box
  18. Path Planning
  19. Motion Planning
  20. Trajectory Optimization