Skip to content

STRUCTURE.md

Latest commit

 

History

History
153 lines (138 loc) · 4.34 KB

STRUCTURE.md

File metadata and controls

153 lines (138 loc) · 4.34 KB
Raw

Course Notes Structure

Legends

[ Priority ] - Needs to be completed before semester 1 starts.

Foundations [ Priority ]

  • An Introduction to AI
    • Machine Learning: Overview and Motivation
    • Learning from data
    • Prediction vs Inference vs Decision Making
  • Definitions and Notation
    • Data, features, labels
    • Supervised vs unsupervised vs semi-supervised
    • Classification vs regression
    • Deterministic vs probabilistic models
    • Parameters, hyperparameters, loss functions
  • Probability Theory
    • Random variables
    • Distributions
    • Expectation, variance
    • Bayes’ rule
  • Statistical Inference
    • Estimation (MLE, MAP)
    • Bias – Variance
    • Covarariance Matrix
    • Correlation
    • Confidence intervals and uncertainty
  • Linear Algebra
    • Vectors, matrices
    • Matrix multiplication
    • Linear transformations
    • Eigenvalues, Eigenvectors and SVD
  • Calculus and Optimisation Basics
    • Gradients
    • Derivatives of common functions
    • Convexity
    • A first look at Gradient Descent

Data

  • Data Types and Structures [ Priority ]
  • Exploratory Data Analysis (EDA) [ Priority ]
  • Data Cleaning and Preprocessing [ Priority ]
    • Missing values
    • Normalisation and standardisation
    • Encoding categorical variables (One-hot)
    • Train/validation/test splits
  • Feature Engineering and Selection [ Priority ]
  • Dimensionality Reduction (PCA introduced here conceptually)

Statistical Learning Theory

  • Learning as Function Approximation
  • Loss Functions and Risk Minimisation
  • Empirical Risk Minimisation
  • Generalisation, Overfitting, and Underfitting [ Priority ]
  • Regularisation (L1, L2)
  • Bias–Variance Revisited with Examples [ Priority ]
  • Generalisation Gap [ Priority ]

Supervised Learning [ Priority ]

  • Supervised Learning Framework
  • KNN
  • Decision Trees
  • Linear Regression
    • Closed-form solution
    • Gradient-based optimisation
    • Regularised regression (Ridge, Lasso)
  • Polynomial Regression
    • Regularised regression
  • Classification
    • Logistic regression
    • Softmax regression
  • Evaluation Metrics
    • Accuracy, precision, recall, F1, ROC, AUC
    • Regression metrics (MSE, MAE, R2)
  • Model Selection and Validation
    • Cross-validation
    • Hyperparameter tuning
    • Learning curves

Unsupervised Learning [ Priority ]

  • Clustering
    • K-means
    • Hierarchical clustering
    • Density-based methods (DBSCAN)
  • Dimensionality Reduction
    • PCA
    • Introduction to Kernel PCA
    • PCA with SVD
  • Mixture Models
    • Gaussian Mixture Models
    • Expectation–Maximisation (EM algorithm)

Semi-Supervised Learning

  • Motivating Limited Label Scenarios
  • Self-training
  • Consistency Regularisation
  • Graph-based Semi-supervised Methods

Learning Parametric Models [ Priority ]

  • Parametric vs Nonparametric Models
  • Maximum Likelihood in Parametric Settings
  • Exponential Family Models
  • Generalised Linear Models
  • Second order gradient methods.

Artificial Neural Networks [ Priority ]

  • The Perceptron
  • Multi-layer Perceptrons (MLPs)
  • Activation Functions
  • Backpropagation (Derivation and Examples)
  • Optimisation Algorithms (SGD, Momentum, Adam)
  • Implicit Regularisation in Neural Nets (Dropout)
  • Explicit Regularisation in Neural Nets (Weight Decay)

Deep Learning

  • Representation Learning [ Priority ]
  • Convolutional Neural Networks [ Priority ]
  • Recurrent Networks & Sequence Models
  • Modern Architectures (Transformers - conceptual)
  • Training Dynamics and Practical Tricks [ Priority ]

Ensemble Methods [ Priority ]

  • Bagging and Bootstrap Aggregation
  • Random Forests
  • Boosting
    • AdaBoost
    • Gradient Boosting
    • XGBoost / LightGBM (conceptual mechanics)

Kernel Methods [ Priority ]

  • The Kernel Trick
  • Support Vector Machines
  • Kernel Ridge Regression
  • Kernel PCA (full revisit)

Gaussian Processes [ Priority ]

  • Nonparametric Bayesian Modelling
  • Covariance Functions and Kernels
  • GP Regression [ Priority ]
  • GP Classification [ Priority ]

Generative Models [ Priority ]

  • Generative vs Discriminative Models
  • Latent Variable Models
  • Variational Autoencoders
  • Generative Adversarial Networks (GANs)
  • Normalising Flows

Performance [ Priority ]

  • Model Evaluation Revisited: Robustness & Stability
  • Calibration
  • Fairness, Bias, Ethical Considerations
  • Computational Efficiency (Floating point errors)