Key takeaways
- Modern deep learning algorithms excel in various tasks, surpassing traditional machine learning methods.
- Larger models are easier to optimise, often leading to global minima in training risk.
- Over-parameterised models typically outperform classical models on test sets.
- Despite fitting complex models to data with minimal training error, they can still show good accuracy on unseen data.
- Understanding the bias-variance trade-off remains crucial in machine learning.

Overview Of Bias-Variance Trade-Offs In ML
Source: elitedatascience
Bias is the difference between the model's expected predictions and the true values and variance refers to the algorithm's sensitivity to specific sets of training data. Bias vs. variance refers to the accuracy vs. consistency of the models trained by the algorithm.
To build a good predictive model, one needs to find a balance between bias and variance that minimises the total error because:
Total Error = Bias^2 + Variance + Irreducible Error
An optimal balance of bias and variance leads to a model that is neither overfitting nor underfitting. The predictor is commonly chosen from linear functions or neural networks with a certain architecture, using empirical risk minimisation (ERM) and its variants.
Bridging The Old And New

Evolution On Other Fronts





