Overfitting

Overfitting occurs when a machine learning model learns the training data too precisely, memorising specific examples rather than learning general patterns. This results in excellent training performance but poor generalisation to new data.

Signs of overfitting:

• Training accuracy much higher than validation accuracy
• Performance degrades on new data
• Model makes confident but wrong predictions
• High variance in predictions

Causes of overfitting:

• Insufficient data: Not enough examples to learn patterns
• Too complex model: More capacity than needed
• Training too long: Model starts memorising
• Noisy data: Learning noise as signal

Prevention techniques:

• More data: Expand training set
• Regularisation: L1/L2 penalties, dropout
• Early stopping: Stop when validation peaks
• Cross-validation: Test on multiple data splits
• Simpler model: Reduce capacity
• Data augmentation: Create training variations