What are Overfitting and Generalisation?

What is Overfitting?

Overfitting is when a model memorises details and noise specific to its training data instead of general patterns. The training score keeps rising while the validation score stalls or drops.

Generalisation

The real goal is generalisation: working well on data the model has never seen. In industrial projects that means a new field, a new machine or next month.

Countermeasures

• Early stopping: stop training when the validation score has not improved for a set number of epochs.
• Data augmentation: enrich the data with transforms such as rotation and brightness changes.
• More, and more diverse, data: often the most effective fix.

Data Leakage

If information from training leaks into the test data, the score looks better than it really is. A common mistake in imaging projects is tiling a large orthophoto and splitting the tiles at random: neighbouring tiles are almost identical. The fix is to split the test by geographic blocks or by time.

Class Imbalance

If one class has very few examples (e.g. a handful of almond trees against hundreds of walnuts), that class scores low. This is a data problem, not a model fault; the fix is collecting more examples of that class.

Related: Model Evaluation Metrics