How it works
Data is split into train, validation and test. The model learns from the training set, hyperparameters are tuned with validation, and only at the end is it evaluated on the test set, which must remain untouched until then. This simulates performance on new data and avoids excessive optimism.
Practical example
You have 10 thousand examples. Use 7 thousand for training, 1.5 thousand for validation and 1.5 thousand for test. The model never sees the 1.5 thousand test examples until the final evaluation, ensuring an honest measurement.
What it's for
It provides an honest estimate of performance. If you use the test set to adjust the model, you lose the ability to know how it behaves in production and you can overestimate quality.
Best practices
Keep the test set untouched until the end. Avoid looking at it during development. If you need to iterate, use the validation set and reserve the test set only for final evaluation.