What does "data leakage" refer to in machine learning?

Data leakage occurs when information is included in the training dataset that should not be available during the training process, leading to overly optimistic performance metrics for the model. This information could be future data or data derived from the outcome variable that the model is trying to predict, which provides an unfair advantage by allowing the model to learn patterns from data that wouldn't be available at prediction time.

This situation can lead to models that perform exceptionally well on training and validation sets but fail to generalize to new, unseen data, as they have essentially 'cheated' by having access to information that they should not have. Recognizing and preventing data leakage is crucial for building robust and reliable machine learning systems.