Understanding loss functions is fundamental to mastering machine learning. In supervised learning, a loss function is a mathematical tool that quantifies how well your model's predictions match the true values. Formally, given a true value $y$ and a predicted value $\hat{y}$, a loss function is often written as $L(y, \hat{y})$. This function measures the "cost" or "penalty" for making an incorrect prediction, providing a numeric value that reflects the error for each prediction made by the model.

Loss functions serve as the bridge between model predictions and the optimization process. During training, machine learning algorithms use the loss function to compute gradients, which in turn determine how model parameters are updated. By minimizing the loss function, the algorithm seeks to improve prediction accuracy. Importantly, loss functions are distinct from evaluation metrics: while loss functions provide a training signal for optimization, evaluation metrics (such as accuracy or F1 score) are used to assess model performance after training. Both are central to the machine learning workflow, but they play different roles in guiding and measuring model success.