A loss function, also known as a cost function, is a mathematical representation of the difference between predicted values and actual outcomes in machine learning models. It quantifies the model's performance, guiding the optimization process during training. Common loss functions include mean squared error for regression tasks and cross-entropy loss for classification tasks.