Losses act as feedback, showing how far our predictions are from reality. The overall cost, which is a combination of these individual losses, measures the quality of our data analysis.

In this article, we will delve into three fundamental types of losses, each with its unique characteristics and applications. Together, we will uncover hidden stories in data using algorithms.

Mean absolute error: Imagine you're trying to predict the weight of different objects. For each object, you weigh it on a scale and record the actual weight. Then, you use a machine learning model to predict the weight of each object based on its features.

Mean Absolute Error (MAE) is a way to measure how accurate your predictions are. It calculates the average absolute difference between your predicted weights and the actual weights. In other words, it measures how far off your predictions are, on average.

The lower the MAE, the more accurate your predictions are. A perfect MAE of 0 would mean that your predictions were always exactly correct. However, a perfect MAE is not realistic, so a low MAE is generally considered good.

Here is the formula for MAE:

$$MAE = \frac{1}{n} \sum_{i=1}^{n} |y_i - \hat{y}_i|$$

• n is the number of datapoints

• y i is the actual value for the i-th data point

• ŷi is the predicted value for the i-th data point

• In summary you calculate the absolute difference between the predicted and actual values and then average these absolute differences over all data points.

Mean squared error:

Imagine you're a carpenter and you're trying to build a perfectly level table. To assess the levelness of the table, you use a spirit level to measure the angle of the tabletop at different points. Mean Squared Error (MSE) is like a tool that helps you quantify the overall levelness of the table.

The lower the MSE, the more level the table is, indicating better craftsmanship. A perfect MSE of 0 would mean that the table is perfectly level at every point, while a higher MSE indicates that there are more variations in the levelness of the table.

MSE measures the average squared difference between the actual angle of the tabletop at different points and the desired angle of 0 degrees. By squaring the differences, MSE gives more weight to larger deviations, making it sensitive to outliers. This means that if there are a few points on the table that are significantly off-level, MSE will be higher.

Here's the formula MSE:

$$MSE = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2$$

• The squared differences in the formula have the effect of penalizing larger errors more heavily than smaller ones. This makes MSE sensitive to outliers, meaning that outliers can have a significant impact on the overall metric.

• Since the differences are squared, the units of MSE are squared units of the original data. This can make interpretation more challenging, especially when comparing models on different scales

• MSE is a differentiable function, which is important for optimization algorithms used in training machine learning models, such as gradient descent

• Similar to MAE, lower MSE values indicate better model performance, and an MSE of 0 represents a perfect fit where the predicted values exactly match the actual values.

Binary cross entropy: often referred to as BCE or log loss, is a loss function commonly used in binary classification problems. It measures the difference between the actual and predicted probability distributions for a binary outcome.

In the context of machine learning and binary classification, let's break down the components of Binary Cross Entropy:

1. Binary classification refers to problems where there are only two possible classes or outcomes.

2. In binary classification, the model predicts the probability of an instance belonging to the positive class (e.g., spam, fraudulent) and the complementary probability of belonging to the negative class. These probabilities are usually denoted as P(y) for the positive class and P(¬y) for the negative class.

3. The actual labels are binary (0 or 1), indicating whether an instance belongs to the positive class (1) or the negative class (0).

4. The Binary Cross Entropy loss for a single instance is calculated using the following formula:

$$BCE(y, \hat{y}) = - \left(y \cdot \log(\hat{y}) + (1 - y) \cdot \log(1 - \hat{y})\right)$$

• “y” is the actual predicted model

• “ŷ” is the predicted probability of belonging to the positive class

• The formula essentially penalizes the model more if the predicted probability diverges from the actual label.

1. For a dataset with multiple instances, the average Binary Cross Entropy loss is calculated:

$$BCE(y, \hat{y}) = - \frac{1}{n} \sum_{i=1}^{n} \left(y_i \cdot \log(\hat{y}_i) + (1 - y_i) \cdot \log(1 - \hat{y}_i)\right)$$

• y is the vector of actual labels.

• ŷ is the vector of predicted probabilities

• n is the number of instances in the dataset

Now we've learned about three important tools that data scientists use: mean squared error, binary cross-entropy loss, and mean absolute error. These tools help us make accurate predictions.

Now that we have learned about losses and costs, we are ready to tackle data science problems. We will use what we have learned to turn raw data into useful information that can help us make better decisions.