The acceptable score

Report a typo

In this task, let's try to understand how to determine whether the MAE score we obtained signals a good or a bad model performance (to start somewhere, we'll keep the threshold simple).

Consider the following dataset (where yy is the ground truth label, and y^\hat{y} is the predicted label):

xx yy y^\hat{y}
0 3.2 4
1 4.7 4.3
2 4 4.6
3 5.6 4.9
4 5.6 5.2
5 6.2 5.5
6 5.1 5.8
7 6.4 6.1

1) Calculate the mean absolute value for this dataset (we will call this value MAEoriginal\text{MAE}_{\text{original}}).

2) Then, calculate the median of the ground truth label.

3) Calculate the MAE for a case where all predictions are equal to the median of the ground truth (so the new dataset the y^\hat{y} column will have the same value, the ground truth median) — we'll call this value MAEbaseline\text{MAE}_{\text{baseline}}.

4) Compare MAEoriginal\text{MAE}_{\text{original}} and MAEbaseline\text{MAE}_{\text{baseline}}. We'll say that MAEoriginal\text{MAE}_{\text{original}} is good if it equals MAEbaseline0.2\text{MAE}_{\text{baseline}} - 0.2 or less.

Your answer function looks like this:

Answer={MAEbaseline+1,if  MAEoriginalis goodMAEbaseline,otherwise\text{Answer}= \begin{cases} \text{MAE}_{\text{baseline}} + 1, & \text{if }\ \text{MAE}_{\text{original}}\, \text{is good} \\ \text{MAE}_{\text{baseline}}, & \text{otherwise} \end{cases}

Enter a number
___

Create a free account to access the full topic