If we already have let's say mean squared error as a loss function which can tell how good our algorithm is, then why we calculate the expectation of loss function as Risk?
Apologies, if this a naive question since i am new to machine learning.
Solved – why do we calculate risk when we already have loss functions
loss-functionsmachine learningrisk
Related Solutions
This is a fairly interesting and novel question! At a formal level, using the frequentist risk function $$(\mathbb{E}_\theta[\hat\theta(X)]-\theta)^2+k\mathbb{E}_\theta[(\hat\theta(X)-\mathbb{E}[\hat\theta(X)])^2],$$ means using (for instance) the loss function defined as $$L(\theta,\hat{\theta})=(\mathbb{E}_\theta[\hat\theta(X)]-\theta)^2+k(\hat\theta-\mathbb{E}_\theta[\hat\theta(X)])^2$$ since there is no reason to prohibit expectations like $\mathbb{E}_\theta[\hat\theta(X)]$ to appear in a loss function. That they depend on the whole distribution of $\hat{\theta}(X)$ is a feature that may seem odd, but the whole distribution is set as a function of $\theta$ and the resulting loss is thus a function of $\theta$, $\hat{\theta}$ and the distribution of $\hat{\theta}(X)$.
I can perfectly forecast an objection coming that a loss function $L(\theta,\delta)$ is on principle a function of a state of nature, $\theta$, and of an action, $\delta$, taking place for instance in the parameter space $\Theta$, hence involving no distributional assumption whatsoever. Which is correct from a game theory perspective. But given that this is statistical decision theory, where a decision $\delta$ will depend on the observation $x$ of a random variable $X$, I see no reason why the generalisation where the loss function depends on the distribution of $X$, indexed by $\theta$, could not be considered. That it may violate the likelihood principle is not of direct concern for decision theory and does not prevent the formal derivation of a Bayes estimator.
MSE has some desirable properties such as easier differentiability (as @user2974951 comments) for further analysis. Differentiability of objective function is in general very important to perform analytical calculations. Taking absolute values is called Mean Absolute Error (MAE in short). It also has applications. It's not like we always prefer MSE or MAE. Another reason, might be penalising large errors more, because if your error is large, its square is much larger. For example, if some error term, $e_i$ is 999, and the other, $e_j$, is $50$; and if we are to choose which term to decrease by an amount of $1$, MAE can choose any of them. But, MSE aims at the larger one since the square decrease is higher.
Best Answer
The risk in machine learning context is the expectation of the loss function over the random variable.
Let's say you have a random variable $x$. If you apply the loss function $L(.)$ to this random variable you get another random variable $y=L(x)$. The expectation is not a function, it's rather an operator $E[.]$, when you apply it on the random variable $y$ you get the constant value $\mu=E[y]$.
So, the result of applying a loss function is the transformation of the random input into another random input. When you get the expectation of that new (transformed) random variable, you get back a number -- risk.