On page 63 Table 3.3. of "The Elements of Statistical Learning" by Hastie el al. (2017) 12th print. It gives the summary results of different regression results on the cancer data. However I don't understand how the std err in the last line is calculated (esp. for the LS column).
The authors on page 61 have the following explanations: "The last two lines of the table give the average prediction error (and its estimated standard error) over the test set."
Still could anyone please help me understand how we get the std err in the last line especially the LS column? Thanks!
Best Answer
Looks like to me that a 10-fold cross validation was used where the dataset was split into 10 equal parts and 9 parts was used to train the model and test on the last part. Repeating this for each of the 10 parts would produce 10 sets of prediction error.
The "Test error" in the table refers to the average of the 10 cross validation errors which can be calculated this way:
For each tuning parameter $\theta$ in the model,
$CV(\theta) = \frac{1}{10} \sum\limits_{k=1}^{10} CV_k(\theta)$
where $CV_k(\theta) = \frac{1}{|F_k|} \sum\limits_{i\in F_k} (y_i -\hat{f_\theta} ^{-k}(x_i))^2 $
The "Standard Error" refers to the error made on the cross-validation estimate which can be obtained by first calculating the SD of the 10 CV errors, and estimating the SE by correcting by a factor:
$SD(\theta)=\sqrt{var(CV_1 (\theta),CV_2(\theta),...,CV_{10}(\theta))}$
$SE(\theta) = \frac{SD(\theta)}{\sqrt{10}}$
Note that this can be generalised to a $K$-fold CV by replace $10$ by $K$ in the formula. Hope this helps!