Solved – Measuring correlation of trained neural networks

Question

I am training an artificial neural network (backpropagation, feed-forward) with non-normal distributed data. Beside the root mean squared error, literature does often suggest the Pearson correlation coefficient for evaluating the quality of the trained net. But, is the Pearson correlation coefficient reasonable, if the training data is not normally distributed? Would it not be more reasonable, to use a rank-based correlation measure, e.g. Spearman rho?

Answer 1

The Pearson correlation coefficient measures linear association. Being based on empirical second central moments, it is influenced by extreme values. Therefore:

• Evidence of nonlinearity in a scatterplot of actual-vs-predicted values would suggest using an alternative such as the rank correlation (Spearman) coefficient;

  • If the relationship looks monotonic on average (as in the upper row of the illustration), a rank correlation coefficient will be effective;

  • Otherwise, the relationship is curvilinear (as in some examples from the lower row of the illustration, such as the leftmost or the middle u-shaped one) and likely any measure of correlation will be an inadequate description; using a rank correlation coefficient won't fix this.

• The presence of outlying data in the scatterplot indicates the Pearson correlation coefficient may be overstating the strength of the linear relationship. It might or might not be correct; use it with due caution. The rank correlation coefficient might or might not be better, depending on how trustworthy the outlying values are.

(Image copied from the Wikipedia article on Pearson product-moment correlation coefficient.)