I want to calculate the difference between a certain value associated with a decision alternative and the value associated with the objectively correct alternative as a measure of decision accuracy in an experiment (e.g. predicted cashflow of certain assets; calculate difference between net cashflow of chosen asset and the asset with the highest net cashflow in the set of alternatives).
To get a standardized measure of accuracy as a dv for my analysis, I was thinking about z-transforming the difference between chosen alternative and objectively correct alternative based on the answers of the experiment's participants. Higher z-values in this case would mean that participants were closer to the correct choice.
My question is, whether this transformation is meaningful and still contains the relevant information about accuracy (i.e. distance betweeen participant's cohioce and 'correct' one. Further I would like to know whether I can use the z-scores in a subsequent multiple linear regression and/or ANOVA/ANCOVA as a dv.
My question is related to this one: regression with z-scores as composite variables? however, I am especially interested whether the z-score are senseful as an accuracy measure.
Thanks a lot!
Best Answer
I would very seldom get away from the raw measurements. z-scores make the interpretation much trickier and have hidden assumptions. First, they assume that the standard deviation is the "gold standard" dispersion measure. This assumes among other things symmetry of the distribution. Second, some z-scores involve a standardization procedure where one subjects an expected value in such a way that linearity is implicitly assumed.
Lastly, when you use z-scores in subsequent analyses you are hiding the uncertainty in the mean and standard deviation used to compute the z-score.