I have a study variable that is measured at the individual level and then aggregated and to the group level. My hypothesis testing is conducted at the group level.
I used an 8-item referent-shift measure of team psychological capital.
When calculating and reporting the coefficient alpha, should I be looking at it across individual level scores, or averaging (aggregating) each item across each team, and then calculating the coefficient alphas across aggregated item scores.
That is, I can find the average value of item 1 within team 1, then the average value of item 2 within team 1 etc. I will then have one value for each scale item per team. I will then calculate the alpha based on n=94 teams. Alternatively, I could leave everything at the individual level, and calculate it that way across all individuals within my sample (n=456 individual team members).
I would think I should measure it at the group level, since I am conducting my analyses as the group level. That is, I should be most concerned about the level of measurement error in the aggregated team-level scale, right? If I calculate it at the team level, I get alpha=0.92, and if I calculate it at the individual level, I get alpha=0.86. At the end of the day, they're both good alphas, but I still want to report the correct number.
Also, in case this is important in answering the question,
I have calculated ICC(1), ICC(2) and rwg for the scale means (rather than on an item by item basis) and they are acceptable for aggregating the scale means to the group level.
Thank you in advance!
Note: Referent-shift measure means that the referent of the scale is the team rather than the individual. So the individual-referent psychological capital item might be I am optimistic about my work. The team-referent psychological capital item would be Members of my team are optimistic about their work. See Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Chan, David Journal of Applied Psychology, Vol 83(2), Apr 1998, 234-246.
Edit: While the scale is measured at the individual level, analyses are conducted at the group level (after aggregation). Hence my confusion over whether I should report individual level alpha or group level alpha… Group level alpha would be found by aggregating each individual item (find average within-group value for each item and then calculate alpha based on that).
Best Answer
I've seen people reporting the two. I don't think there's one golden standard. The alpha at the team level is not too informative, as it is much important to see the ICC2.