Describing your system for forming a composite
You have weighted a set of items using a standard system (i.e., equal distance between numbers for categories [0,1,2,3,4,5], the same scale for each item; items scores then summed to form a scale score).
I would not call the above system "subjective".
It is probably the most common system for scoring psychology and social science multi-item scales when items use the same response scale.
I imagine that you are contrasting such a scoring system with one based on a factor analysis or a related procedure.
Validity of your composite
You are adopting a standard scoring system, and there are good reasons why this system is so common.
- It's very easy to communicate to others how the scoring system works, and therefore can readily be applied in multiple contexts.
- It is not specific to a given sample (in contrast to weightings derived from a factor analysis; although such weightings could be fixed in one sample and applied in others).
- Many scales are designed so that each item is designed to measure the scale and therefore summing over these items is designed to measure the construct.
- Because each item is on the same scale (and the number of scale points is relatively small), the standard deviation for items tend to be fairly similar, and thus the contribution of the item to the scale total tends to be fairly similar.
Nonetheless, such a scoring system is predicated on the idea that each item is a good measure of the underlying construct.
The broader issue of validity relates to whether the scale you have created is a valid measure of whatever it is meant to be a measure of.
There are a wide range of procedures that people use to establish the validity of a given scale both in general, and for a specific sample.
- Factor analysis and reliability analysis are two obvious ways to assess the internal structure of a scale or set of scales.
- Correlating the scale with other measures is another strategy.
Much more could be said about validity and scale construction (see here for some references).
Whether to convert items to Z-scores
Standardising items first before creating a scale mean or sum is one simple way of ensuring that each item has equivalent "importance" in forming a composite.
The problem of unequal importance is more of a problem when combining component variables that are on very different scales (e.g., height in mm, weight in kg).
There are also issues in comparability across studies when adopting a sample specific z-score approach.
In your case, whether you convert each item to a z-score first before summing items or whether you just sum items, my guess is that the two variables will correlate very highly (perhaps greater than .95, but you can check this).
In general the simple sum (or mean) of raw scores is preferable from a comparability perspective. It also communicates how the mean relates to the underlying scale (e.g., a mean of 4.2 on a 1 to 5 scale where 5 means very satisfied indicates that the sample is generally satisfied, whereas the sum of z-scores does not).
Best Answer
Regarding your question about the composite score being greater than the last category, I think it is important to differentiate between the outcome space of your items (in this case 0 to 4) and your "scale".
In the case of your example, the procedure for aggregating the scores is producing a 'scale' that goes from 0 (when a person answers the four questions as 0,0,0,0) and 5.28 (when the answers to the questions are 4,4,4,4). [Note: I noticed that your questions indicates that you are aggregating answers across persons. I am not sure about the context of your analysis, but usually Likert scales are aggregated within subjects.]
In sum, the outcomes in the questions need not coincide with the scale that is created by aggregating them, however, this might make the interpretation of the aggregated scale more difficult. As a final note, it is worth considering that you are assuming that there is an equal spacing between the response categories, not a trivial assumption, and the aggregation method that you are using assumes this interval scale in the response categories.