First of all is need to say that for prediction evaluation, then out of sample, the usual $R^2$ is not adequate. It is so because the usual $R^2$ is computed on residuals, that are in sample quantities.
We can define: $R^2 = 1 – RSS/TSS$
RSS = residual sum of square
TSS = total sum of square
The main problem here is that residuals are not a good proxy for forecast errors because in residuals the same data would be used for both, model estimation and model prediction accuracy. If residuals (RSS) are used the prediction accuracy would be overstated; probably overfitting occur. Even TSS is not adequate as we see later. However we have to say that in the past the mistaken use of standard $R^2$ for forecast evaluation was quite common.
The out of sample $R^2$ ($R_{oos}^2$) maintain the idea of usual $R^2$ but in place of RSS is used the out of sample MSE of the model under analysis (MSE_m). In place of TSS is used the the out of sample MSE of one benchmark model (MSE_bmk).
$R_{oos}^2 = 1 – MSE_m/MSE_{bmk}$
One notable difference between $R^2$ and $R_{oos}^2$ is that
$0 \leq R^2 \leq 1$ (if the constant term is included)
while $-\infty \leq R_{oos}^2 \leq 1$
If $R_{oos}^2 < = > 0$ the competing model perform worse/equal/better than the benchmark one. If $R_{oos}^2 =1$ the competing model predict perfectly the (new) data.
Here we have to keep in mind that the even for the benchmark model we have to consider the out of sample performance. Therefore the variance of the out of sample data underestimate $MSE_{bmk}$.
In my knowledge this measure was proposed for the first time in:
Predicting excess stock returns out of sample: Can anything beat the historical average? - Campbell and Thompson (2008) - Review of Financial Studies.
In it the the bmk forecast is based on the prevailing mean given information at time of the forecast.
While this does depend on the theoretical nature of the model, you might want to try using an F-test, where the F statistic is the variation between sample means/variation between the samples
.
This test is used to compare models in order to determine which one can best explain the variation in the dependent variable. You might consider incorporating this test into a one-way ANOVA: Understanding Analysis of Variance (ANOVA) and the F-test
That being said, you mention that you have used three different questionnaires. Be cautious if the number of observations for the three regression models are not equal, in which case the F-test could also be unreliable. e.g. if a model has 100 observations, the F-test could show a lower "fit" than one with 200, but if the number of observations for the first model had been increased, then it could in fact have the best fit.
You could also compute the power of a test for your three samples - i.e. identification of the minimum number of observations that would be needed for your results to be reliable. If your sample for the three models is shown to be large enough, then tests such as the F-test would be more reliable.
Best Answer
For adjusted $R^2$ the answer is yes.
$$R^2_{adj} = 1- \dfrac{n-1}{n-p}\dfrac{SSRes}{SSTotal}$$
(Some sources may write the denonominator of $R^2_{adj}$ as $n-p-1$. That assumes that $p$ does not consider the intercept term. In the book I used, the author assumes that $p$ does consider the intercept term.)
If you do a regression on 100 observations and get $\frac{SSRes}{SSTotal} = 0.8$, then if you do the same regression on 200 observations and also get $\frac{SSRes}{SSTotal} = 0.8$, you will have a larger $R^2_{adj}$.
For $R^2$, the answer is no...sort of. $R^2$ is something like correlation between variables, and the relationship between quantities doesn't depend on how many times you observe the variables. It doesn't depend on you observing the variables at all! However, as you play with data, you're not feeding the same values into the formula to calculate $R^2$, so you won't end up with the same value for the same reason that you won't end up with the same $\bar{x}$ on 99 observations as you would on 100, though you have no way to know if adding that 100th observation will increase or decrease the value you're calculating. With adjusted $R^2$, you know that, all else being equal, more observations means a larger value.
What happened in your example is that the smaller sample happened to get lucky and find a strong relationship, but when you included more data, that relationship turned out not to be so strong.
Keep in mind that you're very unlikely to find that your $\frac{SSRes}{SSTotal}$ ratio is the same for two different samples, even if they're drawn from the same population. This is true whether the sample sizes are the same or different.