I'm modeling growth curve of several ecosystems with respect to their rainfall-productivity relationship using a simple linear regression
$\text{ANPP}(t)=a+b\cdot\text{Rain}(t)$
and a modified version of the Brody Model
$\text{ANPP}(t)=a\cdot\left[1-e^{(-b\cdot\text{Rain}(t))}\right]$,
i.e. for each ecosystem, the two models are fitted to observed data and compared to each other.
I would like to know why the $R^2, AIC$ and $BIC$ criteria disagree for many ecosystems about the "best model" knowing that the Brody and the the linear regression are both 2-parameter models (i.e. in this case the complexity is not an issue for $R^2$)?
In the above formula, $\text{Rain}(t)$ denotes the accumulated rainfall from time to to time t and to compute the $R^2$, I used the following formula:
r2 <- mss/(mss+rss)
mss <- sum((fitted(model) - mean(fitted(model)))^2)
rss <- sum(resid(model)^2)
My guess is that the models needs to have the same form (which is not the case here: linear form vs exponential form) ) or nested so that their model selection with frequentist or Bayesian approaches such as the $AIC$ and $BIC$ criteria will give the same best result with a simple maximizing fit approach such as the $R^2$.
Best Answer
I would avoid the question altogether as secondary. The issue is the choice between two models, a linear fit and an exponential approach to an asymptote. The linear model may possibly be an acceptable fit depending partly on the range of your data, but a negative intercept implies negative productivity for zero rainfall and a positive intercept implies positive; the first is impossible and the second implausible; nevertheless the model might be tolerable with warnings.
Conversely, the exponential is evidently chosen to have the limiting behaviour of approaching the origin, but how well it fits in practice depends on whether the idea of an asymptote is correct and (simply, but crucially) on the impact of numerous other controls on productivity.
The central question of which model fits better should be clear from plotting data and model fits. Plotting residuals as well would do no harm.
There is scope for disagreement here, but I prefer to approach $R^2$ as the square of the correlation between observed and predicted. This is totally consistent with its use in linear regression and relatively easy to explain to non-statistical users. Either way, be aware that outside linear regression the many different definitions of $R^2$ may not agree.
There is some minor confusion in your question. Whether people use any kind of $R^2$, AIC or BIC does not really stamp them as frequentist or Bayesian. There could be qualifications and details attached to that, but arguments over frequentist and Bayesian approaches to inference arguably have no bearing on your question.
(LATER) It has become apparent that the observations are points on two curves, productivity so far this season and rainfall so far this season. I suppose this was implicit in the term "growth curve" but it was not obvious to me and causes severe qualifications to my original comments.
Cumulation imparts a dependence structure which renders any inferential machinery for $R^2$, AIC and BIC invalid, unless somehow calculations took the cumulation into account. Thus all inferential bets are off unless the cumulation is corrected for in the model assessment. (I've seen allusion to this problem of correlating cumulative curves in the glaciology literature.)
I was supposing that (rainfall, productivity) pairs for each location were from different seasons, which still leaves the possibility of season to season dependence, which in practice is likely to be weaker. To spell out the point, it seems that each location is being characterised by data from one season.
@Armel expresses disagreement with my statement that a curve of the form $ a\cdot(1-\exp[-b\cdot\text{Rain}(t)])$ can not be S-shaped, but does not give any reasons. I am referring to the shape of the curve on a plot with Rain on one axis; the point then is one of elementary calculus that such a curve has no inflexion. Here what I understand by S-shaped is that an inflexion exists at which curvature changes sign. (The relationship between productivity and time could be much more complicated geometrically, depending on the dependence of cumulative rainfall on time.)