1)Can you still describe the ACF of the time series as cutting of despite the spikes around lag 26?
26 and 27 suggest to me that the data is weekly some sort of annual cycle pf order 26 or 52
Are these outliers an indicator that a mixed ARMA model might be more appropriate?
If there are outliers in the observed series then the ARIMA model becomes a Transfer Function Model with dummy inputs.
Outliers in the acf/pacf are usually non-interpretable. Rathe use the acf/paf of a tentative model suggested by the dominant acf/pacf abd then ITERATE to a more complex model.
Which Information Criterion should I choose? AIC? AICC?
The residuals of the three models with the highest AIC do all show white noise behavior, but the difference in the AIC is only very small. Should I use the one with the fewest parameters, i.e. an ARIMA(0,1,1)?
None as it is based upon a trial set of assumed models.
Is my argumentation in general plausible?
Vague question ... even vaguer response.
Are their further possibilities to determine which model might be better or should I for example, the two with the highest AIC and perform backtests to test the plausibility of forecasts?
Simply ITERATE (slowly !) to more/less complicated models incorporating both auto-regessive structure and determinstic structure. See http://www.autobox.com/cms/index.php/blog/entry/build-or-make-your-own-arima-forecasting-mode for a logic flow diagram
EDIT AFTER RECEIPT OF DATA:
I was misled by your comment , you used the word lag of 26 and I incorrectly understood you were talking about the acf but you were talking about time point 26. A data set can be non-stationary in a number of ways. If the mean shifts the remedy for this non-stationarity is de-meaning . In your case the non-stationarity is caused by two separate and distinct trends and one significant increase in error variance. Both of these findings are easily supported by the eye.
Your data has non-stationarity but the remedy for your data's non-stationarity in the mean is not differencing but de-trending as two trends are found (1-29 and 30-65 ) found via Intervention Detection. Furthermore your error variance is non-stationary significantly increasing at period 28 found via Tsay's test for non-constant error variance, See this reference for both procedures http://www.unc.edu/~jbhill/tsay.pdf . After adjusting for the two trends and error variance change and a few pulses, a simple AR(1) model was found to be adequate. Here is the plot of Actual/Fit/Forecast . The equation is here with estimation results here
. The variance change test is here and the plot of the model's residuals is here . I used AUTOBOX a piece of software that I have helped develop to automatically separate signal from noise. Your data set is the "poster boy" for why simple ARIMA modelling is not widely used because simple methods don't work on complex problems. Note well that the change in error variance is not linkable to the level of the observes series thus power transformations such as logs are not relevant even though published papers present models using that structure. See Log or square-root transformation for ARIMA for a discussion on when to take power transformations.
Best Answer
The acf and pacf suggest possible non-ivertibility perhaps due to the wrong fixup for non-stationarity. See Box & Jenkins Table A , Chart B & C for invertiblity requirements. I have seen similar plots ( an I have seen a lot of plots ! ) often suggesting incorrect differencing . Rather than differencing there might be a need for de-meaning based upon either a change in trend in the original series or a change in level (intercept change). If you post your data I will try and help you further. Note that if a series changes level the acf suggests non-stationarity not necessarily remedied by differencing. To prove this to yourself simulate a wn series with a mean shift .. then examine it's acf