I am looking for a good book for regression that is mainly focused on mathematical stuff instead of applied parts. Basically, I want a book that contains all the proofs and mathematical descriptions related to regression analysis in a purely mathematical way.
I looked for recommendations given for other related quarries but I couldn't find any good book dedicated to regression in a pure mathematical way, I mean contains all the proofs and mathematical relations between different concepts.
These are some topics I am mainly interested in :
- Simple Linear Regression (Review estimation)
- Test of the parameters
- Validation of the model and its assumptions ($R^2$ value, test of the model,
residual analysis) - Multivariate Normal and related results
- Multiple Linear Regression
- Estimation of the parameters
- Projection revisited
- Tests for the parameters
- Validation of the model and its assumptions ($R^2$, adjusted-$R^2$, test of the
model, residual analysis) - Variable selection (forward, backward, AIC)
- Multicollinearity (Ridge-regression)
Note: I know it might be possible that a single book does not cover all topics in detail but I am fine with reading a different book for different topics.
Best Answer
I would recommend Seber & Lee (from which I originally learned regression.) Cover most of your topics with proofs. An alternative in the same style, but also covering glm's is Linear Models and Generalizations : Least Squares and Alternatives by Rao et al.
A shorter book with a more geometric viewpoint is The Coordinate-Free Approach to Linear Models by Michael J. Wichura, but it will not cover all your topics.