I hope someone can help with this. I am a statistician looking for a good book on high dimensional probability and data analysis. Basically I am looking for the equivalent of Terry Tao's 2 volume set on Analysis, but for high dimensional probability. Let me qualify what I am looking for.
Now there are a bunch of books out there with these very words in the title. I will list some below. But most of these are geared towards just pure machine learning folks or computer science. So often books on high dimensional data focus on techniques like Principle Components Analysis or Lasso, etc., to analyze high dimensional data. In developing these models, the authors start off with strong parametric assumptions about exponential family distributions or independence, etc. These book lack any sort of organic development of a theory behind adding dimensions to a data set or changes in the patterns of symmetry as a data set grows larger–both in dimensions and in the number of observations.
A basic probability text book will begin with a definition of random variables and work its way towards the Central Limit Theorem. While that is good for an intro stats course, there are a lot of problems with assuming normality even in high dimensional situations.
An example of such a book is:
Geometric Structure of High-Dimensional Data and Dimensionality Reduction
Statistics for High-Dimensional Data: Methods, Theory and Applications
(Please note that I am not critizing any of the books mentioned. I am just saying that these books don't fit my particular need.)
So what I am looking for is a more analytic look at how probability varies as dimensions get rather high. I use Terry Tao's book as an example of a wonderful development of analysis from basic foundations. I am looking for the same treatment for high dimensional data. I am not sure if I should be looking at a book on measure theory, or calculus on manifolds, or where?
Any suggestions would really be appreciated.
Best Answer
It has been some time since the question was asked and I believe the recent text "High Dimensional Probability" by Roman Vershynin might be just what you are looking for. It covers a variety of topics, focusing on the concentration phenomenon on higher dimensions. It is both theoretically heavy and intuitionally approachable.
Its current draft is also available for free, in the author's webpage. https://www.math.uci.edu/~rvershyn/papers/HDP-book/HDP-book.html#