Lets pretend I have no experience with statistics…
what books, in what order, will get me to the point of being able to understand Bayesian probability fastest.
Books for Understanding Bayesian Probability from the Beginning
probabilityreference-request
Best Answer
The folks at LessWrong have put together a couple of resources that can be helpful for the complete beginner. They are also useful if you've only taken a single course in probability and basically just memorized formulas to get through it.
An Intuitive Explanation of Bayes' Theorem
Bayes' Theorem Illustrated (My Way) (note, this isn't written by me; that's just the title).
For two more advanced books that cover practical matters in great detail (and require a bit more mathematical maturity) see:
Bayesian Data Analysis by Gelman, Carlin, Rubin, and Stern. (This is a very very good reference for questions on model checking, simulational approaches, and ignorability of experimental designs.)
Doing Bayesian Data Analysis by Kruschke.
My own opinion is that learning the formula for Bayes' theorem is elementary; it's something that almost always precedes even a first introduction to common discrete random variables. The real "starting point" for seeing the difference in Bayesian methods is when you start needing to compute Maximum Likelihood Estimators (MLE). Then you can directly compare the output of an MLE with the alternative output if you had set up a prior and chased the computation out all the way to a posterior distribution.
There's nothing special about MLEs, of course, but that just happens to be one of the first places where a student is really forced to say, "Gee, I'd really like to have the whole posterior distribution, but it's really hard to compute." Once a student sees the value in that, they are usually ready to start learning about acceptance/rejection, Metropolis-Hastings, and Gibbs sampling, which are the gateway drugs of Bayesian analysis.