$P_i = \mid\psi_i\rangle\langle\psi_i\mid$ is the one-dimensional "projection" operator. By "one-dimensional" it means this projection operator projects $\psi$ onto a single dimension in Hilbert space.
Firstly, any wavefunction $\psi$ can be written as a linear combination of orthogonal components. That is, $\psi = \sum a_i\mid\psi_i\rangle$ where $a_i$ is some coefficient. If there are $n$ such non-zero coefficients, $\psi$ can be thought of as a vector in $n$ dimensions, having components in each direction of length $a_i$ in this $n-dimensional$ Hilbert space. $a_i$ is also the amplitude that the result $\psi_i$ will be obtained if the wave-function is measured in this basis. The probability is amplitude^2.
The projection measurement essentially "projects" the state $\psi$ onto one of these components.
It is easiest to demonstrate why $P_i = \mid\psi_i\rangle\langle\psi_i\mid$ by applying it to the state $\psi$.
$P_i\psi = P_i\sum a_k\mid\psi_k\rangle = \mid\psi_i\rangle\langle\psi_i\mid\sum a_k\mid\psi_k\rangle = \mid\psi_i\rangle\sum a_k\langle\psi_i\mid\psi_k\rangle = a_i\mid\psi_i\rangle$
Therefore, the operator $P_i$ acting on some arbitrary state $\psi$, projects $\psi$ onto its i-th component vector. (This is analogous to projecting a 2D vector onto say its x-component in Euclidean geometry. For instance if a vector $V = ax + by$, then the projection onto x-axis would yield $V_x = ax$)
So since, $P_i\mid\psi\rangle=a_i\mid\psi_i\rangle$ we can easily show that $\langle\psi\mid P_i\mid\psi\rangle=\sum \langle\psi_k\mid a_k^*a_i\mid\psi_i\rangle = \sum a_k^*a_i\langle\psi_k\mid\psi_i\rangle = a_ia_i^* = \mid a_i\mid^2$
Therefore we have shown that $\langle\psi\mid P_i\mid\psi\rangle$ gives the probability of the wavefunction being in the eigen-state $\psi_i$.
The next step is to show how the one in your book is also the probability. Note your book's use of $P_x$ is to represent probability and is not the projection operator. First consider the denominator $\langle\psi\mid\psi\rangle = \sum^j\sum^k\langle\psi_j\mid a_j^* a_k\mid\psi_k\rangle$. The only terms that survive is when i=j. Therefore we arrive at:
$\langle\psi\mid\psi\rangle = \sum^j\langle\psi_j\mid a_j^* a_j\mid\psi_j\rangle = \sum^j a_j^*a_j = \sum^j \mid a_j\mid ^2 $ This is the total probability of any state which, if this is normalized, should be 1. Therefore $\langle\psi\mid\psi\rangle = 1$ for normalized states, otherwise it is the sum of all possible amplitudes^2.
Next we consider the numerator. This is the dot product of the x-components of the state, which will yield $a_x^* a_x = \mid a_x \mid ^2$ Therefore, numerator over denominator gives $\frac{\mid a_x \mid ^2}{\sum^j \mid a_j\mid ^2}$. This is the probability for a particular state x to occur divided by the probability that any of the possible states will occur (which should be 1 for normalized states).
Yes and no. You can just normalize the results with:
$$ \langle \psi | \psi \rangle$$
but you have computed that incorrectly. Remember, the differential solid angle is:
$$ d\Omega = d(\cos{\theta})d\phi, $$
you have used:
$$ d\Omega = d\theta d\phi.$$
I suggest you verify with a table of $l=1$ spherical harmonics.
Best Answer
You are nearly on the right track. I will give a simple explanation without any mathematical rigour.
When the system is in state $$|\psi\rangle = \int c(\alpha)|v_\alpha\rangle d\alpha$$ then, when measuring $\hat{A}$, the probability of obtaining a value in the range $[\alpha, \alpha + d\alpha]$ is $$dP(\alpha) = \frac{|c(\alpha)|^2 d\alpha}{\langle\psi|\psi\rangle}.$$ Note that the probability $dP(\alpha)$ needs to be a differential (when $d\alpha\to 0$ you also have $dP(\alpha)\to 0$). Therefore it is an equation between infinitesimal quantities. Dividing this equation by $d\alpha$ you get a probability density $$\frac{dP(\alpha)}{d\alpha} = \frac{|c(\alpha)|^2}{\langle\psi|\psi\rangle}.$$