I'm reading the 3rd edition of Sakurai and Napolitano's Modern Quantum Mechanics, which (probably rightly) relegates wave mechanics to an appendix. Instead, it carefully develops Dirac's formalism, and uses it to expose quantum mechanics. Because notation is at the heart of this exposition, I want to be sure I get it exactly right.
The authors note that the usual wave function corresponding to a state $\left|\alpha\right>$ in the position basis is given by $\psi_{\alpha}(x') = \langle x' | \alpha\rangle$, and that, given two kets $\left|\alpha\right>$ and $\left|\beta\right>$, the expectation value of some operator depending on position $f(x)$ is given by
\begin{equation*}
\left<\beta\right|f(x)\left|\alpha\right> = \int{\mathrm{d}x'\,\langle \beta |x'\rangle f(x')\langle x' | \alpha\rangle} = \int{\mathrm{d}x'\,\psi^{\ast}_{\beta}(x')f(x')\psi_{\alpha}(x')}.
\end{equation*}
Later, the familiar problem of the infinite square potential well is treated as an exercise, and the usual wave functions are
\begin{equation*}
\psi_{n}(x') = \langle x | n\rangle = \sqrt{\frac{2}{a}}\sin{\left(\frac{n\pi}{a}x'\right)}.
\end{equation*}
Now suppose I want to calculate $\left<x\right>$. I know the way to calculate it is
\begin{equation*}
\left<x\right> = \int{\mathrm{d}x'\psi^{\ast}_{n}(x')x'\psi_{n}(x')},
\end{equation*}
but by the notation developed earlier, we should be able to write
\begin{equation*}
\left<x\right> = \left<\beta\right|x\left|\alpha\right>.
\end{equation*}
Here (at long last) is my question: in this case, what are the kets $\left|\alpha\right>$ and $\left|\beta\right>$? Do I have a tragic misunderstanding of the notation?
Edit: the authors denote the inner product of $\left|\alpha\right>$ and $\left|\beta\right>$ as $\langle \alpha | \beta\rangle$. Therefore, in writing $\langle x' | n\rangle$, they seem to suggest the existence of two kets $\left|x'\right>$ and $\left|n\right>$. Maybe my confusion is here.
Here is an excerpt from the text:
Best Answer
An expectation value is defined for a single state and an operator. For state $|\alpha\rangle $, the expectation value of the position operator $\hat x$ is given by $$ \langle \hat x\rangle _\alpha \equiv \langle \alpha| \hat x |\alpha\rangle. $$
You are not calculating an expectation value if two different states are involved. If two different states are involved as in $\langle \alpha|\hat x|\beta\rangle$, with $|\alpha\rangle \neq |\beta\rangle$, it is common to call it a transition element, coupling element or off diagonal matrix element, depending on context, since it is basically a probability amplitude for a transition between the two involved states, mediated by the operator.
The state is often suppressed when it is clear which system and set of states is considered due to context. The expectation value is then abbreviated as $\langle \hat x\rangle $, leaving out the explicit state we are referring to. But when you expand this you have to use the same state for the bra and ket.
I would also like to add that your expansion is not the most general form. It is correct for operators that admit a diagonal representation in position representation but generally you should insert two resolutions of the identity,
$$ \langle \alpha|\hat O |\alpha \rangle = \int\int dx dx' \langle \alpha|x\rangle \langle x|\hat O|x'\rangle \langle x'|\alpha\rangle = \int\int dx dx' \bar \alpha(x')O(x',x)\alpha(x) $$ Most common operators are diagonal in position representation or local and you have $O(x',x)= O(x)\delta(x-x')$ which enables you to perform one integral. This simplifies the expression to the common form $$ \langle \alpha|\hat O |\alpha \rangle = \int\int dx dx' \bar \alpha(x')O(x',x)\alpha(x)=\int dx \bar \alpha(x) O(x) \alpha(x) $$
Remark due to comment:
It is valid to evaluate the operator first and then calculate the inner product. For example, $\hat f|\beta \rangle =|\gamma\rangle$, such that $$ \langle \alpha| \hat f|\beta \rangle = \langle \alpha|\gamma\rangle = \int dx \langle \alpha|x\rangle \langle x|\gamma\rangle = \int dx \bar \alpha(x)\gamma(x) $$ Then one resolution of identity suffices.
EDIT:
I see now where your confusion stems from. The notation used in the book is rather unlucky in my opinion since does not make the distinction between the position operator and its eigenvalues and its representation explicit. Lets take equation 1.246. $$ \langle \beta| f(x)|\alpha \rangle = \int dx' \beta(x')f(x') \alpha(x') $$ where I use $\langle x|\beta \rangle = \beta(x)$ instead of $\psi_\beta(x)$.
$f(x)$ on the left-hand side from your excerpt is a function of the position operator, so I would prefer to denote it as $f(\hat x)$, for example $f(\hat x)= \hat x^2$. When you switch to the position representation, a function of the position operator simply goes to a function of $x$, you can proof this by Taylor expansion. I.e. $f(\hat x)\rightarrow f(x)$. I would have written the expression as $$ \langle \beta |f(\hat x)|\alpha \rangle = \int dx' \beta(x')f(x') \alpha(x') $$
I would recommend taking a look at a second book that is more verbose.