In the context of Quantum Mechanics and Hilbert spaces, I understand that a function can be interpreted as $\psi(x) = \langle x \vert \psi \rangle$ in the position basis, and things like $$\int_a^b|\psi(x)|^2dx$$make sense interpreting $x$ as a label. But if I think of $\psi(x)$ as an element of $L^2$ and expand $\psi(x)$ in a discrete basis, like $\psi(x)=\sum_a(\phi_a(x),\psi(x))\phi_a(x)$, now the label is $a$ and what does $x$ means now and why the inner product is still the same and does not make any reference to $a$?
Quantum Mechanics – Discrete and Continuous Basis in Quantum Mechanics
discretehilbert-spacenotationquantum mechanics
Related Solutions
The state is a vector in the Hilbert space of the Hamiltonian, which gives it a natural basis in terms of the eigenvectors; distinct eigenvalues then exist in distinct (orthogonal) subspaces - for degenerate values the subspaces are larger, but they are still distinct from all others. Clearly this situation gives many advantages in analysis.
However, this representation, though natural, and dependent only upon the spectral decomposition of the Hamiltonian, is not unique. The experimentalist will prefer to establish a basis which corresponds to his experimental setup! Chosing a different basis will change all of the coordinates, but does not change the states.
To make this clear recall that the state vectors of our Hilbert space can also be viewed a as rays, which are similar to the geometric rays of Euclidean geometry. Imagine the ray first, then superimpose a grid upon it - as you rotate the grid about the origin of the ray the intersections of the grid with the ray define the coordinates for that basis. The coordinates change, but the ray doesn't change. For ordinary geometry the relationships between two rays (angles, projections) don't change, so it is clear that some relationships between the coordinates are fixed - these are properties of a metric space.
Our Hilbert space is a normed space - distances don't mean anything, but a normalized state vector always has a length of 1, even when you change the basis; nor does the length change under unitary operators - hence their name.
All of this becomes clear in a good linear algebra course.
$\lvert A\rangle \langle B \rvert$ is the tensor of a ket and a bra (well, duh). This means it is an element of the tensor product of a Hilbert space $H_1$ (that's where the kets live) and of a dual of a Hilbert space $H_2^\ast$, which is where the bras live. Although for Hilbert spaces their duals are isomorphic to the original space, this distinction should be kept in mind. So we can "feed" a ket $\lvert \psi\rangle$ from $H_2$ to the bra in $\lvert \phi\rangle\otimes \langle\chi\rvert \in H_1\otimes H_2^\ast$, and are left with a state in $H_1$ given by $\langle \chi \vert \psi\rangle \lvert \phi\rangle$. The usual use case for such a tensor product is when $H_1=H_2$ to construct a map from $H_1$ to itself, e.g. the projector onto a state $\lvert \psi \rangle$ is given by $\lvert\psi\rangle \langle \psi \rvert$.
In general, a tensor in $H_2 \otimes H_1^\ast$ corresponds to a linear operator $H_1\to H_2$. In the finite-dimensional case, these are all linear operators, in the infinte-dimensional case, this is no longer true, e.g. $H^\ast \otimes H$ are precisely the Hilbert-Schmidt operators on $H$.
In constract, a tensor $\lvert A\rangle\otimes \lvert B\rangle$ (also just written $\lvert A \rangle \lvert B\rangle$) in $H_1\otimes H_2$, although it corresponds to a bilinear map $H_1\times H_2\to\mathbb{C}$ by definition, is usually not meant to denote an operator, but a state. Given two quantum systems $H_1$ and $H_2$, $H_1\otimes H_2$ is the space of the states of the combined system (as for why, see this question).
Best Answer
When you represent a periodic function by a Fourier series you do pretty much the same thing: you represent it by sines instead of δ-functions.
For your specific example, you have two alternate resolutions of the identity/ completeness relations, $$ 1\!\!1= \int\!\! dx ~|x\rangle\langle x| \\ 1\!\!1= \sum_a |a\rangle \langle a| $$ where, importantly, $\langle a|x \rangle = \phi_a^* (x)$, your change of basis "matrix".
You then have $$ \psi(x)=\langle x|\psi\rangle = \sum_a \langle x|a\rangle \langle a|\psi\rangle = \sum_a \phi_a(x) ~~\psi_a, ~~\hbox{where}\\ \psi_a=\langle a|\psi\rangle= \int\!\! dx ~ \phi^*_a(x)~ \psi(x), $$ by the above completeness relation. In your expression, you used the same x as a variable, and as a dummy integration variable in your inner product--a disastrous practice.
It is crucial to test-drive this equivalence with the Hermite functions $\psi_n(x)=\langle x|n\rangle$, where the discrete index is the natural number identifying the quantum oscillator energy level, if you have not already done so.
Useful link as per your comment.