For finite dimensional spaces, the answer is "yes"; this is a consequence of the Gram-Schmidt orthonormalization process: every finite dimensional inner product space over $\mathbb{R}$ or over $\mathbb{C}$ has an orthonormal basis.
Now let $\mathbf{V}$ be an inner product space, and let $\mathbf{v}_1,\ldots,\mathbf{v}_n$ be an orthonormal basis. Then $T\colon\mathbf{V}\to \mathbf{F}^n$ given by $T\mathbf{v}_i = \mathbf{e}_i$ (i.e., $T$ maps each vector in $\mathbf{V}$ to its coordinate vector relative to the orthonormal basis $\mathbf{v}_1,\ldots,\mathbf{v}_n$) is an invertible linear transformation such that for all $\mathbf{x},\mathbf{y}\in\mathbf{V}$, $\langle \mathbf{x},\mathbf{y}\rangle = T\mathbf{x}\cdot T\mathbf{y}$, where the right hand side is the standard dot product on $\mathbf{F}^n$ ($\mathbf{F}=\mathbb{R}$ or $\mathbb{C}$).
The bra-ket notation "in the abstract" represents a Hermitian inner product on the Hilbert space of states, where by definition linearity holds in second argument and swapping the arguments conjugates the result: $\langle \phi | (z+w) \psi \rangle=z\langle \phi | \psi \rangle+w\langle \phi | \psi \rangle$ and $\langle \phi | \psi \rangle=\overline{\langle \psi | \phi \rangle}$. This implies $\langle (z+w) \phi | \psi \rangle=\overline{z+w}\langle \phi | \psi \rangle$, so the product is conjugate-linear in the first argument. (Note that this is a matter of convention; mathematicians often choose the Hermitian inner product to be linear in the first entry and conjugate-linear in the second one. This does not affect things in any substantial way, but I think both your sources use the "linear in second entry" convention, so we'll stick to that).
This explains the blue question mark in (3).
In the case when $\psi$ and $\phi$ are column vectors in some $\mathbb{C}^n$, the "usual" or "standard" inner product given by $\langle \phi | \psi \rangle= \sum_{j=1}^n \overline{\phi_j}\psi_j$ (where each of the components $\phi_j$ and $\psi_j$ is a complex number). This has the properties above, that is, it's an example of a Hermitian inner product (making $\mathbb{C}^n$ into a Hilbert space). It can also be written in matrix notation as $\langle \phi | \psi \rangle=\bar{\phi}^T\psi=\psi^T\overline{\phi}$.
Suppose now that $A$ is a (bounded, linear) operator on our Hilbert space then its Hermitian conjugate $A^\dagger$ is DEFINED by $\langle \phi, A \psi \rangle=\langle A^\dagger \phi, \psi \rangle$ (this is uniquely defined since knowing product of $A^\dagger \phi$ with all vectors specifies $A^\dagger \phi$ uniquely, and one can check that the resulting map $A^\dagger$ is linear and bounded). Thus your equation (1) is a definition of $A^\dagger$.
We have $(XY)^\dagger=Y^\dagger X^\dagger$. Indeed, from definitions $\langle (XY)^\dagger u, v\rangle=\langle u, XYv \rangle=\langle X^\dagger u, Yv \rangle=\langle Y^\dagger X^\dagger u, Yv \rangle$ for all $u,v$, so $(XY)^\dagger=Y^\dagger X^\dagger$. This extends to products of more than one operators, so that, for example, $(ABCD)^\dagger=D^\dagger C^\dagger B^\dagger A^\dagger$.
If the Hilbert space is $\mathbb{C}^n$, then dagger becomes "conjugate transpose" operation, since it is easy to check that in that case $\langle \phi, A \psi \rangle=\overline{\phi}^T A\psi=(A^T\overline{\phi})^T\psi=\langle \overline{A}^T \phi, \psi\rangle$.
The bra-ket notation is then extended to be $\langle \phi | A|\psi\rangle=\langle \phi | A\psi\rangle$ by definition. We have from above that this equals $\langle A^\dagger \phi | \psi\rangle$.
Unitary matrices can be defined EITHER as ones with $\langle U\phi| U\psi\rangle=\langle \phi| \psi \rangle$ OR as ones for which $U^\dagger=U^{-1}$. It is immediate from definition of $\dagger$ that these are equivalent.
Now all the operations in your question marks are explained:
Suppose we have an expression like $\langle\phi |ABCD|\psi\rangle$. By definition this is $\langle\phi |ABCD\psi\rangle$. This is also equal to $\langle\phi|ABC |D\psi\rangle$ or $\langle\phi|AB |CD\psi\rangle$ etc. This means that we can move the righttmost operator inbetween-bars to the right. It is also, as we noted above, by the definition adjoint, equal to $\langle (ABCD)^\dagger \phi |\psi\rangle$. As established before, this is the same as $\langle D^\dagger C^\dagger B^\dagger A^\dagger \phi |\psi\rangle$. and the same as $\langle (BCD)^\dagger A^\dagger \phi |\psi\rangle=\langle A^\dagger \phi |(BCD)\psi\rangle=\langle A^\dagger \phi|BCD |\psi\rangle$. This means that we can move the lefttmost operator inbetween-bars to the left with a dagger.
If the operator is unitary, then the dagger becomes inverse.
As a final note, when $\psi$ and $\phi$ are not finite-dimensional, but, instead lie in the space of complex-valued $L^2$ functions (on some space $X$) then the Hermitian product is the L^2 inner product $\langle \phi| \psi \rangle=\int_X \overline{\phi(x)}\psi(x) dx$ and our extended definition becomes $\langle \phi|A|\psi\rangle=\int_X \overline{\phi(x)}A(\psi(x)) dx$ agreeing with the formula in your post.
Best Answer
In order to do what you suggest on the elements of a set, such as a set of vectors, the set must be closed under the operation in question. Real numbers are closed under the operations of addition and multiplication so we can speak of repeated addition and repeated multiplication in that context. But the set of vectors is not closed under the dot-product operation, so repeated dot products do not make much sense in that context.