So here is the abstract approach:
$$ \langle \psi | H | \psi \rangle = \frac{1}{5}\bigg( \langle \phi_1 | H | \phi_1 \rangle + 2\langle \phi_1 | H | \phi_2 \rangle + 2\langle \phi_2 | H | \phi_1 \rangle + 4 \langle \phi_2 | H | \phi_2 \rangle \bigg) \,.$$
Now you know that $H|\phi_1\rangle = E_1 |\phi_1 \rangle$ and $H|\phi_2\rangle = E_2 |\phi_2 \rangle$ --- or rather, you can easily check that the functions you've given are indeed eigenstates of the Hamiltonian:
$$ \frac{-\hbar^2}{2m} \frac{d^2}{dx^2} \phi_n = \frac{n^2 \pi^2 \hbar^2}{2m L^2} \phi_n \equiv E_n \phi_n \,.$$
The functions $\phi_n$ are also normalised, as you can check, and are orthogonal to one another --- this must be the case, because they are the eigenfunctions of a Hermitian operator (with different eigenvalues). Hence the expression above becomes:
$$\langle \psi | H | \psi \rangle = \frac{1}{5}\bigg( E_1\langle \phi_1 | \phi_1 \rangle + 2E_2 \langle \phi_1 | \phi_2 \rangle + 2E_2\langle \phi_2 | \phi_1 \rangle + 4E_2 \langle \phi_2 | \phi_2 \rangle \bigg) \,.$$
$$\langle \psi | H | \psi \rangle = \frac{1}{5}\bigg( E_1 + 4E_2\bigg) \,.$$
Substituting in the form of the energies gives:
$$\langle \psi | H | \psi \rangle = \frac{17}{5} \frac{\pi^2 \hbar^2}{2m L^2} \,.$$
This is, to me, easier than computing the integral you've given, although the integral you've given is correct (or almost --- the Hamiltonian should have factor of $\hbar$ squared in front of the second derivative). If you tried to compute the integral, you would find a good deal of cancellation due to the orthogonality of the functions involved. If you're good at quickly spotting when an integral vanishes, e.g.:
$$ \int_0^L \sqrt{\frac{2}{L}} \sin \left(\frac{\pi x}{L }\right) \sqrt{\frac{2}{L}} \sin \left(\frac{2\pi x}{L }\right) \,dx = 0$$
then the above prescription might not seem any simpler. But as far as cleanness of approach goes, it's nicer to invoke the orthogonality of the eigenfunctions --- which is a result of central importance in this kind of problem, and one which you will prove in any introductory QM course --- before delving into explicit computations.
================================================================================
NB: I anticipate that you may not have met the above notation yet. There isn't anything to it. For our purposes, just take $\langle \psi | H |\psi \rangle$ to mean
$$ \int \psi^*(x) H \psi(x) \,dx \,,$$
from which you should be able to see how the first line follows. The statement that two functions are orthogonal amounts to $\langle \phi_1 | \phi_2 \rangle = 0$, whilst the statement that a function is normalised amounts to $\langle \phi_1 | \phi_1 \rangle = 1$.
Since you want a bit of mathematical rigor:
A quantum state is a self-adjoint positive trace class operator on a Hilbert space with trace 1. This is called density matrix $\rho$. In its simplest form, given $\psi\in \mathscr{H}$, $\rho$ is the orthogonal projector on the subspace spanned by $\psi$. Let $E_\rho(\cdot):D_\rho\subset\mathcal{A}(\mathscr{H})\to \mathbb{R}$ be the map defined as:
$$E_\rho(A)=\mathrm{Tr}(A\rho)\; ,$$
where $\mathcal{A}(\mathscr{H})$ is the space of self-adjoint operators, $\mathrm{Tr}$ is the trace on $\mathscr{H}$ and
$$D_\rho=\{A\in \mathcal{A}(\mathscr{H})\; ,\; \mathrm{Tr}\lvert A\rho\rvert<+\infty\}\; .$$
The map $E_\rho(\cdot)$ has all the properties of an expectation in probability theory. I don't know if it is possible to characterize the measure $\mu$ associated to it (maybe by means of the projection valued measures associated to $\rho$ by the spectral theorem, but it is not straightforward at least for me).
Best Answer
Yes, that is, in fact, the mathematically well defined norm of one-particle quantum states in relativistic quantum mechanics (and relativistic QFT) for scalar Klein-Gordon particles. However there is a substantial difference with respect to the non-relativistic case concerning the Hilbert space which does not contain all possible solutions of the scalar KG equation.
To appreciate this difference it is convenient to start form the momentum representation. Consider a sufficiently smooth real solution of KG equation. If it decays sufficiently fast in space can be expanded as (I henceforth assume to deal in a 4-dimensional Minkowski spacetime referring to standard Minkowski coordinates with $c=\hbar=1$, $kx := \sum_{a=1}^3 k^ax^a$, $E(k):= \sqrt{k^2+m^2}$), then $$\psi(t,x) = \int_{\mathbb{R}^3} \frac{\phi(k)}{\sqrt{2E(k)}} e^{i(kx - E(k)t)} + \frac{\overline{\phi(k)}}{\sqrt{2E(k)}} e^{-i(kx - E(k)t)} \:\:\frac{d^3k}{(2\pi)^{3/2}}\:.\tag{1}$$ The quantum state (at time $t$) associated to this solution is just one half of this decomposition: $$\Psi_t(x):= \int_{\mathbb{R}^3} \frac{\phi(k)}{\sqrt{2E(k)}} e^{i(kx - E(k)t)} \:\:\frac{d^3k}{(2\pi)^{3/2}}\:.\tag{2}$$
I stress that,
(a) differently from the real field $\psi$, $\Psi$ is complex in general;
(b) $\Psi$ is still a solution of KG equation, but it cannot have spatial compact support;
(c) in spite of the differences above, $\psi$ and $\Psi$ carry exactly the same amount of information.
As a matter of fact, with some elementary computations, one sees that $$i\int_{\mathbb{R}^3} \overline{\Psi_t(x)} \frac{\Psi'_t(x)}{\partial t} - \Psi'_t(x) \frac{\overline{\Psi_t(x)}}{\partial t} d^3x = \int_{\mathbb{R}^3} \overline{\phi(k)}\phi'(k) d^3k\:.\tag{3}$$ Let us focus on the right-hand side.
(a) It is evidently positive-defined, so that the norm is positive as it should be;
(b) it is also Poincaré-invariant in view of the structure of the unitary representation of the Lorentz group (the translational part acts trivially in terms of standard phases). If passing to the notation $K = (K^0,\vec{K})$ for the four momentum, so that $\vec{K}=k$ and $K^0:= E(k)$, and $\Lambda \in O(1,3)_+$, $$(U_{\Lambda}\phi)(\vec{K}):= \sqrt{\frac{E(\vec{\Lambda K})}{E(\vec{K})}} \phi (\vec{\Lambda K})\:.$$ (Notice that $\frac{d\vec{K}}{E(\vec{K})}=\frac{d\vec{\Lambda K}}{E(\vec{\Lambda K})}$ as is well known and this fact assures the Poincaré invariance of the considered scalar product.)
(c) It does not depend on $t$.
Hence we have a properly defined Hilbert space (independent from $t$). Technically speaking one has to deal with a conveniently smooth vector space of functions $\psi$ which admits Fourier transform when divided with $E^{1/2}$ and finally he/she should take the completion of this space with respect to the said scalar product.
In particular, the expression for the scalar product given in the left-hand side of (3) is valid only for quantum states (2) and not for complete solutions of the KG equation as in (1).
REMARK. Relativistic QM has a problematic status in view of several issues in particular related to the definition of the position operators (there are several possibilities, but none is completely convincing and all are non-local). In particular $|\Psi_t(x)|^2$ cannot be interpreted as the probability density to find the particle at $x$ (when time is $t$). Generally speaking only one-particle states whose energy content is smaller than the mass of the considered particle (thus photons are ruled out) have some chances to have some physically meaningful interpretation. However, the Hilbert space constructed above has a deep relevance also in the standard approach to QFT. Indeed, the symmetric Fock space of QFT is exactly constructed upon this one-particle Hilbert space. Together with the vacuum state, that Hilbert space is the crucial building block of the Fock space construction.
Let us come to the issue regarding the expression of the expectation value of an observable. It is clear form the discussion above that, if we deal with the momentum representation, where the scalar product is $$\langle \phi , \phi' \rangle := \int_{\mathbb{R}^3} \overline{\phi(k)} \phi'(k) d^3k$$ nothing relevant changes with respect to the standard formalism. The expectation value $<A>_\phi $, defined from the spectral theory for the selfadjoint operator $A$, satisfies the identity (if $\phi$ stays in the domain of the operator) $$<A>_\phi = \int_{\mathbb{R}^3} \overline{\phi(k)} (A\phi)(k) d^3k\:.$$ To export this identity in the spacetime representation is a hard issue and it strictly depends on the nature of $A$. Abstractly speaking, from (2), the spacetime representation $A_{st}$ of $A$ is defined as $$(A_{st}\Psi)_t(x):= \int_{\mathbb{R}^3} \frac{(A\phi)(k)}{\sqrt{2E(k)}} e^{i(kx - E(k)t)} \:\:\frac{d^3k}{(2\pi)^{3/2}}\:.$$ With this definition, the expectation value (provided a number of mathematical hypotheses are fulfilled) of $A$ in the spacetime reads from (3) $$<A>_\phi = i\int_{\mathbb{R}^3} \left[\overline{\Psi_t(x)} \frac{(A_{st}\Psi)_t(x)}{\partial t} - (A_{st}\Psi)_t(x) \frac{\overline{\Psi_t(x)}}{\partial t}\right] d^3x\:.$$ To find the explicit expression of $A_{st}$ is usually difficult, barring trivial cases, as the momentum.
A nice fact is that the Schroedinger equation with Hamiltonian $H:= \sqrt{k^2+m^2}$ in momentum representation, when represented in spacetime and for the considered states implies the KG equation. However, the spacetime representation of the said Hamiltonian is a pseudodifferential operator, that is a non-local operator $$H_{st} = \sqrt{-\Delta_x + m^2I}$$ where $\Delta_x$ is the spatial Laplacian: $$i\frac{\partial \Psi_t}{\partial t}= \left(\sqrt{-\Delta_x + m^2I} \Psi_t\right)(x)$$