What you want is use the $\varepsilon$-$\delta$ definition of continuity/semi-continuity
$$
\forall \varepsilon > 0, \quad \exists \delta > 0 \quad \text{s.t.} \quad |x-x_0| < \delta \quad \Longleftrightarrow \quad |f(x) - f(x_0)| < \varepsilon.
$$
Since $|f(x) - f(x_0)| < \varepsilon$ is equivalent to $f(x_0) - \varepsilon < f(x) < f(x_0) + \varepsilon$, this statement is equivalent to
$$
\forall \varepsilon > 0, \quad \exists \delta > 0 \quad \text{s.t.} \quad |x-x_0| < \delta \quad \Longleftrightarrow \quad f(x_0) - \varepsilon < f(x) < f(x_0) + \varepsilon
$$
which means continuity is equivalent to upper and lower semi-continuity.
ADDED : I'll assume the definition of semi-continuity is the following, i.e. that the preimage of an open set of the form $\{ y \, | \, y > a \}$ is open for any upper semi-continuous function, and that the preimage of an open set of the form $\{ y \, | \, y < a \}$ is open for any lower semi-continuous function. If this is not the definition you have, let me know. I'm just guessing those definitions from the $\varepsilon$-$\delta$ definitions of continuity.
We show that continuity $\Longleftrightarrow$ upper and lower semi-continuity.
$(\Longrightarrow)$ This one is clear, since if the pre-image of any open set is open, then in particular are the pre-images of those of the form $(a, \infty)$ and $(-\infty,a)$.
$(\Longleftarrow)$ If $f^{-1}((a,\infty))$ and $f^{-1}((-\infty,b))$ are open sets, then since $f^{-1}((a,b)) = f^{-1}((a,\infty)) \cap f^{-1}((-\infty,b))$ and that the intersection of two open sets is open, then $f^{-1}((a,b))$ is open. Now any open set $\mathcal O$ in $\mathbb R$ is of the form
$$
\bigcup_{n=0}^{\infty} (a_n, b_n)
$$
where all the intervals $(a_n,b_n)$ are pairwise disjoint and $a_n \in \mathbb R \cup \{-\infty\}$, $b_n \in \mathbb R \cup \{\infty\}$. But
$$
f^{-1} \left( \bigcup_{i \in I} \mathcal O_i\right) = \bigcup_{i \in I} f^{-1} (\mathcal O_i)
$$
for any collection of sets indexed by any set $I$ (prove this trivially by the definition of pre-images and show $\subseteq$/$\supseteq$), thus,
$$
f^{-1}(\mathcal O) = \bigcup_{n=0}^{\infty} f^{-1}((a_n,b_n))
$$
which is an open set.
Hope that helps,
Best Answer
It is true. See the book
Engelking, Ryszard. "General topology."
On p.428 5.5.20 you find the following result as an exercise:
A $T_1$-space $X$ is normal and countably paracompact if and only if for each pair $f,g$ of real-valued functions on $X$, where $f$ is upper semicontinuous and $g$ is lower semicontinuous such that $f(x) < g(x)$ for all $x \in X$, there exists a continuous $h : X \to \mathbb R$ such that $f(x) < h(x) < g(x)$ for all $x$.
You will also find references to papers containing proofs, for example
Miroslav Katětov, On real-valued functions in topological spaces, Fundamenta Mathematicae 38 (1951), 85–91
as quoted in Dave L. Renfro's link.