The point is precisely that $g$ is Lebesgue measurable, but not Borel measurable.
If it was Borel measurable, we would get the contradiction you describe.
Also note the way the measurability is proved: Completeness of the Lebesgue measure (on the Lebesgue measurable sets) is invoked. When restricted to the Borel sets, this completeness fails.
Let $\mathcal A$ be the set of all Borel subset $B$ of $\Bbb R$ such that $f^{-1}(B)$ is also a Borel subset of $\Bbb R$. Since $f$ is Borel-measurable we have $(c,\infty)\in \mathcal A$ for all $c\in\Bbb R$.
Let $\sigma(\mathcal A)$ be the smallest $\sigma$-algebra containing the set $\mathcal A$. Since, the operation $f^{-1}$, i.e. operation of taking inverse commutes with the countable union operation and taking complement operation, so we have $\sigma\big(\{f^{-1}(B):B\in\mathcal A\}\big)=\big\{f^{-1}(X): X\in\sigma(\mathcal A)\big\}.$
Now, since $\sigma(\mathcal A)$ is a $\sigma$-algebra we have $(a,\infty)\cap (b,\infty)=(a,b)\in \sigma(\mathcal A)$ for all $a,b\in\Bbb R$.
Similarly, $(-\infty,a']=\Bbb R\backslash (a',\infty)$ is also in $\sigma(\mathcal A)$ for all $a'\in\Bbb R$ as $\sigma$-algebra is closed under complement.
Hence, $(-\infty,a)=\bigcup_{n=1}^\infty\big(-\infty,a-\frac{1}{n}\big]$ is also an element of $\sigma(\mathcal A)$ for all $a\in\Bbb R$ as $\sigma$-algebra is closed under countable union.
Also, every open subset of $\Bbb R$ can be written as a countable union of open intervals of $\Bbb R$ and every $\sigma$-algebra is closed under countable union.
Therefore, every open subset of $\Bbb R$ is an element of $\sigma(\mathcal A)$. In other words, the set $\tau(\Bbb R)$ of all open subsets of $\Bbb R$ is a subset of $\mathcal A$.
But, the Borel-$\sigma$ algebra $\mathcal B(\Bbb R)$ of $\Bbb R$ is the smallest $\sigma$-algebra containing all open subsets of $\Bbb R$, i.e. $\sigma\big(\tau(\Bbb R)\big)=\mathcal B(\Bbb R)$. Hence, $\sigma(\mathcal A)\supseteq \mathcal B(\Bbb R)$ as $\mathcal A\supseteq \tau(\Bbb R)$.
Finally, For any $Y\in\mathcal B(\Bbb R)\implies Y\in \sigma(\mathcal A)\implies f^{-1}(Y)\in \sigma\big(\{f^{-1}(B):B\in\mathcal A\}\big)\subseteq \mathcal B(\Bbb R)$. The last inclusion is due the fact that each set $f^{-1}(B)\in \mathcal B(\Bbb R)$ for all $B\in \mathcal A$ from definition of $\mathcal A$. Hence, $\sigma\big(\{f^{-1}(B):B\in\mathcal A\}\big)\subseteq \sigma\big(\mathcal B(\Bbb R)\big)=\mathcal B(\Bbb R)$.
Best Answer
Note that by definition, $h^{-1}$ maps $\mathcal{B}_\mathbb{R}$ into $\mathcal{B}_\mathbb{R}$, so we need only prove that the same is true of $g^{-1}$, i.e. that all continuous functions $g:\mathbb{R}\to\mathbb{R}$ are Borel-measurable. More generally, let $g:X\to Y$ be a map of topological spaces, and I claim that $g^{-1}$ takes $\mathcal{B}_Y$ into $\mathcal{B}_X$.
Why is this true? Let $$\mathcal{M} = \{E\subseteq Y\mid g^{-1}(E)\in\mathcal{B}_X\}$$ then note that $\mathcal{M}$ is easily seen to be a $\sigma$-algebra, and we must show that $\mathcal{B}_Y\subseteq\mathcal{M}$. But to do this, we need only show that $V\in\mathcal{M}$ for all opens $V\subseteq Y$, and this follows since $g^{-1}(V)\subseteq X$ is open, by continuity, and therefore, $g^{-1}(V)\in\mathcal{B}_X$. If this is too general for you, simply replace $X$ and $Y$ with $\mathbb{R}$ and everything remains true.
Therefore, we have proved that continuous maps are Borel-measurable, and the statement is proved.