Yes, this sounds reasonable. Starting with a basis $\alpha_1,\ldots,\alpha_n$ of an extension $L / K$, you can multiply them with elements of $\mathcal{O}_K$ to assume that they in fact lie in $\mathcal{O}_L$. After that, one has the following result, whose proof I'll more or less quote.
Lemma (cf. Neukirch, I.2.9). Write $d$ for the discriminant of $\alpha_1,\ldots,\alpha_n$. Then $d\mathcal{O}_L \subseteq \mathcal{O}_K \alpha_1 + \cdots + \mathcal{O}_K \alpha_n$.
Proof: If $\alpha = c_1 \alpha_1 + \cdots + c_n \alpha_n \in \mathcal{O}_L$, with coefficients $c_i$ in $K$, then the $c_i$ are a solution of the system of linear equations
$$\operatorname{Tr}_{L/K}(\alpha_i\alpha) = \sum_j \operatorname{Tr}_{L/K}(\alpha_i\alpha_j)c_j,$$
and as $\operatorname{Tr}_{L/K}(\alpha_i \alpha) \in \mathcal{O}_K$, they are given as the quotient of an element of $\mathcal{O}_K$ by the determinant $\det \operatorname{Tr}_{L/K}(\alpha_i \alpha_j) = d$. Therefore the $dc_j$ live in $\mathcal{O}_K$, and the desired result follows.
I presume this is enough to infer finite generation of $\mathcal{O}_L$ over $\mathcal{O}_K$.
Note that in this case
$$\mathcal O_L\subseteq d^{-1}\alpha_1\mathcal O_K+\cdots+d^{-1}\alpha_n\mathcal O_K$$
and the latter is a free $\mathcal O_K$-module and in particular finitely generated. We may consider the special case $K=\Bbb Q$ and $\mathcal O_K=\Bbb Z$. Then, $\mathcal O_L$ is as a submodule of a finitely generated module over a Noetherian ring is itself finitely generated. This by passes the usual trace argument in so far as this argument is hidden in the lemma. Using this on $\mathcal O_K\subseteq\mathcal O_L$ for a finite extension of number fields $L/K$ then gives your result.
However, I am not completely sure if the $\alpha_i$ do form an integral basis necessarily. The argument I presented only proves that $\mathcal O_L$ is itself finitely generated and hence finite over $\mathcal O_K$ containing the latter (which is the question asked in the title and what the linked answer is concerned with; please correct me if your questions was something else).
Best Answer
I think it's true and the following proof should go through.
Lemma. (Basis Extension Theorem for Modules) Let $\def\O{\mathcal{O}}\def\Z{\mathbb{Z}}M$ be an $R$-module and $N \subseteq M$ be a submodule such that:
Then $M$ is free and any basis of $N$ can be extended to a basis of $M$.
Proof. Recall/show that any short exact sequence $0 \to N \to M \to F \to 0$ of $R$-modules with $F$ free splits. In our case that yields $M \cong N \oplus M/N$. $\quad\square$
Now let $L/K$ be an extension of number fields. By the Lemma, it suffices to show that:
$\O_K$ is free by the (algebraic number) theory, of course. (Or apply this proof here first to $K/\mathbb{Q}$!) To show that $\O_L/\O_K$ is free, it suffices to show that it's torsion-free (because we work over $\Z$, which is a PID). But $\O_L/\O_K$ is torsion-free because $\O_K$ is integrally closed. Indeed, assume $x \in \O_L$ satisfies $nx \in \O_K$ for some $n \in \Z_{>0}$. Then $x \in \operatorname{Frac}(\O_K) = K$ and $x$ is integral over $\Z$ because $x \in \O_L$. Since $\O_K$ is integrally closed, this implies $x \in \O_K$.