As said in a comment, the proof of $\underline{\dim}_B F \ge \dim_H F$ for unbounded sets can proceed as
$$\underline{\dim}_B F = \sup_{E \subset F, \rm ~bounded} \underline{\dim}_B E \ge \sup_{E \subset F, \rm ~ bounded} {\dim}_H E = \dim_H F$$
where the last step is based on the fact that the Hausdorff dimension is countably stable:
$$
F = \bigcup_{n=1}^\infty (F\cap B(0,n)) \implies \dim_H F = \sup \dim_H(F\cap B(0,n))
$$
The inequality $\underline{\dim}_B F \le \overline{\dim}_B F$ for unbounded sets follows directly from the definition, since both sides are suprema taken over bounded sets, and the upper dimension is known to be larger for those.
Below is a proof I have for the lower box dimension, from some handwritten notes I wrote in January 1991. I don't have time now to check the details, other than a quick proof-reading of the expressions, so let me know if something seems amiss and I'll try to address your concerns at some later time.
Let $\delta > 0$ be given. Then there exists $k$ such that $\delta_{k+1} < \delta < \delta_{k}.$ Therefore,
$$\frac{\log N_\delta(F)}{-\log\delta} \;\; \leq \;\; \frac{\log N_{\delta_{k+1}}(F)}{-\log\delta} \;\; \leq \;\; \frac{\log N_{\delta_{k+1}}(F)}{-\log\delta_k} \;\; = \;\; \frac{\log N_{\delta_{k+1}}(F)}{-\log\left(\frac{\delta_k}{\delta_{k+1}} \cdot \delta_{k+1}\right)} $$
$$ = \;\; \frac{\log N_{\delta_{k+1}}(F)}{-\log\left(\frac{\delta_k}{\delta_{k+1}}\right) - \log \delta_{k+1}} \;\; \leq \;\; \frac{\log N_{\delta_{k+1}}(F)}{\log c - \log \delta_{k+1}}, $$
where for the last inequality note that
$$\frac{\delta_{k+1}}{\delta_k} \geq c \;\; \implies \;\; \log \frac{\delta_k}{\delta_{k+1}} \leq \log \frac{1}{c} = -\log c \;\; \implies \;\; \log c \leq -\log\left(\frac{\delta_k}{\delta_{k+1}}\right),$$
and so using $\log c$ in "the last inequality" results in a smaller denominator (hence, a larger value).
Therefore,
$$ \liminf_{\delta \rightarrow 0^+}\, \frac{\log N_{\delta}(F)}{-\log \delta} \;\; \leq \;\; \liminf_{k \rightarrow \infty}\, \frac{\log N_{\delta_{k+1}}(F)}{\log c - \log \delta_{k+1}} $$
$$= \;\; \liminf_{k \rightarrow \infty}\, \left[ \frac{\;\;\frac{\log N_{\delta_{k+1}}(F)}{-\log \delta_{k+1}}\;\;}{\frac{\log c}{-\log \delta_{k+1}} + 1} \right] \;\; = \;\; \liminf_{k \rightarrow \infty}\, \frac{\log N_{\delta_{k+1}}(F)}{-\log \delta_{k+1}} \;\; = \;\; \liminf_{k \rightarrow \infty}\, \frac{\log N_{\delta_k}(F)}{-\log \delta_k}, $$
where in the second to last equality above we are using the fact that $\frac{\log c}{-\log \delta_{k+1}} \rightarrow 0$ as $k \rightarrow \infty.$
Interchanging $k$ with $k+1$ in the above is easily seen to reverse each inequality, and so the other inequality also holds.
Best Answer
The result holds true as long as $\mathcal{H}^s(F) > 0$, though a slightly different argument is required. The reason is that for $r>0$ we have $$\mathcal{H}^s(rF) = r^s\mathcal{H}(F),$$ where $rF=\{rx:x\in F\}$. Thus, for sufficiently large $r$ we have $$\dim_H(rF)\leq\underline\dim_B(rF)\leq\overline\dim_B(rF).$$ Furthermore, $\dim(rF)=\dim(F)$ for all these definitions of dimension.