Here is example 2 in chapter 3.29 in Munkres:
The space $\Bbb{R}^\omega$ [defined as $\Bbb{R} \times \Bbb{R} \times \Bbb{R} \times …$ endowed with the product topology] is not locally compact…For if $B = (a_1,b_1) \times … \times (a_n,b_n) \times \Bbb{R} \times …$ were contained in a compact subspace, then its closure $\overline{B}$ would be compact, which it is not.
Would the following be a sufficient way of filling in the details? Recall that every compact subspace of a Hausdorff space ($\Bbb{R}^\omega$ is in fact metrizable) is necessarily closed. Thus this compact space containing $B$ must be closed. Now since the closure of $B$ is the smallest closed set containing $B$, the closure of $B$ must be contained in this compact set also. Again, recall that every closed subspace of a compact subspace is itself compact, making $\overline{B}$ compact too. However, if it were compact, then its image under the continuous map $\pi_{n+1}$, the projection of the $(n+1)$-th factor, would also be compact, but this is a contradiction since the image is $\Bbb{R}$, which is known not to be compact.
Best Answer
We can show more:
Proposition: The product space $\prod X_\alpha$ is locally compact if and only if each $X_\alpha$ is locally compact and $X_\alpha$ is compact for all but finitely many values of $\alpha.$
Proof. Recall that the projections $\pi_\beta:\prod X_\alpha\to X_\beta$ are continuous open maps. Suppose that $\prod X_\alpha$ is locally compact. Let $\mathbf{x} \in \prod X_\alpha.$ Then there exist a compact subspace $C$ of $\prod X_\alpha$ containing a neighborhood of $\mathbf{x}.$ This neighborhood contains a basis element for $\prod X_\alpha$ containing $\mathbf{x},$ say $U=\prod U_\alpha$ where $U_\alpha=X_\alpha$ for all but finitely many indices, say $\alpha_1, \ldots, \alpha_n.$ Let $\beta \neq \alpha_i$ for all $i=1,\ldots,n.$ Then $\pi_\beta(C)$ is compact and contains $\pi_\beta(U)=X_\beta,$ so $\pi_\beta(C)=X_\beta$ and $X_\beta$ is compact. Hence $X_\alpha$ is compact for all but finitely many values of $\alpha.$ Now we prove that $X_{\alpha_i}$ is locally compact for $i=1,\ldots,n.$ Let $x \in X_{\alpha_i}.$ Let $\mathbf{x} \in \prod X_\alpha$ be such that $x_{\alpha_i}=x.$ There is a compact $C$ containing a basis neighborhood $U$ of $\mathbf{x}.$ Then $\pi_{\alpha_i}(C)$ is a compact subspace of $X_{\alpha_i}$ containing the neighborhood $\pi_{\alpha_i}(U)$ of $x.$ So the spaces $X_{\alpha_i}$ are (at least) locally compact.
Now suppose that each $X_\alpha$ is locally compact and $X_\alpha$ is compact for all but finitely $\alpha.$ Let $\mathbf{x} \in \prod X_\alpha.$ For each $\alpha,$ there exists a compact subspace $C_\alpha$ of $X_\alpha$ containing a neighborhood $U_\alpha$ of $x_\alpha.$ By hypothesis, for all but finitely many indices we can assume that $C_\alpha=U_\alpha=X_\alpha.$ By the Tychonoff theorem, $\prod C_\alpha$ is compact, and it contains the neighborhood $\prod U_\alpha$ of $\mathbf{x}.$ Hence $\prod X_\alpha$ is locally compact.