An imprecise version of the question is that when $A$ and $B$ are regular rings, is $A \otimes B$ also regular? Please allow me to put more restrictions, here I am only interested in the case when $A$ and $B$ are finitely generated $k$-algebras. When $k$ is perfect, the answer is yes, see http://arxiv.org/abs/math/0210359. In fact, we can view $A$ and $B$ as coordinate rings of some affine $k$-varieties say $X$ and $Y$ respectively. Since $k$ is perfect, regularity is equivalent to being smooth and it can be shown easily $X \times_k Y$ is smooth. Hence the conclusion. Now when $k$ is not perfect, and assume in addition $X$ and $Y$ are both geometrically (absolutely) integral, moreover they contain some (regular but) not smooth point (so the above method doesn't apply), then is $X \times_k Y$ still regular?
Example, $k=\mathbb{F}_p(t)$, $p>2$, $A=k[x,y]/(x^p-x^{p-1}y-t)$, $X=\operatorname{Spec} A$. Then it is easy to show $X$ is geometrically integral, the maximal ideal generated by $(Y)$ in $A$ is regular but not smooth, and that $A$ is regular. The question is that is $X\times_k X$ regular?
Note, it is easy to produce a counter example when $X$ is not assumed to be geometrically integral, e.g. $A=k[x]/(x^p-t)$, then it is easy to show $X\times_k X$ is a $0$-dimension local ring but not a domain, therefore it can not be regular.
Best Answer
This is inspired by Tom Goodwillie's answer.
Proof. The first projection $X\times_k X\to X$ is faithfully flat, so the regularity of $X\times_k X$ implies that of $X$. To prove the smoothness of $X$, we can suppose $X$ is connected and affine. The diagonal morphism $\Delta: X\to X\times_k X$ is then a closed immersion from a regular scheme to a regular scheme. Therefore $\Delta$ is locally complete intersection. If $J$ is the ideal sheaf on $X\times_k X$ defining $\Delta(X)$, then $\Delta^*(J/J^2)$ is locally free of rank the codimension of $X$ in $X\times_k X$ which is equal to $\dim X$.
Now $\Delta^*(J/J^2)$ is isomorphic to the sheaf of differential forms $\Omega^1_{X/k}$ on $X$ (see Hartshorne), so the latter is locally free of rank $\dim X$. This implies that $X$ is smooth.