One method is to divide the tetrahedron into 8 identical sections or two sets of 4 identical sections and then to find the smallest bounding cube as the sections are randomly rotated. For 8 cubes, I haven't been able to beat 1/2 yet.
Covering an edge-2 tetrahedron with 4 unit cubes is easy (link to code). With some twisting and optimization, the tetrahedron can be covered with 4 cubes with edge length 0.969975. This can likely be improved by allowing the cubes to interact.
Note that the question for how best to cover a triangle with two squares wasn't solved correctly until 2009. Instead of jumping to 8 cubes, it might be best to settle 2 cubes, 3 cubes, and 4 cubes first.
For a tetrahedron with edge length 2 ...
1 cube of edge length 1.41421356237309504
2 cubes of edge length 1.41421356237309504 (??)
3 cubes of edge length 1.3233 (below)
4 cubes of edge length 0.969975 (above)
5 cubes of edge length .848528 (below)
Here's a picture for 3 cubes. The tetrahedron has a base triangle ABC. One cube covers points A, D, mid-AB, mid-AC, and the center of the base. But there is likely improvement by shifting the points mid-AB, mid-AC.
Five cubes seems to have a simple solution.
The key insight is to first determine the minimum distance between the body diagonal and the face diagonal. Without loss of generality let the cube have unit side length and take the body diagonal to be the segment joining $(0,0,0)$ to $(1,1,1)$; then there are six face diagonals that do not intersect the body diagonal. These can be partitioned into two groups of three such that each group lies in a plane, and form the sides of equilateral triangle of side length $\sqrt{2}$ perpendicular to the body diagonal, which passes through the centers of these triangles.
Consequently, the minimum distance between the body diagonal and any such face diagonal is simply $$\frac{\sqrt{2}}{2\sqrt{3}} = \frac{1}{\sqrt{6}}.$$
Since the tetrahedron is regular, with two vertices on the body diagonal and two on the face diagonal, this means distance between two non-adjacent edges of the tetrahedron is $1/\sqrt{6}$. If the side length of the tetrahedron is $2s$, then this implies the distance between $(s, -\frac{1}{2\sqrt{6}}, 0)$ and $(0, \frac{1}{2 \sqrt{6}}, s)$ is $2s$; i.e., we require $$s^2 + \frac{1}{6} + s^2 = 4s^2,$$ or $$s = \frac{1}{2\sqrt{3}}.$$
Therefore, the tetrahedron's volume is $$V = \frac{(2s)^3}{6\sqrt{2}} = \frac{1}{18\sqrt{6}},$$ and the volume for the original cube of side length $a$ is $$V(a) = \frac{a^3}{18 \sqrt{6}}.$$
We could also have found this by noting that the circumscribed cube to the tetrahedron has edge length $1/\sqrt{6}$, thus the edge of the tetrahedron, being also the face diagonal of the circumscribed cube, has length $\sqrt{2}$ times this. And the circumscribed cube's volume is just $\frac{1}{6 \sqrt{6}}$, which is $3$ times the volume of the inscribed tetrahedron, since the four congruent tetrahedra inside the cube but outside the regular tetrahedron each has volume $1/6$ that of the cube. So the desired tetrahedron's volume is again $1/(18 \sqrt{6})$.
For your understanding and enjoyment, please see the animation below, which illustrates the six tetrahedra that can be thus formed with a single body diagonal. A representative set of coordinates for a single tetrahedron, up to symmetry, is $$\left\{ \left(\frac{6 - \sqrt{6}}{12}, \frac{6 + \sqrt{6}}{12}, 0 \right), \left(\frac{6 + \sqrt{6}}{12}, \frac{6 - \sqrt{6}}{12}, 0 \right), \left(\frac{1}{6}, \frac{1}{6}, \frac{1}{6}\right), \left(\frac{1}{2}, \frac{1}{2}, \frac{1}{2}\right) \right\}.$$
Best Answer
As usual in similar problems the key to the solution is a good drawing. Let the edge $AB$ and the height of the tetrahedron drawn from $D$ be parallel to the plane of the figure. Observe that the fourth vertex $C$ of the tetrahedron coincides (in projective sense) with the foot of the height.
The point $Z$ is the center of the tetrahedron. Let $X$ and $Y$ be the centers of the spheres lying on the heights drawn from $A$ and $B$, respectively. Obviously the spheres have the largest possible radius if they touch each other. From the symmetry the touch point $N$ lies (in the projective sense) on the line $DC$.
From the similar triangles $AXQ$ and $AZC$ we have: $$ \frac{XQ}{QA}=\frac{ZC}{CA}\implies\frac{r}{\frac12-r}=\frac{\sqrt\frac1{24}} {\frac12}\implies r=\frac1{2+\sqrt{24}}. $$