Simple derivation:
From my naive perspective, you are looking for a distance between points $(0,0,\dots,0)$ and $(R,R,\dots,R)$. Since you are in $n$-dimensional Euclidean space, their separation is $\sqrt{(R-0)^2 + \dots + (R-0)^2} = \sqrt{n} R$. Would that be sufficient?
Looking at it geometrically, if the length in $(n-1)$ dimensions is $l_{n-1}$, you can use the fact that, since the $n^{th}$ direction is perpendicular to any direction in the $(n-1)$ dimensional subspace, Pythagorean addition of distances holds and $l_n = \sqrt{l_{n-1}^2 + R^2}.$ Starting from $l_1 = R$, you get $l_n = \sqrt{n} R$ by induction.
More detailed derivation using differential geometry:
To make it more explicit, one can use the metric of $n$-dimensional Euclidean space $g_{ab} = \delta_{ab}$ for $a,b \in [1,2,\dots,n]$. The "distance" $s$ is then defined as
$$
(\mathrm{d} s)^2 = \sum_{a,b} g_{ab} \mathrm{d}x^a \mathrm{d}x^b \, ,
$$
in general. Let's have a curve $x^a = x^a(t)$ parametrised by $t$. Then
$$
\frac{\mathrm{d} s}{\mathrm{d} t} = \sqrt{\sum_{a,b} g_{ab} \frac{\mathrm{d}x^a}{\mathrm{d}t} \frac{\mathrm{d}x^b}{\mathrm{d}t}} = \sqrt{\sum_a \left ( \frac{\mathrm{d}x^a}{\mathrm{d}t} \right ) ^2 } \, .
$$
The diagonal going from $(0,0,\dots,0)$ to $(R,R,\dots,R)$ can be described by the curve $x^a(t) = Rt$ for $t \in [0,1]$. The total length of the curve is
$$
s = \int_0^1 \mathrm{d} s = \sqrt{\sum_a \left ( \frac{\mathrm{d}x^a}{\mathrm{d}t} \right ) ^2 } \mathrm{d}t = \int_0^1 \sqrt{\sum_a \left ( R \right ) ^2 } \mathrm{d}t = \int_0^1 \sqrt{n} R \mathrm{d} t = \sqrt{n} R \, .
$$
Therefore the length of the diagonal in $n$ dimensions is $\sqrt{n} R$.
Best Answer
I believe that by "diagonal" you mean "diameter", which is a more general term from geometry which means "the greatest distance that two points on the shape can reach from each other". In the standard dimensions that we can visualize, these would be the distances between the vertices of the square and the cube.
In multiple dimensions the idea still generalizes, so the answer, in short, is $\sqrt{n*a^2}$, where $n$ is the number of dimensions. But why?
Let's assume that one corner of the (hyper)-cube, which in four dimensions is known as a tesseract, is at the origin, (0,0,...,0). Then the corners can all be given coordinates. For example, one corner is at (a,0,...,0), and the furthest corner from the origin would be at (a,a,...,a). The distance from this corner to the origin, then, is given by:
$$D = \sqrt{a^2 + a^2 + ... + a^2} = \sqrt{n*a^2}$$