I have a proof I was hoping someone could review. Thanks in advance!
Let A be a matrix that is both triangular and unitary.
Then the columns of A are of length 1 and are orthogonal to each other.
Assume A is upper triangular, in this case.
Then set a$_n$ = column $n$ in A.
Clearly, since A is upper triangular and the columns are of length 1, then $ a$_1$ = $$
\begin{bmatrix}
\pm 1 \\
0 \\
0 \\
\vdots \\
0 \\
\end{bmatrix}
$$
$
Now column $a_2$ has a similar composition with $a_1$. That is, on the diagonal of the entire matrix A, its second component here, $y_2$, has the value $\pm$ 1 and zeros in all other components. This is due to first only two options exists for non-zero entries in the columns since A is upper triangular. Second since $a_1$*$a_2$ = 0 by orthogonality we have:
$$
1*y_1 + 0*y_2 + 0 + 0 + … + 0 = 0
$$
Where $a_2$ is made up of components $y_1$, $y_2$ , … $y_n$. So clearly $y_1$ must be zero, and for i > 2 we know $y_i$ = 0, by the upper triangular property
$ a$_2$ = $$
\begin{bmatrix}
0 \\
\pm 1 \\
0 \\
\vdots \\
0 \\
\end{bmatrix}
$$
$
The remainder of the proof uses induction after examining these first two cases as the base case.
Let the following claim hold for $a_n$, we wish to prove that $a_(n+1)$ is a zero vector except in its n+1 component (where $\pm$ 1 resides). Then given the $a_(n+1)$ column we note that by the orthogonality with all the previous n columns we can show that 0 must be the value of the first n components in $a_(n+1)$.
for any ith component in the column for i < n+1 we have:
$a_1$ * $a_n+1$ = 0 => the 1st component must be zero (similarly to above)
$a_2$ * $a_n+1$ = 0 => the 2nd component must be zero (similarly to above)
and so on … until we conclude that each component up to the n+1 component must be equal to zero by orthogonality.
Hence only the $n+1$ component in the column $a_(n+1)$ can be equal to $\pm$ 1 since all other inner products with the previous n columns indicate the remaining components must be zero (and any components below are zero due to A being upper triangular).
Hence for the upper triangular case, the matrix A must be diagonal.
I believe the lower triangular case to be a similar proof, but have not delved into it yet.
Thanks!
Best Answer
I think our OP H_1317's proof is conceptually correct.
Here is a more somewhat more abstract proof:
Suppose $A$ is upper triangular; then I claim that $A^{-1}$ is also upper triangular; for we may write
$A = D + T, \tag 1$
where $D$ is diagonal and $T$ is strictly upper triangular; that is, the diagonal entries of $T$ are all zero; we observe that, since $A$ is triangular, $\det(A)$ is the product of the diagonal entries of $A$; since $A$ is unitary, it is non-singular and thus $\det(A) \ne 0$, so none of the diagonal entries of $A$ vanish, and the same applies to $D$; therefore $D$ is invertible and we may write
$A = D(I + D^{-1}T); \tag 2$
we next observe that $D^{-1}T$ is itself strictly upper triangular, hence nilpotent; in fact we have
$(D^{-1}T)^n = 0, \tag 3$
where $n = \text{size}(A)$; the nilpotence of $D^{-1}T$ allows us to write an explicit inverse for $I + D^{-1}T$; indeed, we have the well-known formula
$(I + D^{-1}T) \displaystyle \sum_0^{n - 1} (-D^{-1}T)^k = I + (-1)^n (D^{-1}T)^n = I; \tag 4$
thus,
$(I + D^{-1}T)^{-1} = \displaystyle \sum_0^{n - 1} (-D^{-1}T)^k; \tag 5$
since every matrix $(-D^{-1}T)^k$ occurring in this sum is upper triangular, we see that $(I + D^{-1}T)^{-1}$ is upper triangular as well; from (2),
$A^{-1} = (I + D^{-1}T)^{-1}D^{-1}, \tag 6$
which shows that $A^{-1}$ is upper triangular.
Having established my claim, we now invoke the unitarity of $A$:
$A^\dagger A = AA^\dagger = I, \tag 7$
i.e.,
$A^\dagger = A^{-1}; \tag 8$
we have by definition
$A^\dagger = (A^\ast)^T = ((D + T)^\ast)^T = (D^\ast + T^\ast)^T = (D^\ast)^T + (T^\ast)^T, \tag 9$
from which we see that $A^\dagger$ is lower triangular when $A$, and hence $A^{-1}$, is upper triangular; then only way (8) can hold is with $T = 0$; therefore we see that
$A = D \tag{10}$
is a diagonal matrix.
Of course, if $A$ is lower triangular the same result binds, the proof almost identical to that given above. $OE\Delta$.