Assume $V$ is a finite-dimensional vector space over $\mathbb{R}$, and that $T: V \to V$ is a (linear) isomorphism.
When is it possible to construct an inner product on $V$
making $T$ an isometry?
(Hopefully, I am looking for necessary & sufficient conditions $T$ should satisfy, i.e. a full characterization of the situation).
What I have so far:
A necessary condition: all the real eigenvalues of $T$ are of absolute value $1$. (Since $ T(v)=\lambda v \Rightarrow \langle v,v\rangle=\langle Tv,Tv\rangle = \langle \lambda v, \lambda v\rangle = \lambda^2\langle v, v\rangle$ and an eigenvector $v$ must be nonzero.)
This condition is certainly not sufficient:
For example look at $A$ = $\begin{pmatrix} 1 & 1 \\\ 0 & 1 \end{pmatrix}: \mathbb{R}^2 \to \mathbb{R}^2$. It is an automorphism which has only one eigenvalue ($\lambda = 1$). However, $A\begin{pmatrix} x \\ y \end{pmatrix}= \begin{pmatrix} x+y \\ y \end{pmatrix}$, hence $A^n\begin{pmatrix} x \\ y \end{pmatrix}= \begin{pmatrix} x+ny \\ y \end{pmatrix}$ and the requirement $A:(\mathbb{R}^2,\langle \rangle) \to (\mathbb{R}^2,\langle \rangle) $ to be an isometry for some inner product $\langle \rangle$ implies: $\lVert \begin{pmatrix} x \\ y \end{pmatrix}\rVert^2=\lVert A^n\begin{pmatrix} x \\ y \end{pmatrix}\rVert^2\Rightarrow x^2 \lVert e_1\rVert^2+y^2 \lVert e_2\rVert^2+2xy\langle e_1,e_2\rangle = (x+ny)^2 \lVert e_1\rVert^2+y^2 \lVert e_2\rVert^2+2y(x+ny) \langle e_1,e_2\rangle \Rightarrow 0=(2nxy+n^2y^2)\lVert e_1\rVert^2+2ny^2 \langle e_1,e_2\rangle$.
So we get that $0=(2xy+ny^2)\lVert e_1\rVert^2+2y^2 \langle e_1,e_2\rangle$ for any $x,y\in \mathbb{R}, n\in \mathbb{N}$ which is a contradiction since $\lVert e_1 \rVert > 0$.
Some sufficient conditions:
1) If $T$ is diagonalizable over $\mathbb {R}$ (with all eigenvalues $1$ or $-1$, by our necessary condition), then let ${V_1,…,V_n}$ be a basis of eigenvectors of $T$ , and define $\langle v_i,v_j\rangle = \delta_{ij}$. $T$ will be an isometry.
This condition is certainly not necessary:
just take a rotation (say by $90^{\circ}$) in the plane. note that it is diagonalizable over $\mathbb{C}$. My guess is that if our transformation is diagonalizable over $\mathbb{C}$ (with all eigenvalues with absolute value 1) a similar construction like the above will work. One problem I see with this approach is that an odd-dimensional $\mathbb{R}$-vector space cannot even be considered as a $\mathbb{C}$-vector space. (Though we can always complexify…).
2) $T$ is of finite order. (Then we just start with any inner product on $V$ and construct a new one via summing over iterates of $T$, i.e: $\langle v,w \rangle ' = \sum_{i=0}^{n-1} \langle T^iv,T^iw \rangle $). Note that (as explained for instance here) this implies $T$ is diagonalizable over $\mathbb{C}$, but of course not necessarily over $\mathbb{R}$. (Think about our rotation again.)
Actually, I have now understood that condition (1) implies $T$ is of order 2, (I think the reverse implication also holds, i.e $T^2=Id\Rightarrow T$ is diagonalizable). So condition (1) is a particular case of (2).
However, (2) is not necessary, since any rotation of irrational multiple of 2$\pi$ is an isometry w.r.t the standard product, but of infinite order.
I somehow think the right way to handle this question is to think over $\mathbb{C}$, but I am not sure how to do this.
Best Answer
Hint If $T$ is an isometry of the inner product $(x, y) \mapsto \langle x, y \rangle$, then for any $P \in GL(V)$, $P^{-1} T P$ is an isometry of the inner product $(x, y) \mapsto \langle P x, P y \rangle$ (it is not hard to verify that this indeed defines an inner product, but you may wish to prove it anyway): Thus, the property that a given transformation admit such an inner product is invariant under similarity (that is, is an invariant of the conjugacy class of $T$ in $GL(V)$).
On the other hand, we already know canonical representatives of each similar class in $GL(V)$: These are given by the analogue of the Jordan normal form for real matrices. These matrices are relatively simple and so one can check more directly for a general matrix of this form whether such an inner product exists.
Instead of attacking this immediately, you might like to prove first the following lemmata:
One can essentially treat each "Jordan block" separately, or more precisely, given a direct sum decomposition $V = \oplus V_a$ and linear transformations $T_a : V_a \to V_a$, then there is an inner product on $V$ preserved by $T := \oplus T_a$ iff for each $a$ there is an inner product on $V_a$ preserved by $T_a$.
$T$ cannot have any nonsimple "Jordan blocks", that is, $T$ is block diagonal, where each block has the form $$\phantom{(\ast)} \qquad \begin{pmatrix} \lambda \end{pmatrix} \qquad \text{or} \qquad \begin{pmatrix} \alpha & -\beta \\ \beta & \alpha\end{pmatrix} \qquad (\ast).$$
The proof of (2) is essentially the one you give for your counterexample $\begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}.$
With these two facts in hand, it's essentially enough to solve the problem for the two transformations in $(\ast)$ above. Since the sole eigenvalue of $\begin{pmatrix} \lambda \end{pmatrix}$ is $\lambda$, by the first observation in the question a sufficient and necessary condition for this block is $\lambda \in \{\pm 1\}$.