Here is a general guideline for $2 \times 2$ orthogonal matrices.
They have one of the two forms
$$\text{Either} \ \ R = \begin{bmatrix}
a &-b\\[0.3em]
b & \ \ \ a\\[0.3em]
\end{bmatrix} \ \ \ \ \text{or} \ \ \ \ S = \begin{bmatrix}
a & \ \ \ b\\[0.3em]
b & -a\\[0.3em]
\end{bmatrix}$$
with norm $1$ column vectors (thus $a^2+b^2=1$), the first case with $\det(A)=a^2+b^2=1$, the second with $\det(A)=-(a^2+b^2)=-1$.
More precisely, they have the form (you have cited the first one, the second one is less known...):
$$R_{\theta} = \begin{bmatrix}
\cos(\theta) & -\sin(\theta)\\[0.3em]
\sin(\theta) & \ \ \ \cos(\theta)\\[0.3em]
\end{bmatrix} \ \ \ \ \ \ \text{or} \ \ \ \ \ \ S_{\alpha}=\begin{bmatrix}
\cos(2 \alpha) & \ \ \ \sin(2 \alpha)\\[0.3em]
\sin(2 \alpha) & -\cos(2 \alpha)\\[0.3em]
\end{bmatrix} $$
where $\theta$ is the rotation angle, of course, and $\alpha$ is the polar angle of the axis or symmetry i.e., the angle of one of its directing vectors with the x-axis.
Thus, for your question, once you have recognized that a matrix is a symmetry matrix, it suffices to pick the upper left coefficient $ \cos(2 \alpha)$ and identify the possible $\alpha$s, with a disambiguation brought by the knowledge of $ \sin(2 \alpha)$.
I assume that an improper rotation means an element of the orthogonal group with determinant $=-1$. In other words, an $n$-dimensional improper rotation is represented by a matrix $R$ such that $RR^T=I_n$ and $\det R=-1$.
All such matrices have $\lambda=-1$ as an eigenvalue. This is because
$$\det(R+I)=\det(R+RR^T)=\det R \det (I+R^T)=-\det(I+R^T)=-\det(R+I),$$
which implies that $\det(R+I)=0$.
This has the following corollary
A 2-dimensional improper rotation is just the orthogonal reflection w.r.t. a line through the origin.
Proof. We saw above that $\lambda_1=-1$ is an eigenvalue of $R$. If $\lambda_2$ is the other eigenvalue, then $\lambda_1\lambda_2=\det R=-1$, so we can conclude that $\lambda_2=1$. If $\vec{u}$ is an eigenvector belonging to $\lambda_2$, and $\vec{v}\perp\vec{u}$ is another unit vector, then (because $R$ preserves lengths and angles) we can conclude that $R\vec{v}\perp R\vec{u}$.
But here $R\vec{u}=\vec{u}$, and in 2D the only unit vectors $\perp\vec{u}$ are $\pm\vec{v}$. So we can conclude that $R\vec{v}=\pm\vec{v}$. The plus sign cannot occur, for then we would have $R=I_2$. Therefore $R\vec{v}=-\vec{v}$. This implies that $R$ is the orthogonal reflection w.r.t. the line spanned by $\vec{u}$.
Best Answer
I understand your statement now. I will summarize my answers here.
As I mentioned in my comments, $A$ is a rotation matrix since it can be written in the "standard" form under the new basis. In other words, it rotate any vector about the axis that is in the direction of $u_1$.
Notice that this last statement "it rotate any vector about the axis that is in the direction of $u_1$" is independent of the basis.
Your statement "to prove that there's a rotation such that for an orthonormal basis, applying the rotation to vectors of that basis gives A1, A2 and A3" is also correct. Because it also says the columns of $A$ is an orthonormal basis, which is true. But I think this is kind of a confusing way to prove a matrix is a rotation. Also you'll have to construct another rotation to prove it. And in what form would you construct another rotation?
A more straightforward way, as above, is to prove that the matrix $A$ rotates vectors.
Edit:
This is with regard to the matrix $A$ represented in terms of basis $u_1, u_2, u_3$. We can write a linear transformation $T$ as a matrix in terms of any basis using the following way. Given basis $e_1, e_2, e_3$, the matrix in terms of this basis is calculated by $$[Te_1\quad Te_2 \quad Te_3]$$
where $Te_i$ is the column coordinate vector after you apply $T$ to $e_i$ in terms of this basis.
Looking at the question in this way, we see that $Au_1=\begin{pmatrix}1\\0\\0\end{pmatrix}$, $Au_2=\begin{pmatrix}0\\ \cos\theta\\ \sin\theta\end{pmatrix}$, $Au_3=\begin{pmatrix}0\\ -\sin\theta\\ \cos\theta\end{pmatrix}$.
Then $$\begin{bmatrix} 1 & 0 & 0 \\ 0 & \cos\theta & -\sin\theta \\ 0 & \sin\theta & \cos\theta \\ \end{bmatrix}$$
is exactly the matrix $A$ in terms of the basis $u_1, u_2, u_3$.