Clearly the adjoint of $T$ is defined as
$$
T^*x:=\sum\limits_{n=1}^\infty\overline{\lambda}_n\langle x,x_n\rangle x_n.
$$
To see that, note that $\langle Tx,x_n\rangle=\lambda_n\langle Tx,x_n\rangle$ and
$$
\langle x,y\rangle=\sum_{n=1}^\infty \langle x,x_n\rangle\overline{\langle y,x_n\rangle},
$$
and hence
$$
\langle Tx,y\rangle=\sum_{n=1}^\infty \langle Tx,x_n\rangle\overline{\langle y,x_n\rangle}
=\sum_{n=1}^\infty \lambda_n\langle x,x_n\rangle\overline{\langle y,x_n\rangle},
$$
while
$$
\langle x,T^*y\rangle=\sum_{n=1}^\infty \langle x,x_n\rangle\overline{\langle T^*y,x_n\rangle}
=\sum_{n=1}^\infty \langle x,x_n\rangle\overline{\bar\lambda_n\langle y,x_n\rangle},
=\sum_{n=1}^\infty \lambda_n\langle x,x_n\rangle\overline{\langle y,x_n\rangle}.
$$
This is not a geometric intuition. Nevertheless I find it pretty enlightning, so I hope to be helpful.
Given a linear operator $T:V\to V$, you can define the transpose operator:
$$T^t:V^*\to V^*$$
$$\ \ \ \ \ \ \ \ \ \varphi \mapsto \varphi\circ T$$
The transpose operator is essentially the simplest way you can imagine to construct an operator on $V^*$ using an operator $T$ on $V$.
If you understand this then the adjoint is straight forward!
In fact the idea is to use the fact that $V$ is isomorphic to its dual $V^*$. Normally there aren't natural isomorphism between a space and its dual, but in this case you are in a inner product (finite dimensional ) space!
So you can construct a canonical isomorphism(actually if the field is $\mathbb{C}$, this is conjugate-linear):
$$\omega:V\to V^*$$
$$\ \ \ \ \ \ \ \ \ v \mapsto \langle \bullet,v\rangle=\omega_v$$
(Basically you associate to a vector $v$ the linear functional $w\mapsto \langle w,v \rangle$).
Now the adjoint is simply the transpose "re-interpreted" through this natural isomorphism:
$$T^*=\omega^{-1}\circ T^t \circ \omega:V\to V$$
As you can see the adjoint is simply a natural reinterpretation of the transpose, so if you find the transpose intuitive, you should also find the adjoint intuitive.
Clearly I should prove that my definition of $T^*$ coincides with yours, and this is straightforward:
$$\langle T(x),y \rangle=\langle x,T^*(y) \rangle \ \text{for all } x,y \iff $$
$$\iff \omega_y(T(x))=\omega_{T^*(y)}(x) \ \text{for all } x,y \iff $$
$$\iff \omega_y\circ T=\omega_{T^*(y)} \ \text{for all } y \iff $$
$$\iff T^t(\omega_y)=\omega_{T^*(y)} \ \text{for all } y \iff $$
$$\iff (T^t\circ \omega)(y)=(\omega \circ T^*)(y) \ \text{for all } y \iff $$
$$\iff T^t\circ \omega=\omega \circ T^* \iff $$
$$\iff T^*=\omega^{-1}\circ T^t \circ \omega $$
Best Answer
This problem is what I met when reading the book A course in robust control theory. The result is provided but not giving the proof.
However, I find this result may not be correct. The adjoint vector of $Q^*$ should be in this way: $$(Q^*z)(t):=\int_0^\infty f^*(\tau-t)z(\tau)d\tau.$$