Consider a symmetric matrix $H(t)$ parametrized smoothly by $t\in\mathbb R$.
Suppose there are orthonormal eigenvectors $\phi_i(t)$ with corresponding eigenvalues $\lambda_i(t)$:
$$
H(t)\phi_i(t)=\lambda_i(t)\phi_i(t).
\tag{1}
$$
Given $H(0)$ and $H'(0)$, I would like to find $\phi_i'(0)$.
If $H(0)$ has degenerate eigenvalues, I do not see how to find all components of the eigenfunctions corresponding to a degenerate eigenvalue at $t=0$.
Let me elaborate.
$\newcommand{\ip}[2]{\langle#1,#2\rangle}$
Let me drop the argument $(t)$ whenever $t=0$.
First, the eigenvectors are normalized, yielding
$$
\ip{\phi_i'}{\phi_i}=0.
\tag{2}
$$
Differentiating (1) with respect to $t$ at $t=0$ gives
$$
H'\phi_i
+H\phi_i'
=
\lambda_i'\phi_i
+\lambda_i\phi_i'.
\tag{3}
$$
Taking inner product with $\phi_i$ and using symmetry and (2) gives
$$
\lambda_i'=\ip{\phi_i}{H'\phi_i}.
\tag{4}
$$
These are the first order perturbations of the eigenvalues.
I also want to find the first order perturbation of the eigenvectors, $\phi_i'$.
Doing so is equivalent with finding the inner product $\ip{\phi_j}{\phi_i'}$ for all $j$.
Taking inner product of (3) with $\phi_j$ for $j\neq i$ gives
$$
\ip{\phi_j}{H'\phi_i}
=
(\lambda_i-\lambda_j)\ip{\phi_j}{\phi_i'}.
$$
Thus if $\lambda_j\neq\lambda_i$, we obtain
$$
\ip{\phi_j}{\phi_i'}
=
\frac{\ip{\phi_j}{H'\phi_i}}{\lambda_i-\lambda_j}.
\tag{5}
$$
On the other hand, if $\lambda_j=\lambda_i$, we obtain $\ip{\phi_j}{H'\phi_i}=0$.
That is, the block of $H'$ corresponding to the eigenspace of $H$ of eigenvalue $\lambda_i$ is diagonal.
We have freedom in choosing the orthonormal basis when the spectrum degenerates, and it must be chosen in this way to make $\phi_i(t)$ depend continuously on $t$.
Therefore the diagonal values in (4) are in fact eigenvalues of this block matrix.
For $i=j$ the desired inner product $\ip{\phi_j}{\phi_i'}$ is given by (2).
If $i\neq j$ and $\lambda_i\neq\lambda_j$, it is given by (5).
But what is the inner product $\ip{\phi_j}{\phi_i'}$ when $i\neq j$ and $\lambda_i=\lambda_j$?
It seems to me that the presented calculations give no constraints.
I have lost no information by taking the inner product of (3) with all $\phi_j$ and looking at all the inner products instead of the vector equation.
I must be missing something.
(This puzzled me for quite a while and I found no answer elsewhere, so I decided to share the question and an answer here. Actually, I figured it out while writing this question. Other answers are very much welcome!)
Best Answer
The given data does not determine the inner product $\newcommand{\ip}[2]{\langle#1,#2\rangle}\ip{\phi_j}{\phi_i'}$ when $i\neq j$ and $\lambda_i=\lambda_j$.
For a concrete example, let $$ H(t) = \begin{pmatrix} 2t&t^2\\t^2&0 \end{pmatrix}. $$ This has been chosen so that $H(0)=0$ is degenerate and $H'(0)$ is already diagonal. The normalized eigenvector (unique up to sign) corresponding to the eigenvalue $t(1-\sqrt{1+t^2})$ is $$ \frac1{\sqrt2} \begin{pmatrix} -\sqrt{1-\frac1{\sqrt{1+t^2}}} \\ \sqrt{1+\frac1{\sqrt{1+t^2}}} \end{pmatrix}. $$ The lower component is $1+O(t^2)$ and the upper one is $t/2+O(t^3)$.
However, if you take $H(t)=\operatorname{diag}(2t,0)$, the data $H(0),H'(0)$ is the same and $\phi_i(0)$ are also unchanged. However, $\phi_i'(0)$ does change: the eigenvectors are independent of $t$ now. (The inner product $\ip{\phi_i}{\phi_i'}$ is zero in both cases, the only difference is in the direction orthogonal to $\phi_i$.) The missing inner products depend on higher order derivatives of $H(t)$ at $t=0$.
The desired components can be found using second order perturbation theory, at least when $H'(0)$ is non-degenerate. Take two distinct indices $i$ and $j$ so that $\lambda_i=\lambda_j$. We also assume that $\lambda_i'\neq\lambda_j'$; this is the mentioned non-degeneracy assumption. Using the observations in the question, we find $H'\phi_j=\lambda_j'\phi_j$. This becomes useful once we differentiate (1) twice at $t=0$ and take an inner product with $\phi_j$. We find $$ \ip{\phi_j}{\phi_i'} = \frac{\ip{\phi_j}{H''\phi_i}}{2(\lambda_i'-\lambda_j')}. $$ The details are left as an exercise. If $H'$ is degenerate, I assume higher order derivatives are needed.