Using robphy's answer to my initial question, I try to implement what I understand to be his suggested approach, and I get it to almost work, but I get a small error at the end that I hope someone can spot.
As a reminder, I am using the wave 4-vector definition
$$\kappa^\mu = \frac{2\pi}\lambda (1,\mathbf n) $$
where $\pmb n$ is the unit 3-vector in the direction photon motion and $\lambda$ is the photon's wavelength. To be a spacetime 4-vector, my text book requires an object $\kappa^\mu$ to satisfy the Lorentz transformation equation $\kappa^{\mu^\prime}=\Lambda^{\mu^\prime}_\nu\kappa^\nu$ between inertial frames K and $K^\prime$.
I define the inertial frame $K^\prime$ to be a boost with speed v in the $x^1$ direction of frame K. A photon is fired from a source at the origin $O^\prime$ of frame $K^\prime$, travels in the $x^1x^2$ plane, and arrives at the origin $O$ of frame K along a path that makes an angle $\theta$ with the $x^1$-axis. I set the speed of frame $K^\prime$ to be $v = c\;cos\,\theta.$
$K^\prime$ offset from Frame K" />
Because the speed v is less than c (in my original attempt, I tried to use speed v = c for the photon), we can now define $$\gamma\equiv\frac{1}{\sqrt{1-\frac{v^2}{c^2}}}=\frac{1}{\sqrt{1-\frac{c^2 cos^2\theta}{c^2}}}=\frac{1}{sin\theta}.$$
The Lorentz equation is then the boost equation with offset, namely
$$x^{\mu^\prime}=\Lambda^{\mu^\prime}_\nu x^\nu+a^\mu$$
where $a^\mu$ is the offset and the matrix form of the Jacobian is
$$\Lambda^{\mu^\prime}_\nu =\begin{pmatrix}
\gamma & -\frac{\gamma v}{c} & 0 & 0 \\
-\frac{\gamma v}{c} & \gamma & 0 & 0 \\
0 & 0 & 1 & 0 \\
0 & 0 & 0 & 1 \\
\end{pmatrix}\;.$$
An observer in frame $K^\prime$ sees the photon travel along the $x^{2^\prime}$-axis from $O^\prime$ to $O$, so that $\mathbf n^\prime$ is a unit vector along the $x^{2^\prime}$ axis as shown. An observer in frame K sees the photon travel diagonally and so its unit vector $\mathbf n$ is on the diagonal path as shown. So,
$$\mathbf n=(cos\theta, sin\theta, 0),\qquad \mathbf n^\prime=(0, 1, 0),$$
$$k^\mu=\frac{2\pi}{\lambda}(1,\mathbf n)=\frac{2\pi}{\lambda}(1,cos\theta,sin\theta,0),$$
$$k^{\mu^\prime}=\frac{2\pi}{\lambda^\prime}(1,\mathbf n^\prime)=\frac{2\pi}{\lambda^\prime}(1,0,1,0).$$
Also,
$$\frac{\gamma v}{c}=\frac{1}{c}\frac{1}{sin\theta}\,(c\;cos\theta)=\frac{cos\theta}{sin\theta.}$$
So,
$$\Lambda^{\mu^\prime}_\nu =
\begin{pmatrix}\frac{1}{sin\theta} & -\frac{cos\theta}{sin\theta} & 0 & 0 \\ -\frac{cos\theta}{sin\theta} & \frac{1}{sin\theta} & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\
\end{pmatrix},$$
and
$$\Lambda^{\mu^\prime}_\nu k^\nu=\begin{pmatrix} sin\theta \\ 0 \\sin\theta \\ 0 \\ \end{pmatrix}$$
and, so, the Lorentz transformation for $\kappa$ is
$$\frac{2\pi}{\lambda^\prime}\;\begin{pmatrix} 1 \\0 \\1 \\0 \\ \end{pmatrix}=\frac{2\pi}{\lambda}\;\begin{pmatrix} sin\theta \\0 \\sin\theta \\0 \\ \end{pmatrix}.$$
Equations 2 and 4 are 0 = 0 and equations 1 and 3 are $\frac{2\pi}{\lambda^\prime}=\frac{2\pi}{\lambda}\,sin\theta, or,$
$$\frac{\lambda^\prime}{\lambda}=\frac{1}{sin\theta}.$$
That is, $\kappa$ satisfies this Lorentz transformation if and only if this ratio is true. So, my last step is to confirm the ratio but I get that
$$\frac{\lambda^\prime}{\lambda}=sin\theta.$$
so either my proof or my confirmation (or both) has a mistake that I hope someone can spot.
I get my confirmation result by using the boost equation with offset, given just under the figure. I use just the equation for $x^{0^\prime}$. To use the equation, we must identify the offset $a^0$, which represents the time instant when origin O lies on the (moving) $x^{2^\prime}$ axis. WLOG we define that to be time $t=a^0=0$. With that convention the $x^{0^\prime}$ equation is
$$c t^{\prime}=x^{0^\prime}=\gamma x^0 – \gamma \frac{v}{c}x^1 + a^0=\gamma(c t – \frac{c\: cos\theta}{c} x^1)= \gamma (c t – cos\theta\; x^1).$$
If we set $t=t_\lambda$, the time in frame K for the photon to travel 1 wavelength, the corresponding time for one wavelength in $K^\prime$ is $t^\prime=t_{\lambda^\prime}$, and the $x^1$ distance, which corresponds to one wavelength $\lambda$ along the vector n, is $x^1=\lambda\; cos\theta$. So,
$$\lambda^\prime=c\; t_{\lambda^\prime}=\gamma (c t_\lambda – cos\theta\; x^1)=\gamma(\lambda – \lambda cos^2\theta)= \gamma \lambda sin^2\theta$$
which implies that
$$\frac{\lambda^\prime}{\lambda}=\gamma sin^2\theta=\frac{sin^2\theta}{sin\theta}=sin\theta,$$
not quite what I was trying to show.
Best Answer
The Lorentz Transformations are used to map between inertial frames of timelike-observers. The $"v"$ in the boost transformation is the relative spatial-velocity between the frames. In some sense, the boost transforms to a frame in which one of the timelike inertial observers is at rest.
However, the photon does not have a timelike 4-momentum... it has a lightlike one. So, one cannot use the spatial-velocity of the photon as $"v"$ in the transformation (since one can't transform into a frame where the photon is at rest... no such timelike-frame exists.)
However, the Lorentz Transformation still applies... between two inertial timelike-observers. So, pick two such that their 4-velocities are co-planar [in spacetime] with the photon 4-momentum. (That is, choose inertial observers such that one observer can use this photon to signal the other.)
In fact, if one solves the eigenvalue problem for the boost transformation, one finds that your photon 4-momentum is an eigenvector [the principle of the speed of light], and the eigenvalue is associated with the relative-Doppler factor between frames. (One ends up with light-cone coordinates, which underlies the Bondi k-calculus.)
My answer is primarily about the (1+1)-case since that already addresses the essence of your question.