Photon spin projection to arbitrary axis

For photons (and any massless particle) we consider only a spin projection into the direction of motion (helicity). Why it's meaningless to talk about projection of photon's spin into some arbitrary direction? Is this because we can't measure it (photon does not have a rest frame)?

Answer

The reason is indeed kind of related to the absence of the rest frame.

The angular momentum with respect to axis $x$ acting on a state (object) $|\psi\rangle$ is given by $$ J_x |\psi\rangle = \lim_{\Delta \phi\to 0} \frac{i\hbar}{\Delta\phi} \left(|\psi\rangle_{{\rm rotated\,\,by\,\,}\Delta\phi\,\,{\rm around\,\,}x} - |\psi\rangle \right) $$ So the state $|\psi\rangle$ may only be an eigenstate of $J_x$ – it has a well-defined value of $J_x$ – if its wave function remains essentially unchanged, up to a modified phase (which is not directly physically measurable) when we rotate the object around the $x$ axis.

If the particle is in the rest frame, its momentum is $\vec p=0$. In that case, the only changes of $|\psi\rangle$ induced by rotations are those that have something to do with the polarization vectors or spinors, i.e. with the intrinsic (spin) part of the angular momentum.

However, if $\vec p\neq 0$, and the momentum of a photon is inevitably nonzero because photons can't have a rest frame, then the rotation around $x$ also changes the value of $\vec p$, assuming that $\vec p$ is pointing in a different direction than $x$. This is equivalent to saying that there is also a nonzero orbital angular momentum, $\vec L=\vec r\times \vec p$.

So the rotation maps $|\psi\rangle$ into a completely different state, one with a different direction of $\vec p$. Consequently, the state isn't an eigenstate of rotations around $x$ and it is therefore not an eigenstate of $J_x$, either.

For massless particles, one may only find eigenvectors of $J_x$ for particle states whose momentum $\vec p$ goes along the same axis $x$, i.e. one may find eigenvalues of $\vec J\cdot \vec p / |\vec p|$, known as the helicity.

Only the total angular momentum is conserved. But even if you tried to artificially separate the spin of the photon (and similarly Weyl neutrinos) from its orbital angular momentum, you will fail to define the spin with respect to other directions than the direction of motion. It's because the polarization vectors $\vec \epsilon$ of photons are transverse to the direction of motion so there don't exist any physical states of the photons at all that would have $\vec \epsilon$ parallel to $\vec p$. Such longitudinal states would be needed to define all $SO(3)$ rotations of a given photon state, i.e. to study the transformation of the state under all components of $\vec J$.