It is known that the Kitaev Hamiltonian and its spin-liquid ground state both break the SU(2) spin-rotation symmetry. So what's the spin-rotation-symmetry group for the Kitaev model?
It's obvious that the Kitaev Hamiltonian is invariant under π rotation about the three spin axes, and in some recent papers, the authors give the "group"(see the Comments in the end) G={1,eiπSx,eiπSy,eiπSz}, where (eiπSx,eiπSy,eiπSz)=(iσx,iσy,iσz), with S=12σ and σ being the Pauli matrices.
But how about the quaternion group Q8={1,−1,eiπSx,e−iπSx,eiπSy,e−iπSy,eiπSz,e−iπSz}, with −1 representing the 2π spin-rotation operator. On the other hand, consider the dihedral group D2={(100010001),(1000−1000−1),(−10001000−1),(−1000−10001)}, and these SO(3) matrices can also implement the π spin rotation.
So, which one you choose, G,Q8, or D2 ? Notice that Q8 is a subgroup of SU(2), while D2 is a subgroup of SO(3). Furthermore, D2≅Q8/Z2, just like SO(3)≅SU(2)/Z2, where Z2={(1001),(−100−1)}.
Comments: The G defined above is even not a group, since, e.g., (eiπSz)2=−1∉G.
Remarks: Notice here that D2 can not be viewed as a subgroup of Q8, just like SO(3) can not be viewed as a subgroup of SU(2).
Supplementary: As an example, consider a two spin-1/2 system. We want to gain some insights that what kinds of wavefunctions preserves the Q8 spin-rotation symmetry from this simplest model. For convenience, let Rα=e±iπSα=−4Sα1Sα2 represent the π spin-rotation operators around spin axes α=x,y,z, where Sα=Sα1+Sα2. Therefore, by saying a wavefunction ψ has Q8 spin-rotation symmetry, we mean Rαψ=λαψ, with |λα|2=1.
After a simple calculation, we find that a Q8 spin-rotation symmetric wavefunction ψ could only take one of the following 4 possible forms:
(1)|↑↓⟩−|↓↑⟩, with (λx,λy,λz)=(1,1,1) (Singlet state with full SU(2) spin-rotation symmetry), which is annihilated by Sx,Sy, and Sz,
(2)|↑↓⟩+|↓↑⟩, with (λx,λy,λz)=(−1,−1,1), which is annihilated by Sz,
(3)|↑↑⟩−|↓↓⟩, with (λx,λy,λz)=(1,−1,−1), which is annihilated by Sx,
(4)|↑↑⟩+|↓↓⟩, with (λx,λy,λz)=(−1,1,−1), which is annihilated by Sy.
Note that any kind of superposition of the above states would no longer be an eigenfunction of Rα and hence would break the Q8 spin-rotation symmetry.
The set G gives the representation of the identity and generators of the abstract group of quaternions as elements in SL(2,C) which are also in SU(2). Taking the completion of this yields the representation Q8 of the quaternions presented in the question.
From the description of the symmetry group as coming from here, consider the composition of two π rotations along the ˆx, ˆy, or ˆz axis. This operation is not the identity operation on spins (that requires a 4π rotation). However, all elements of D2 given above are of order 2.
This indicates that the symmetry group of the system should be isomorphic to the quaternions and Q8 is the appropriate representation acting on spin states. The notation arising there for D2 is probably from the dicyclic group of order 4×2=8 which is isomorphic to the quaternions.