quantum mechanics - Half-integer eigenvalues of orbital angular momentum

Why do we exclude half-integer values of the orbital angular momentum? It's clear for me that an angular momentum operator can only have integer values or half-integer values. However, it's not clear why the orbital angular momentum only has integer eigenvalues. Of course, when we do the experiments we confirm that a scalar wavefunction and integer spherical harmonics are enough to describe everything. Some books, however, try to explain the exclusion of half integer values theoretically. Griffiths evokes the "single valuedness" argument, but he writes that the argument is not so good in a footnote. Shankar says that the $L_z$ operator only is Hermitian when the magnetic quantum number is an integer, but his argument isn't so compelling to me. Gasiorowicz argues that the ladder operators don't work properly with half-integer values. There are some low impact papers (most of them are old) that discuss these subjects, although they are a little bit confusing.

So, basically, my question is: Does anyone have a decisive argument on why do we exclude the half-integer values from the orbital operator spectrum?