quantum mechanics - Why we use $L_2$ Space In QM?

I asked this question for many people/professors without getting a sufficient answer, why in QM Lebesgue spaces of second degree are assumed to be the one that corresponds to the Hilbert vector space of state functions, from where this arises? and why 2-order space that assumes the following inner product:

$\langle\phi|\psi\rangle =\int\phi^{*}\psi\,dx$

While there is many ways to define the inner product.

In Physics books, this always assumed as given, never explains it, also I tried to read some abstract math books on this things, and found some concepts like "Metric weight" that will be minimized in such spaces, even so I don't really understand what is behind that, so why $L_2$? what special about them? Who and how physicists understood that those are the one we need to use?

Answer

Here we will assume that OP is not questioning the fundamental physical principles/postulates/axioms of quantum mechanics, such as, e.g., the need to have a Hilbert space $H$ in the first place, etc; and that OP is only pondering the role of $L^2$-spaces (as opposed to, e.g., $L^1$-spaces).

Let us for concreteness and simplicity consider the 3-dimensional position space $\mathbb{R}^3$. One uses the $L^2$-space $H=L^2(\mathbb{R}^3)$ as a Hilbert space for various reasons:

1. 
   

   To have a well-defined norm

   $$\tag{1} ||\psi||_p~:=~\left(\int d^3x ~ |\psi(x)|^p\right)^{\frac{1}{p}}, \qquad p~=~2.$$ [The norm (1) actually works for any $L^p$-space $L^p(\mathbb{R}^3)$ with $p\geq 1$.]
   
   


2. 
   

   To have a well-defined inner product/sesqui-linear form,

   $$\tag{2} \langle \phi, \psi\rangle ~:=~\int d^3x ~ \phi^*(x)\psi(x).$$ In particular, the integrand $\phi^*\psi$ should be integrable, i.e. i) Lebesgue measurable, and ii) the absolute-valued integrand should have a finite integral: $$\tag{3} \int d^3x ~ |\phi^*(x)\psi(x)|~<~\infty.$$ Proof of eq.(3): Notice the inequality $$\tag{4} (|\phi(x)|-|\psi(x)|)^2 \geq 0\qquad \Leftrightarrow\qquad 2|\phi(x)^*\psi(x)| \leq |\phi(x)|^2+|\psi(x)|^2,$$ so that the integrand $\phi^*\psi$ in the inner product (2) becomes integrable $$\tag{5} 2\int d^3x ~ |\phi^*(x)\psi(x)|~\stackrel{(1,4)}{\leq}~ ||\phi||^2_2+||\psi||^2_2~<~\infty,$$ because we demand that $\phi$ and $\psi$ are square integrable, i.e. that $\phi,\psi\in L^2(\mathbb{R}^3)$. Note in particular that eq.(3) does not hold in general for $\phi,\psi\in L^p(\mathbb{R}^3)$ with $p \neq 2$.
   
   


3. 
   

   To ensure that the normed vector space $H$ is complete. See also this Phys.SE answer. [This actually works for any $L^p$-space $L^p(\mathbb{R}^3)$ with $p\geq 1$.]

   
   
   


4. 
   

   To make sure that e.g. the set $C^{\infty}_c(\mathbb{R}^3)$ of infinitely many times differentiable functions with compact support are included in the space $H$. [This actually works for any $L^p$-space $L^p(\mathbb{R}^3)$ with $p\geq 1$.]

   
   


5. 
   

   Note that all the other $L^p$-spaces $L^p(\mathbb{R}^3)$ with $p\neq 2$ are not Hilbert spaces (although they are Banach spaces). This is related to the fact that the dual $L^p$-space is $L^p(\mathbb{R}^3)^*\cong L^q(\mathbb{R}^3)$ where $\frac{1}{p}+\frac{1}{q}=1$. Hence an $L^p$-space is only selfdual if $p=2$. Selfduality implies that there is an isomorphism between kets and bras.

   
   


6. 
   

   It is true that other Hilbert spaces (modeled over the position space $\mathbb{R}^3$) do exist, but they would typically rely on additional structure. (E.g., one could use another integration measure $d\mu$ than the Lebesgue measure $d^3x$.)

   
   

In conclusion, the $L^2$-space $H=L^2(\mathbb{R}^3)$ is the simplest and most natural/canonical choice.