diffusion - Why is Johnson noise a Gaussian process?

Noise processes in engineering and physics are frequently assumed to be Gaussian processes. This allows use of convenient analytical techniques. The question then arises as to why natural processes are Gaussian. In particular I'd like to understand why electrical Johnson noise is a Gaussian process.

---

Possible line of reasoning

One line I've found is that the Ornstein-Uhlenbeck process is Gaussian and satisfies the Fokker Planck equation. This could suggest that any physical process obeying the Fokker Planck equation is an Ornstein-Uhlenbeck process, and therefore a Gaussian process. However, this still leaves the question of why Johnson noise obeys the Fokker Planck equation.

An obvious related question is whether or not Johnson noise is an Ornstein-Uhlenbeck process.

Answer

There are several ways I can interpret the question, so my main focus is going to be on the autocorrelation of an Ornstein-Uhlenbeck (O-U) process. So what is an O-U process and how is it different from regular Brownian diffusion?

The stochastic differential equation (SDE) for Brownian diffusion of a particle can be written as

$$\mathrm{d}x_t = \mathrm{d}W_t$$ where $x$ is displacement and $\mathrm{d}W$ a stochastic process such that is we integrate both sides, $x_T - x_0 = \int_0^T\mathrm{d}W_t = \mathcal{N}(0, T)$, the Gaussian distribution with mean $0$ and variance $T$. A bit more physics-y notation would be $$\frac{\mathrm{d}x}{\mathrm{d}t} = \eta$$ where $\eta$ is a Gaussian random variable.

So you can see in your mind's eye that you as you keep adding these small random displacements to random directions, you're going to end up with diffusion. Another way to describe diffusion, and more common to physicists, is a partial differential equation (the Fokker-Planck equation), where you write the probability distribution of the particle as a function of time and position. Yet another way to write this is as a Wiener path integral (the transformation between these representations go through the Feynman-Kac equation).

Finally, moving to the O-U equation:

$$\mathrm{d}x_t = -x_t\mathrm{d}t + \mathrm{d}W_t$$ or if you prefer $$\frac{\mathrm{d}x}{\mathrm{d}t} = -x + \eta$$ How does the new term change the behaviour? Instead of diffusing to infinity, you are constraining the particle with a linear force. In other words, you have a particle in a harmonic potential, and some noise added to the position. The solution is only a change of variables away: $x_t = e^{-t}y_t$, so by Ito's lemma we have: $$e^{-t}\mathrm{d}y_t - e^{-t}y_t\mathrm{d}t = \mathrm{d}x_t = -e^{-t}y_t\mathrm{d}t + \mathrm{d}W_t$$ so $$y_T-y_0 = \int_0^Te^s\mathrm{d}W_s$$ that is $$x_T = x_0e^{-T} + \int_0^Te^{(s-T)}\mathrm{d}W_s = \mathcal{N}\left(x_0e^{-T}, \frac{1}{2}(1-e^{-2T})\right)$$ where the last step just follows from substituting to the definition of expectation value and variance.

So what do we see? We see that our observation $x_T$ at time $T$ depends on where we last saw $x$. But that this dependence (autocorrelation) dies out exponentially. Indeed, after an infinite amount of time, we have $x_T = \mathcal{N}(0,1)$. Note that we did not get $x_T = \mathcal{N}(0,T)$, but rather that our Gaussian is constrained in size and will get to be that size at an exponential speed (as the effect of the last observation dies out).

Ok, so you have a resistor in series with an inductor over some voltage and you write

$$L\frac{\mathrm{d}I}{\mathrm{d}t} = -RI + V$$ Look familiar? Suppose the voltage is nearly zero, but as the charge carriers themselves undergo diffusion, this voltage is actually stochastic. That's an O-U process. Now, suppose you measure the current at time 0, and then again at time $T$. In what sense are these Gaussian and independent? As discussed in the previous section, if you wait long enough, each measurement is going to be independent from one another and they are both going to sample the same, finite distribution (as opposed to infinitely spreading distribution we saw in diffusion). This Gaussianity and its autocorrelation is what I believe you were after. Finally, let's turn to something I don't think you were after, but may think as interesting nonetheless.

Now why should $V$ be stochastic, and why should it be Gaussian? This is a longer discussion and I am only going to very qualitatively and briefly go over it. Suppose we have some simple system: A particle in a potential and a large number of harmonic oscillators (suggestively called "the heat bath") coupled to it, so that we might write the Hamiltonian (a bit simplified from the Caldeira Leggett model) as

$$\mathcal{H} = \frac{p^2}{2m} + V(q) + \sum_i \left(\frac{p_i^2}{2m_i} + \frac{1}{2}m_i\omega_i^2x_i^2\right) + q\sum_ix_i$$

Turns out that this can actually be turned into a generalized Langevin equation of the form

$$\frac{\mathrm{d}^2q}{\mathrm{d}t^2} = -\frac{\mathrm{d}V}{\mathrm{d}q}-\int_0^T\mathrm{d}t\frac{\mathrm{d}q}{\mathrm{d}t}\xi(T-t) + R(t)$$ where the memory kernel $\xi(T-t)$ and the "random force" $R(t)$ are both some functions of the stuff in the Hamiltonian. What is remarkable is that the random force, while deterministic, behaves exactly as if it were a "real" Gaussian thermal force (fluctuation-dissipation theorem, its correlation with the other quantities etc). So, we're just going to say that it is random and in this way integrate out some degrees of freedom that we don't care about.

This can be done more rigorously for basically arbitrary systems describing some phase space evolution by using Mori-Zwanzig theory. There you take a projection operator, which projects onto the subspace that contains the degrees of freedom of interest, and the other degrees of freedom then behave like a thermal bath. Many books make this out to be something very complicated, but it's really just matrix (or really, operator) algebra.

Point being that if you have a system and have perfect knowledge of the entire phase space, if you drop out some details, those details will in many ways act as if there was a Gaussian thermal bath pushing the system around.