Physics terminology about smeared and unsmeared fields

Let $M$ be a smooth manifold and denote $C^\infty_0(M)$ the space of smooth functions with compact support. In Mathematics a distribution is defined to be a continuous linear functional $\phi : C^\infty_0(M)\to \mathbb{R}$. The space of distributions is usually denoted $\mathfrak{D}'(M)$.

So a distribution is a map that takes a function and outputs a number in a linear and continuous way. The Delta distribution centered at $x\in M$ is for example

$$\delta_x[f]=f(x).$$

Another way to create distributions is to pick $f\in C^\infty_0(M)$ and define

$${f}^\diamond[g]=\int_M fg.$$

That much is fine. The problem is the following: in Physics one often forgets all this and treats distributions as functions. So a Physicst will almost never bother writing $\phi[f]$ or just $\phi$. They write $\phi(x)$ which is not really correct, since $\phi$ isn't a function on $M$ at all.

The issue though is that there is a terminology around which makes me quite confused. One often talks about "smeared" fields written as

$$\varphi[f]=\int_M\varphi(x)f(x)$$

and talks about the field in "unsmeared form" writting it just $\varphi(x)$. This confuses further, because it is known that it is not true that given $\varphi$ there is $f$ such that $\varphi = f^\diamond$.

This terminology may be found for example in Fewster's notes on QFT on curved spacetime, but I've seem it elsewhere.

This seems to imply that when one picks $\phi\in C^\infty_0(M)$ it is unsmeared and when one picks $\phi^\diamond\in \mathfrak{D}'(M)$ and apply it to a function it is smeared (but notice that $\phi^\diamond[f]$ is a real number, not even a field anymore after applying to $f$).

So what really is this smeared and unsmeared terminology about and how does this makes contact with distribution theory from mathematics?