newtonian mechanics - Physical intuition about the inertia tensor

I'm studying Mechanics on Goldstein's book (Classical Mechanics) and Spivak's book (Physics for Mathematicians) and I'm in doubt about the physical intuition about the inertia tensor. On both books, the inertia tensor appears naturally when computing the angular momentum $L$ of a rigid body which, for simplicity, is only rotating.

The inertia tensor is then defined as the linear operator $I : \mathbb{R}^3 \to \mathbb{R}^3$ given by

$$I(\phi) = \sum_{i} m_i b_i \times (\phi \times b_i),$$

where $b_i\in \mathbb{R}^3$ are the initial positions of the particles of the body, and $m_i$ their masses. With this definition, it is shown that

$$L = I(\omega),$$

being $\omega$ the angular velocity of the rigid body. All of that, from the mathematical point of view, is fine.

Now, what is the physical intuition behind this? The linear operator $I$ allows one to relate, in a linear way the angular velocity and the angular momentum. This looks much like mass relates in a linear way velocity and momentum. But on the latter case, mass is a scalar while $I$ is a linear transformation.

What is, then, the best way to physically understand the inertia tensor?