thermodynamics - Definition of temperature ambiguity

Temperature is defined as $$\begin{equation} \frac{1}{T} \equiv \left(\frac{\partial S}{\partial U}\right)_{N,V}. \end{equation}$$ However, a system with a fixed temperature does not have a fixed amount energy, rather the ensemble average of the energy of the system is fixed. This means that the definition of temperature is ambiguous unless it's made clear what we mean by the internal energy $U$. In the definition of temperature, does $U$ represent the ensemble average of the energy of the system?