thermodynamics - Why the temperature is getting lower when the universe is expanding

As we know, if an ideal gas expands in vacuum, as its energy is unchanged, the temperature remains the same. An ideal gas's energy does not depend on volume. In general, the energy is $kT$ times the total degrees of freedom, like in an ideal gas, the total degrees of freedom is $N$ particles plus three dimensions, $3N$.

Then if the total energy of the universe is $kT$ times the total degrees of freedoms of the universe, as the universe expands, its energy and entropy should not change, but if the temperature falls, the number of degrees of freedom should grow. It is quite puzzling to me that the universe is having more and more new degrees of freedom. It seems to be contradictory to the entropy argument.