quantum mechanics - Hilbert space of harmonic oscillator: Countable vs uncountable?

Hm, this just occurred to me while answering another question:

If I write the Hamiltonian for a harmonic oscillator as $$H = \frac{p^2}{2m} + \frac{1}{2} m \omega^2 x^2$$ then wouldn't one set of possible basis states be the set of $\delta$-functions $\psi_x = \delta(x)$, and that indicates that the size of my Hilbert space is that of $\mathbb{R}$.

On the other hand, we all know that we can diagonalize $H$ by going to the occupation number states, so the Hilbert space would be $|n\rangle, n \in \mathbb{N}_0$, so now the size of my Hilbert space is that of $\mathbb{N}$ instead.

Clearly they can't both be right, so where is the flaw in my logic?

Answer

This question was first posed to me by a friend of mine; for the subtleties involved, I love this question. :-)

The "flaw" is that you're not counting the dimension carefully. As other answers have pointed out, $\delta$-functions are not valid $\mathcal{L}^2(\mathbb{R})$ functions, so we need to define a kosher function which gives the $\delta$-function as a limiting case. This is essentially done by considering a UV regulator for your wavefunctions in space. Let's solve the simpler "particle in a box" problem, on a lattice. The answer for the harmonic oscillator will conceptually be the same. Also note that solving the problem on a lattice of size $a$ is akin to considering rectangular functions of width $a$ and unit area, as regulated versions of $\delta$-functions.

The UV-cutoff (smallest position resolution) becomes the maximum momentum possible for the particle's wavefunction and the IR-cutoff (roughly max width of wavefunction which will correspond to the size of the box) gives the minimum momentum quantum and hence the difference between levels. Now you can see that the number of states (finite) is the same in position basis and momentum basis. The subtlety is when you take the limit of small lattice spacing. Then the max momentum goes to "infinity" while the position resolution goes to zero -- but the position basis states are still countable!

In the harmonic oscillator case, the spread of the ground state (maximum spread) should correspond to the momentum quantum i.e. the lattice size in momentum space.

The physical intuition

When we consider the set of possible wavefunctions, we need them to be reasonably behaved i.e. only a countable number of discontinuities. In effect, such functions have only a countable number of degrees of freedom (unlike functions which can be very badly behaved). IIRC, this is one of the necessary conditions for a function to be fourier transformable.

ADDENDUM: See @tparker's answer for a nice explanation with a slightly more rigorous treatment justifying why wavefunctions have only countable degrees of freedom.