mathematical physics - Why is a Hermitian operator a "quantum random variable"?

To me, as a stupid mathematician, a random variable is a measurable function from some probability space $(\Omega, \sigma, \mu)$ to $(\Bbb{R}, B(\Bbb{R}))$. This makes sense. You have outcomes, events, and probabilities of these events. A random variable is just assigning these numbers.

I took QM as an undergrad and I remember computing eigenvalues, expectations, etc. of various operators and I never quite got what an operator is in QM. A (quantum) random variable is a Hermitian operator on some Hilbert space. You can compute probabilities and expectations by some formulas involving eigenvectors and orthogonal projections.

I must admit, even after my QM class, I don't get this. Is this random variable in any way related to my ignorant definition? Could we model say a coin flip or dice roll using this? Or is this type of random variable only for quantum things? Why not model all of quantum mechanics using my stupid definition? I know in quantum mechanics we can allow negative probabilities, so why not just have some signed measure space and forget all the business about Hilbert spaces?

Answer

I'm going to try to explain why and how density operators in quantum mechanics correspond to random variables in classical probability theory, something none of the other answers have even tried to do.

Let's work in a two-dimensional quantum space. We'll use standard physics bra-ket notation. A quantum state is a column vector in this space, and we'll represent a column vector as $\alpha|0\rangle + \beta |1 \rangle.$ A row vector is $\gamma \langle 0 | + \delta \langle 1 |\,$.

Now, you might think that a probability distribution is a measure on quantum states. You can think of it that way, but it turns out that this is too much information. For example, consider two probability distributions on quantum states. First, let's take the probability distribution

$$ \begin{array}{cc} |0\rangle & \mathrm{with\ probability\ }2/3,\\ |1\rangle & \mathrm{with\ probability\ }1/3. \end{array} $$

Next, let's take the probability distribution $$ \begin{array}{cc} \sqrt{{2}/{3}}\,\left|0\right\rangle +\sqrt{1/3}\, \left|1\right\rangle & \mathrm{with\ probability\ }1/2,\\ \sqrt{{2}/{3}}\,\left|0\right\rangle -\sqrt{1/3}\, \left|1\right\rangle & \mathrm{with\ probability\ }1/2. \end{array} $$

It turns out that these two probability distributions are indistinguishable. That is, any measurement you make on one will give exactly the same probability distribution of results that you make on the other. The reason for that is that $$ \frac{2}{3} |0\rangle\langle0| +\frac{1}{3}|1\rangle\langle 1| $$ and $$ \frac{1}{2}\left(\sqrt{2/3}\left|0\right\rangle +\sqrt{1/3}\, \left|1\right\rangle\right) \left(\sqrt{2/3}\left\langle 0\right| +\sqrt{1/3}\, \left\langle 1\right|\right) +\frac{1}{2}\left(\sqrt{{2}/{3}}\left|0\right\rangle -\sqrt{1/3}\, \left|1\right\rangle\right) \left(\sqrt{2/3}\left\langle 0\right| -\sqrt{1/3}\, \left\langle 1\right|\right) $$ are the same matrix.

That is, a probability distribution on quantum states is an overly specified distribution, and it is quite cumbersome to work with. We can predict any experimental outcome for a probability distribution on quantum states if we know the corresponding density operator, and many probability distributions yield the same density operator. If we have a probability density $\mu_v$ on quantum states $v$, we can predict any experimental outcome from the density operator $$ \int v v^* d \mu_v \,. $$

So for quantum probability theory, instead of working with probability distributions on quantum states, we work with density operators instead.

Classical states correspond to orthonormal vectors in Hilbert space, and classical probability distributions correspond to diagonal density operators.