quantum mechanics - Derivation of position operator in QM

For the definition of the momentum operator $$\hat{P } = -i \hbar \nabla$$ in quantum mechanics, as I understand you can derive this by either considering a more general definition of momentum, i.e. 'canonical momentum' which is an operator and then apply this operator to wave functions. This is shown here in this wiki entry. We can alternatlively start with translations and use the face that the momentum operator is a generator of translations as is done here in this wiki entry.

What I am interested in is what is the more fundamental derivation for the position operator: $$\hat{X} = x.$$ To this point I have considered that the motivation for defining position operator is from the definition of the expectation value $$\langle x \rangle = \int dx x |\psi(x)|^2 = \langle \psi|x | \psi \rangle$$ where $| \psi \rangle$ is normalised. Is this the full extent of the motivation or the main point to consider? Is there another motivation for the position operator or is it taken from this motivation and confirmed by experiments to be acceptable?

Answer

${\hat X} = x$ because we choose to work in a basis of eigenvectors of ${\hat X}$, i.e. wave-functions.

We have a Hilbert space ${\cal H}$ which is a vector space on which we can choose any basis we wish. We most often choose to work in a basis that diagonalizes the position operator ${\hat X}$. Basis states satisfy $$ {\hat X} | x \rangle = x | x \rangle $$ Once such a basis is chosen, any state $|\Psi\rangle$ in ${\cal H}$ can be expanded in it, i.e. we can write $$ | \Psi \rangle = \int dx | x \rangle \Psi(x) $$ The function $\Psi(x)$ is called the wave-function. We can invert this to find $$ \Psi(x) = \langle x | \Psi \rangle $$

Now, what is the meaning of $\big({\hat X} \Psi \big)(x)$? By definition it is the wave-function of the state ${\hat X} | \Psi \rangle$, i.e. it is $\langle x | {\hat X} | \Psi \rangle$. It is then immediately obvious that $$ \langle x | {\hat X} | \Psi \rangle = x \langle x | \Psi \rangle = x \Psi(x) $$ Thus, we find that $\big({\hat X} \Psi \big)(x) = x \Psi(x)$. For this reason, we write for convenience ${\hat X} = x$, but one must remember that this is only true in the coordinate basis.

Momentum operator in coordinate basis: You might then ask how one can derive the expression for the momentum operator ${\hat P}$ in the coordinate basis. This is done as follows.

By definition $\big( {\hat P} \Psi \big)(x) = \langle x | {\hat P} | \Psi \rangle $. Then, $$ \big( {\hat P} \Psi \big)(x) = \int \frac{dp}{2\pi \hbar} \langle x | p \rangle \langle p | {\hat P} | \Psi \rangle = \int \frac{dp}{2\pi \hbar} \langle x | p \rangle \langle p | \Psi \rangle p = \int dx' \frac{dp}{2\pi \hbar} \langle x | p \rangle \langle p | x' \rangle \langle x' |\Psi \rangle p $$ Next, using $\langle x | p \rangle = e^{\frac{i}{\hbar} p x }$, we have $$ \big( {\hat P} \Psi \big)(x) = \int dx' \frac{dp}{2\pi \hbar}e^{\frac{i}{\hbar} p (x-x') } p \Psi(x') $$ We now write $$ e^{\frac{i}{\hbar} p (x-x') } p = - i \hbar \frac{\partial}{\partial x}e^{\frac{i}{\hbar} p (x-x') } $$ Using this, we can explicitly perform the integral over $p$ and we find $$ \big( {\hat P} \Psi \big)(x) = - i \hbar \frac{\partial}{\partial x} \Psi(x) $$ Thus, in coordinate basis, we write $$ {\hat P} = - i \hbar \frac{\partial}{\partial x} $$

PS - Of course, we are free to choose any basis we want. We could for instance work in momentum basis. In this basis, we would have $$ {\hat P} = p~, \qquad {\hat X} = i \hbar \frac{\partial}{\partial p} ~. $$