photons - Photoelectric effect absorption coefficient decreases with energy, why?

Consider the diagram below: (Author: Joshua Hykes source: Wikipedia)

From this diagram we can see that the absorption coefficient for the photoelectric effect generically decreases with the increase in energy of the photon. What is the physical reason for this decrease. (i.e. why does the probability of photoelectric absorption decreases with increasing photon energy).

Note: I am aware that the large spin is due to more inner electrons been able to be lifted.

Edit For Clarity

I will here rephrase the question: in the above diagram there is a general decrease in the absorption coefficient due to the photoelectric effect (i.e. the absorption of a photon by an electron and subsequent emission of that electron) with energy (ignoring the occasional absorption edge). This is of course linked to a decrease in the probability of the photoelectric effect occurring (and a corresponding decrease in the cross section). My question is: what is causing this decrease with increasing energy. (For the purpose of this question I am ignoring the contribution due to the Compton effect and pair production and simply focusing on the cross section due to the photoelectric effect.)