cosmology - Distance of universe's most distant objects in relation to expansion of the universe

Was reading this article about the Hubble XDF and it had the quote:

The most distant objects here are over 13 billion light years away, and we see them when they were only 500 million years old.

But if these galaxies were closer to us 500 million years after the big bang (and wouldn't they have to be -- since we've been expanding ever since?) then why did it take so long for the light from where they were 500 million years after the big bang to reach us? Shouldn't that light have hit earth a lot sooner since it was a lot closer back then? What principle am I missing here?