geometry - Why do objects appear smaller when viewed from a distance?

Yes, I know all about perspective (I'm an artist). I even have some basic knowledge of descriptive geometry. I know how it works. My question is more about why it works.

I have a sneaking suspicion it has something to do with the curvature of space. However, I'm not certain whether the two concepts are related.

Another thing that's been bothering me for a while (this may or may not be related to the question: please enlighten me) is the relation of the vanishing point to the horizon:

Strictly speaking, the geometrical horizon is where the vanishing point is. The actual horizon is a bit lower, but the difference is negligible.

Thus, if you sketch a perfectly straight street lined with buildings of equal size stretching all the way to the horizon, you shouldn't even be able to see anything beyond 2.9 miles (the distance to the horizon): all parallel lines will merge - and vanish. And yet, if the buildings are large enough, they will be visible beyond that point.

Which would suggest that smaller objects become a point source before they reach the true vanishing point, and that the true vanishing point is a lot further than 2.9 miles (which is why we can see the actual shape of the moon, as well as some of the planets: they're not point sources; they're actual disks). I would even go so far as to posit that the reason we see stars as point sources is they aren't large enough; and the Andromeda Galaxy, which appears to the naked eye as a cloud, is not a point source; which leads me to believe that the absolute vanishing point is at an infinite distance from the observer.

With all of the above in mind, my question still is - WHY do objects appear smaller in the distance?

P.S. "That's how perspective works" is not a good answer in this case unless physics has nothing more to contribute at this juncture.

P.P.S. Yes, about those angles (the light hitting the eye, etc). Why should the angle get narrower as the distance increases? Well, for one thing, that's how our brain interprets the signal it receives from the eye. I daresay if it were the other way around (i.e. if the angle got broader as the distance increased), our brain would find a way to adjust, and we'd all be saying "It's bigger because it's further away, that's how it works").