The fossil record shows that humans are increasing in size, not decreasing. We can even dig up corpses from 500 years ago to compare.

The size of an organism likely has more to do with the survivability in their environment over time than anything else. I have read Darwin's work and it more or less still holds up today.