The reality is that disease was the primary cause of death, and it was bound to occur due to the fact that the modern understanding of disease and infection was still centuries in the future. You can blame conquistadors for those killed in conflict, but that number is a tiny fraction compared to those who died of disease.