And as said, that would also be valid for a gay male athlete or a gay male ref.

There are so many factors which could make a ref impartial. You could say a ref who was raised in the New England may by impartial against the Green Bay Packers and thus shouldn't call any games with them. Maybe there is a closet racist ref. Maybe a ref hates the colour purple and thus is impartial in games with the Vikings.

You've got way too much time on your hands if you have think about female refs banging the male athletes, or just maybe cuck fetish, who knows.