What does it even take for a male character to be really disliked by fandom or even criticised to the extent female characters are all the time - especially if he's pretty?

The million dollar question. I don't know much about how it goes in fandoms other than SV, but it's enough to last me a lifetime.