If what others say or criticize about us is true, why are we unhappy?
If what others say or criticize about us is not true, then we shouldn't be unhappy at all.
And so, either what others say about us is true or not true, there's no reason for us to be unhappy.
It is just a reaction coming from the egoism if the mind feels unhappy about something, it's all because it is something that the ego doesn't like, doesn't want, and doesn't agree with.
A well-trained mind free from egoism will not react the same.