People are going to keep talking sh** and there's nothing you can do about it. It's healthier to not care about what others think and worry about yourself. Easier said than done... but still.
I used to, but then it felt like my entire city was conspiring against me and everywhere I went people were talking trash. I'm slowly rebuilding my confidence tho. I think it's more jealousy then anything, people hate on good looking people if they aren't so **** 'em.