You probably already know this, but worrying about the gossip will only hurt you. You can ask someone you trust if anything is ever said about you in your absence but don't worry about it man. I know it's easy for me to say that you should just chill, but I worry about what people are thinkning when they look at me. (Body dismorphic syndrome) I can relate, and you should just assume that when they bring you up it's positive.
You can hire me as a private detective. I charge 150$ an hour, but believe me... it's worth every cent. I can already tell you that the evil gossipers are saying bad stuff about you. That will be 150$, you will be billed accordingly.
I also give out mortgage loans. I can get you one at a wooping 99%. Most companies will probably offer at 5-20%, depends on how filthy poor you are... it's all about the refinancing. But.... I'll just take all of your money right away, without slowly taking it away... to save you from the pain. Did I mention that I'm also a bankruptcy agent and a divorce lawyer?
Try to ignore the gossip. If its really juicy it will bounce back to you. If you go out of your way looking for it, you will not feel any better.