It means that you are making an impact when people are attracted to what you do and talk about you. 

It doesn't matter how idiotic the fake friend talking behind you is. It doesn't matter how stupid your neighbor is to gossip with others about what you do. 

It doesn't matter how wrong or how much you fail on your journey. What's important is that you are always one step ahead of everyone else, even if they don't know it, even if you don't know yet, so get busy and move on. You're doing great.