Podcasting can be polarising. This is great! Attract the people who resonate and repel those who don't. The former will love you and spread the word, the latter are better gone.