People who have no religious affiliation are the fastest growing "religious" group in America. While the headlines often talk about the many people who are leaving religion, they seldom mention how difficult this can be for people.