Often we try to shame or hate parts of ourselves out of existence. This approach is a dead end. The only way to heal, or to create lasting change, is through love. There is a modality of psychology called Internal Family Systems (IFS). IFS states that each of us is made up of multiple parts, and underlying them is a person's core or true Self. Like members of a family, a person's inner parts can take on extreme roles or subpersonalities. So when you say “a part of me feels guilty”, there actually is a part of you that feels guilty. While there might be another part of you that feels excited about the very same thing.
A core tenet of IFS is that every part has a positive intent, even if its actions are now counterproductive or cause dysfunction. Parts can do extreme behavior, that at one point was needed to protect you, however, might now be a way of behaving that no longer serves you.
There is no need to fight with, coerce, or eliminate parts; the IFS method promotes internal connection and harmony to bring the mind back into balance.
For more information on IFS, check out these:
Video: an explanation of IFS (this is the first in a 4-part series, by the creator of IFS). Self Therapy Book. Gives a great description of IFS.