Prefix of CON-fidence (adj.) "swindling."

Both what you like and what you dislike distort your perception if you like somebody, you'll exaggerate them in your mind. If you dislike somebody, for sure you'll exaggerate them in your mind. Exaggeration means you're unable to see things as they are. Confidence tends to leave many within this space.