Caleb initially obeyed the Lord and drove out the sons of Anak, who had previously intimidated the sons of Israel (Num. 13:30-33).  He did by faith what all Israel should have done.  Israel—for the most part—operated from fear rather than faith, and this led to policies of appeasement with those who should have been driven out.  Their decisions of toleration sowed the seeds of their future destruction because they allowed themselves to be exposed to the religious and ethical values of people who were opposed to the Lord.