Rashi explains that the separate camp of the Levites served two purposes: to guard the Mishkan and to act as a boundary between it and the rest of the people. The gurading of this separation nullifies even the potential anger from Hashem.