The rule "don't wear white after Labor Day" is a tradition rooted in the late 19th and early 20th centuries, originating from wealthy New Yorkers who would pack away their summer white clothes for fall in the city's dirt-filled streets after returning from their summer vacations. 

You might get a visit from the fashion police-