St Augustine's population swelled with loyalists and prisoners' including signers of the Declaration of Independence. By early 1781, St Augustine had one of the largest populations in British North America and was heavily fortified. British St Augustine was at its apex, but American victory at Yorktown would change everything.