Victor Hugo made this story up after he saw an ugly guy working on the restoration of a cathedral.  That is actually cited as being his inspiration for the character.  He wrote a 940 page book because of how ugly he thought a construction worker was. 

The book was written over 3 years and published in 1831. This means there is a chance the construction worker who inspired the book was still alive when it came out, and may have read it without knowing the whole thing was secretly based on how ugly some other guy thought he was.