When Joseph was 22 his father heard of the death of Peter Perry, a visionary in Port Perry who had great ambitions. His death left a large gap, which needed to be filled. Hiram immediately saw this as a great opportunity for his sons. He purchased a parcel of land, known as lot 5 (now the Tweed and Hickory Building) for his sons to make their fortune.