Columba was born in Queen Street, Dublin, Ireland on April 1, 1858, into a large and very religious family; three of his sisters became nuns. His father, William Marmion was from Clane, Co. Kildare. His mother, Herminie Cordier was French, prompting his biographer, Dom Raymond Thibaut to remark: "He owes to his Celtic origin his penetrating intelligence, his lively imagination, his sensibility, his exuberance and his youthful spirit. The French blood which ran in his veins contributes to his clearness of mind, his habit of clear perception, his ease of exposition, and his uprightness of character. From the combination of the two he derives his constant gaiety and his generosity of heart with all the strength, devotion, and fine feeling which this noble quality implies." He was baptized with the name "Joseph Aloysius". From a very early age he was seemingly "consumed with some kind of inner fire or enthusiasm for the things of God." He was educated at the Jesuit Belvedere College in Dublin.

He entered the seminary at the age of 16. At the time he entered the seminary, his "faith was very strong";[ he perceived "something more than simple theoretical theses" in Catholic doctrine, in particular "that a man's love for God is measured by his love for his neighbor."