Born in England, Brabazon came to Australia as a boy in 1912. His family settled on a small wheat farm property near Glenrowan in north east Victoria. Brabazon started to write poetry in his youth. After the family had to sell the farm, he took a job as a rouse-a-bout for a shearing company near Broken Hill. Later he returned to Melbourne, and after various occupations finally decided to devote his life to art. Besides writing poetry, he studied music but felt most attracted to painting as his medium of expression. He attended classes at the Gallery Art School and was close to the Angry Penguins group.
Brabazon became interested in the writings of mystics, particulalry Ramakrishna. He joined a Sufi group, travelled to San Francisco and became a Sufi teacher himself. In 1959, he went to India and lived with Meher Baba, a leader of the Sufi Order, for ten years as a devotee-poet. In 1969 Brabazon returned to Australia and lived on a property near Nambour. He continued to write until his death in June 1984 from Alzheimer's disease.