Born in Hobart, Patrick Miller moved to Sydney in 1934 and was educated by the Sisters of Mercy, Neutral Bay, and the Marist Brothers, at both Mosman, Sydney, and Hamilton, Newcastle. Following his secondary education Miller worked in the Postmaster General's Department, transferring to Perth in the early 1950s.
Miller joined the R.A.F. and spent nine years as a Sergeant Signaller with Coastal Command. He later worked for the Society of St Vincent de Paul.