Jim Leslie, the second son of Stuart Deacon Leslie and Dorothy Clare Leslie (nee Murphy), grew up in Melbourne and attended Trinity Grammar School. In 1941 Leslie joined the army and fought with the Australian armed forces in New Guinea against the Japanese.
At the end of World War II, Leslie embarked on a career with the Vacuum Oil Company, working in Fiji, America and New Zealand. He also worked on the board of Qantas and was Chancellor, Deakin University.