Lipman began his working life as an office boy in Adelaide in the 1930s. After the Second World War, in which he served as a commando officer, he trained as a dentist before racing horses, starting a merchant bank, setting up the International College of Hotel Management and a travel agency, Angas Travel, and growing grapes in the Adelaide Hills. He has written about his life in his memoir, Luck's Been a Lady ([Adelaide : Rex J. Lipman], 2000).
