Rex Ellis grew up in McLaren Vale, South Australia. From the age of seventeen he worked as a jackaroo, overseer and fencing contractor on stations in South and Western Australia. In 1965 he began working for an Australian safari company, before setting up Transcontinental Safaris and in 1976, Ellis began running camel expeditions as the Outback Camel Company.