Roger Hart was born at Ninan Bay. His mother became pregnant with him while working at the Hart homestead. She moved on to Barrow Point where she was married to a Gambiilmugu-warra man, but she was abducted by a Wuuri-warra man and Roger Hart was left behind. The Gambiilmugu-warra man became Hart's father and looked after him for a few years but the dangers of police raids for Indigenous children with European heritages became too great for the community. In 1932, Hart was taken to Cape Bedford mission to live. At the mission, Hart's name was changed a few times before Mrs Schwarz, the missionary's wife, decided to call him Roger.