Patsy Cohen had an Indigenous mother and non-Indigenous father. When her father left to serve in World War Two, her mother was charged with neglecting her children. When Cohen was just an infant she and her siblings were made wards of the state and stolen from their mother. Cohen was shifted to different Institutions and families for several years. At the age of nine she was taken to Ingelba and placed in the care of her grandmother and family for a few years. Due to the grandmother's ill health, Cohen was stolen from her family again and placed in an institution for older female wards.
She became pregnant at an early age, but was determined to keep her child even though her own father suggested she give her baby up. After she left the institution for older female wards she returned to Armidale where she married her first husband, then later married Jack Cohen.