Shorty Harrison spent his childhood in Barham and Cummerangunja Mission. When he was a young boy, his Indigenous mother left their family to live with another man, Jack Little (the uncle of Jimmy Little (qv), later to be famous as a singer). Eventually, Harrison's father was unable to care for his children as a sole parent. Shorty was the only one out of seven children who remained with his father, the rest were placed with family and friends.
Harrison started work at a young age. His first job was as a camp cook while his father was making fences. He then worked as a digger for five years, creating irrigation systems. When he turned eighteen, he applied to join the army during World War II, but his application was denied because at that time he was working in a protected industry as a grape picker. He held licences for several heavy vehicles, therefore was able to use his skills bulldozing in the forestry industry, grading roads, clearing areas for different councils in Victoria and New South Wales. While he lived in Apollo Bay, Victoria, he worked as a fisherman, during the couta (King Mackerel) season. Harrison also was a caretaker for a school camping ground, and the Director of the Walgett Bowls Club.
(Source: 'Shorty' Bill Harrison, All Hearts Bleed the Same: Reflections, Recollections and Memories, 1994)