Agnes was the eldest of seven children of Dr William Gosse and his Scottish wife Agnes (Grant). She was educated at Rawdon House School, Hoddesdon, by Mrs Sarah Ellis ("anti-feminist" author of Women of England, and wife of the Rev'd William Ellis, missionary to Madagascar) before coming to SA on the Elizabeth, via Pt Melbourne, on 31 Dec, 1850. They lived principally in a house on North Tce where the Shell Building now stands. Her father practised medicine in Adelaide.
Agnes was educated privately, probably by the Rev'd James McGowan. She travelled frequently between Australia and England before and after her marriage. She met the colonist, philanthropist and politician, Alexander Hay, in 1867, and became his second wife in 1872 when she was 34. His first wife, Agnes (Kelly) had died of Bright's Disease in 1870, leaving four children. Owners of "Linden", at Burnside, and "Mount Breckan" at Victor Harbor, the Hays were lavish entertainers, and Agnes also officiated at a number of openings of buildings, and layings of foundation stones. On these occasions she frequently spoke against feminism, and promoted Bible Readings in State schools.
Alexander Hay died suddenly on 4 Feb 1898, and over the next eleven years Agnes and her two daughters engaged in travel to the UK and to France, Germany and Norway. Encouraged by her second son, William Gosse Hay (qv), the novelist, she began writing travel articles for the Advertiser and the Chronicle, and for UK newspapers, and she wrote a biography of her husband (Footprints) in 1899. Her eldest son died in 1901 of Bright's Disease. In 1902 she published Ober-Ammergau: and its Great Passion Drama of 1900. Her book of memoirs and family history (After Glow Memories, published in 1905) is written as though fictional.
She and her youngest daughter, Helen, perished in Agnes' 18th sea voyage, 1909, when the SS Waratah disappeared mysteriously with 200 passengers aboard between Durban and Cape Town. Also lost was the manuscript for a third novel, apparently set in SA.