Growing up in Melbourne, Australia, Dalle fell in love with punk rock and grunge as a teenager, and began playing guitar when she was 13. She formed the band Sourpuss while still in her teens, and eventually moved to Los Angeles after meeting and falling in love with Rancid's Tim Armstrong; the pair married in 1997. The following year, she formed the Distillers, whose raw punk drew comparisons to Courtney Love and PJ Harvey on their 2000 self-titled debut album. Despite numerous lineup changes, the band released two more albums. Sing Sing Death House was originally released in 2000 and reissued two years later thanks to the surprise hit single "Seneca Falls." The band's major-label debut, Coral Fang, appeared in 2003 and found Dalle using her maiden name again after her divorce from Armstrong.