In 1861, Sullivan began to make an impact on the London music scene. His music to Shakespeare's The Tempest was performed at the Crystal Palace winter concerts in 1862; the following year he published six Shakespeare songs. In 1864, Sullivan became the organist at Covent Garden, where his ballet L'île enchantée had its premiere. In 1867, he collaborated with F.C. Burnand on two operettas, Cox and Box and The Contrabandista. Five years later, he worked for the first time with W.S. Gilbert when they jointly created the light opera Thespis for the Gaiety Theatre. The piece was moderately successful, but not enough for its creators to continue working together immediately. In 1875, however, the theater manager Richard D'Oyly Carte reunited the to compose an afterpiece for a production of Offenbach's La Périchole. The result was Trial By Jury, a brief satire of the judicial process, featuring Sullivan's brother, Fred, in the lead. The afterpiece proved so successful that this time the partnership continued. In 1877, their full-length opera The Sorcerer premiered under Carte's auspices at the Opera Comique, followed by H.M.S. Pinafore in 1878. Pinafore proved wildly popular, solidifying a three-way partnership of Sullivan, Gilbert, and Carte that resulted in eight more operettas before a quarrel broke up the team. After a reconciliation, Gilbert and Sullivan wrote two more operas, Utopia Ltd. (1893) and The Grand Duke (1896), but these failed to catch on and the partnership ended permanently. During the run of Patience (1881), Carte moved his company from the Opera Comique to the Savoy; from then on, Gilbert and Sullivan's operas became collectively known as Savoy Operas. In these operas, Sullivan's unerring sense of musical parody, with targets ranging from Handel to Verdi, perfectly matched Gilbert's witty social satire.
While Sullivan is known today for his partnership with Gilbert, he was active in other compositional forms throughout his career. One of his best known compositions in his day was The Lost Chord (1877), a song to a text by Adelaide Proctor written at the time of Fred Sullivan's untimely death. Sullivan was acclaimed for his oratorios, including The Prodigal Son (1869), and hymns, of which "Onward, Christian Soldiers" (1871) was and remains particularly popular. An "Irish" Symphony in E minor is sometimes still performed. Sullivan also composed operettas with librettists other than Gilbert; these include Haddon Hall (1892) with Sydney Grundy and The Rose of Persia (1900) with Basil Hood. In 1883, Queen Victoria rewarded Sullivan for his contributions to English music with a knighthood.
Sullivan's dearest ambition was to compose grand opera. Carte, wishing to establish an English opera tradition, built the English Opera House, and here Sullivan's opera Ivanhoe, to a libretto by Julian Sturgis, premiered in 1891. The opera failed and remained Sullivan's only attempt at grand opera.
Sullivan never married, but in the 1860s or 1870s he met Mary Frances (Fanny) Ronalds, an American-born woman separated from her husband who eventually settled in London. Mrs. Ronalds, who mixed in the same aristocratic and artistic circles as Sullivan, never divorced her husband but carried on a discreet relationship with Sullivan until his death in London on November 22, 1900.