During World War I he joined the British Army; while in the service he got his first taste of conducting by leading an all-volunteer orchestra. After his service ended in 1919, he returned to the Queen's Hall Orchestra. He also resumed performances as a cello soloist, appearing with the Bournemouth Municipal Orchestra. He joined and toured with the International String Quartet beginning in 1923. Soon after, he organized and conducted a chamber orchestra in Chelsea.
The British National Opera Company engaged him to conduct on tour. On December 12, 1927, he attracted attention by successfully substituting as conductor of the London Symphony Orchestra when Thomas Beecham was indisposed; during the 1928 season he began to conducting opera regularly in London at both Sadler's Wells and Covent Garden. In 1933 he was appointed permanent conductor of the Scottish Orchestra, remaining in that position for three seasons. At the same time he was the conductor of the Leeds Symphony Orchestra, and guest-conducted several orchestras at home and abroad. During the 1936-1937 season he accepted a ten-week position as guest conductor of the New York Philharmonic. This led to an invitation to assume the principal conductorship of that orchestra, in which capacity he succeeded Toscanini. Barbirolli remained with the Philharmonic through 1942. During his tenure in New York, he gave the world premieres of the Violin Concerto and the Sinfonia da Requiem of Benjamin Britten, then resident in New York. While in the United States, Barbirolli also appeared with various American and Canadian orchestras. Barbirolli was never a favorite of the New York critics during his time with the Philharmonic, and many audience members wanted something more flamboyant than his objective approach. And his relationship with the members of the orchestra wasn't always harmonious; but he had the support of Arthur Judson, the orchestra's manager, who especially appreciated the fact that the conductor tended to work well with guest soloists, whose performances were a major source of prestige for the orchestra. He was reportedly offered a new contract at the end of his fifth year, but to accept would have required him to take American citizenship, something he was unwilling to do with England locked in the midst of a battle for survival. He returned to England in 1943 and began his long tenure as conductor of the Hallé Orchestra in Manchester, which he ultimately transformed into a world-class ensemble rivaling the London Symphony Orchestra and the London Philharmonic, as well as such postwar ensembles as the Royal Philharmonic and the Philharmonia; in 1958 he reduced the number of yearly concerts he performed with them when he took the title Conductor-in-Chief, and in 1968 he retired from the Hallé with the title Conductor Laureate for Life. His repertoire centered on the late Romantic era, and on British composers Elgar, Vaughan Williams, and Delius; he led the first performances of Vaughan Williams' Symphony No. 7 and Symphony No. 8; the composer also bestowed the nickname "Glorious John" upon the conductor. Aside from the music of Britten, he showed little interest in music of modern tendencies; late in his career, though, he developed a particular affinity for Gustav Mahler. Barbirolli left a notable recorded legacy that extends well into the stereo LP era.