Ermler studied at the Leningrad Conservatory, with Boris Khaikin as his conducting instructor, and attended rehearsals and performances of the Leningrad Philharmonic led by Evgeny Mravinsky as often as possible, learning much through observation of Mravinsky. He made his own debut at the podium with the Leningrad Philharmonic in 1952. A year later, he conducted his first operatic performance, of Mozart's The Abduction from the Seraglio, in Leningrad. He joined the staff of the Bolshoi Opera in 1956, and conducted performances of Beethoven's Fidelio, Rossini's The Barber of Seville, Tchaikovsky's Eugene Onegin, and Mozart's The Marriage of Figaro, among other works, during his first season. In 1964 he made his debut as a ballet conductor, with Stravinsky's Petrushka and The Firebird, and he made his British debut with the Bolshoi company in 1974. Ermler's international career included performances in Milan, Prague, London, Montreal, Tokyo, Berlin, with the Bolshoi orchestra and also as the guest conductor of the Royal Philharmonic, the Royal Liverpool Philharmonic, and the London Symphony Orchestra. He also conducted at such distinguished opera venues as Covent Garden, the Seattle Opera, and the Edinburgh Festival. His last appointment was with the Seoul Philharmonic Orchestra, in 2001 - 2002. Although he started his career in the works of Mozart and Beethoven, Ermler proved strongest in his approach to the music of his own country, most notably Tchaikovsky, Borodin, and Rimsky-Korsakov. His approach to conducting yielded very finely played and sung performances, which are also uncommonly rousing and vigorous, and rich in color as well as details.
Ermler's recording career was largely centered on the Melodiya label. This has resulted in the sporadic availability of much of his work, as Melodiya's various licensing agreements caused recordings to appear and disappear very suddenly. His operatic recordings, especially of the Russian Romantic repertory, are all highly praised, and considered among the best performances available.