In 1963, she joined the Deutsche Oper, Berlin. Her Covent Garden debut was in 1970, as Susanna. She went on to sing in the major opera houses of the world, including the Bavarian State Opera in Munich, the Vienna State Opera, the Metropolitan Opera in New York, and the Opéra de Paris. She sang regularly at the Salzburg and Munich Festivals.
Among her other roles were Zerlina in Don Giovanni, Marzelline in Fidelio, and Mélisande in Debussy's Pélleas et Mélisande, and Pamina in the Magic Flute. She was considered outstanding in the role of Sophie in Strauss' Der Rosenkavalier. As the list of roles indicates, she had a bright voice suggesting youth and freshness, and she was regarded an excellent actress. Other roles with which she was particularly associated include the Marschallin in Der Rosenkavalier, the Countess in Le Nozze di Figaro, and Agathe in Der Freischütz. Mathis is also a highly regarded oratorio and recital singer, known for direct, simple expression of the classic lieder repertory, and is also highly regarded as a Bach singer, as well as singing the oratorios of Joseph Haydn and numerous works of Mozart and Handel.
She has made many recordings and received numerous recording prizes, including the Prix Mondial du Disque. Other honors include the Hans-Reinhard-Ring from the Swiss Society for Theater, the Arts Prize of the city of Lucerne, the Buxtehude Prize from the Senate of Lübeck, and the Mozart Medal of the Mozarteum in Salzburg. In 1979 the Bavarian government declared her a Bayerische Kammersangerin.