Gedda's father was Russian, his mother Swedish, and in 1929 his parents moved to Leipzig, where his father became cantor and choirmaster of a Russian Orthodox Church. As a child he sang in the church and performed Russian songs at weddings and parties. Having no wish to remain in Germany as the Nazi party rose to power, the family returned to Sweden in 1934.
As a teenager, he developed a strong ambition to become an opera singer, but unable to come up with the money to fund his studies, he became a bank teller, though he continued to participate in singing competitions. A wealthy bank client who was also an instrumentalist with the Stockholm Opera happened to hear Gedda mention his aspirations, and asked Karl-Martin Oehmann, a music teacher who had himself been an operatic tenor, to take him on as a pupil. Gedda later credited the development of his spectacular technique to Oehmann's teaching. Shortly thereafter, he won the Christine Nielsson Scholarship, which enabled him to begin music studies at the Stockholm Conservatory as a full time pupil in 1950.
In 1952, Gedda's recording and performing careers began simultaneously. Walter Legge, the famous producer for EMI, was in Sweden, and agreed to audition some of the students. He was immediately bowled over by Gedda's voice, musicianship, and technique; given that Gedda spoke fluent Russian, he signed him on the spot to take on the role of Dmitri in his upcoming recording of Boris Godunov, starring Boris Christoff -- all despite the fact that Gedda had yet to make his stage debut (which came only a short time later, in a production of Adolphe Adam's Le Postillon de Lonjumeau)! This began a lifelong association with EMI. He made his La Scala debut in the 1952-1953 season as Don Ottavio (Mozart's Don Giovanni), and Carl Orff asked him to create the role of the Bridegroom in Il Trionfo dell'Afrodite; his Paris Opera debut in 1954 was as Huon in Weber's Oberon, and his Covent Garden debut in 1954 was as the Duke of Mantua. These were soon followed by debuts at Salzburg (Don Ottavio, 1957) and the Metropolitan Opera (Faust).
His life-long association with EMI made him one of the most widely-recorded tenors, though in the compact disc era, EMI had been rather dilatory in re-releasing many of his recordings.