Stern's family moved to the United States and settled in San Francisco when he was one year old. His mother, a professional singer, gave him his first music lessons. He began studying the violin at the San Francisco Conservatory in 1928. In 1932 he became the third immensely talented San Francisco-area boy to train with the San Francisco Symphony concertmaster Louis Persinger (the others were Menuhin and Ruggiero Ricci). However, he considered Naoum Blinder, with whom he studied until the age of 15, his only true teacher. Stern made his debut with the San Francisco Symphony on February 18, 1936, with Pierre Monteux conducting the Third Concerto by Saint-Saëns. After his New York debut in 1937, he returned to San Francisco for further study. He re-entered concert life on February 18, 1939, again giving a recital in New York. Soon he was one of the leading American violinists, particularly noticed for his young age, and his January 8, 1943, recital at Carnegie Hall (his first solo performance there) was a smash hit. In 1943 and 1944 Stern entertained American troops in Iceland, Greenland, and the South Pacific. After the war he toured Australia in 1947, and made his first trip to Europe in 1948. He played at Pablo Casals' Prades Festival from 1950-1952, and the Edinburgh Festival in 1953. His tour of the U.S.S.R. in 1956 was an early sign of one of the recurrent thaws in the Cold War. In 1960 he formed a durable trio with pianist Eugene Istomin and cellist Leonard Rose; the group played the complete trio literature by Beethoven in bicentennial celebrations of the composer's birth. He recorded mostly for Columbia (which subsequently became CBS, then Sony Classics), with major orchestras and conductors, with the Stern-Rose-Istomin Trio, and in sonata and other duet repertory with his regular partner, Alexander Zakin. He made several appearances at the White House.
In the late 1950s, when the City of New York planned the construction of the Lincoln Center complex, it became clear that the plans as they stood would entail the destruction of the old Metropolitan Opera house and of Carnegie Hall. The latter, one of the finest concert halls in the world acoustically, was saved for posterity by the actions of a group Stern formed in 1960. Stern was chosen president of the Carnegie Hall Corporation, formed to supervise the artistic program of the great concert hall. He also was involved in the formation of the U.S. National Endowment of the Arts, and was appointed to its initial advisory board. He served as chairman of the board of the American-Israel Cultural Foundation, which aids the careers of young musicians. Stern, among other honors, was named Officer of the Légion d'honneur of France.