Milstein began his concert career at age ten in Odessa, and soon after he played Glazunov
's concerto with the composer conducting. He continued to tour the Soviet Union for the next five years. During this time, Milstein made numerous joint appearances with Vladimir Horowitz
, and Horowitz's sister Regina also joined them as Milstein's accompanist. In 1925, Milstein and Horowitz were encouraged by government officials to make a concert trip outside of Russia; Milstein would never return. Milstein recalled in his memoirs that the dramatic "grand manner" of Horowitz immediately made the pianist a star, while Milstein, a much more reserved person, did not have such immediate success. In 1926, he went to Brussels to consult with and discuss matters of interpretation with the great violinist and teacher Ysaÿe.
He made his American debut with Leopold Stokowski and the Philadelphia Orchestra in 1929, and made his New York debut in 1930. He soon established his base there, eventually becoming a United States citizen in 1942. He may not have become a concert-hall idol like Horowitz, but he had a strong musical reputation and was always in demand. When Arturo Toscanini ended his tenure as music director of the New York Philharmonic in 1936, he asked for Milstein as soloist in his final concert. After World War II Milstein made his home primarily in London, teaching master classes around the world. He was widely regarded as a sympathetic and approachable teacher.
He also established a major recording career and remains best known for his landmark recordings of the complete solo works of J.S. Bach
, becoming a pioneer of the Bach solo violin literature at a time when few players programmed these pieces, and he eschewed more superficial works that were a primary part of the violin soloist's repertory. His 1950s recording of the Bach solo partitas and sonatas on the American Capitol Records label are exemplary traversals of that great cycle and are still counted as classics of recording art. Milstein maintained a remarkably long career, keeping the muscular strength and fluid joint motion he needed until his retirement at the age of 83, acquiesced to only after he broke his arm in a fall.