Sheldon was born Sidney Schechtel in Chicago, Illinois, to Otto Schechtel, a German Jewish father, and Natalie Marcus, a Russian Jewish mother. At 10, he made his first sale, $10 for a poem. During the Depression, he worked at a variety of jobs, attended Northwestern University and contributed short plays to drama groups.

In 1937 he moved to Hollywood, California, where he reviewed scripts and collaborated on a number of B movies.After serving in the military during World War II as a pilot in the War Training Service, a branch of the Army Air Corps, Sheldon returned to civilian life and moved to New York where he began writing musicals for the Broadway stage while continuing to write screenplays for both MGM Studios and Paramount Pictures. He earned a reputation as a prolific writer; for example, at one time he had three musicals on Broadway: a rewritten , , and , which earned him the Academy Award for Best Original Screenplay of 1947.