Looking for what he described as "a psychological release valve," he joined the California-based comedy group The Groundlings in 1975 . Hartman met comedian Paul Reubens while working with the group and the two became friends, often writing and working on material together. One such collaboration was the character of Pee-wee Herman and the script of the feature film Pee-wee's Big Adventure. Also, Hartman worked with Paul Reubens in "Cheech and Chong's Next Movie" in a small role. Hartman also met Jon Lovitz while with The Groundlings. Hartman was fluent in German and a video circulated once of him doing a standup routine in the language.

Hartman worked part time as a graphic artist, including designing album covers for popular rock bands. Hartman's covers include:

Poco's 1978 album Legend Firesign Theatre's 1980 album Fighting Clowns Three album covers for the band America Crosby, Stills & Nash's logo. Contrary to popular belief Phil Hartman did not create the cover for Steely Dan's 1977 album Aja