Stevens was born Harold Ray Ragsdale on January 24, 1939, in Clarkdale, Georgia. While attending high school, Stevens formed his first band, a rhythm and blues group he named the Barons. Following his graduation from high school, Stevens enrolled in Georgia State College as a music major.