Ben Phelps was born in Seoul, South Korea. He was adopted by his parents and grew up in the Midwest in Golden Valley, Minnesota. He attended Robbinsdale Armstrong High School, and was a varsity swimmer during his time in high school.