He was born Joseph Peter Breck, the son of a jazz musician, also named Joseph (nicknamed "Jobie"). Over time, his father worked with such legendary greats as Fats Waller, Bix Beiderbecke, Paul Whiteman and Billie Holiday. Nicknamed "Buddy" while young, Peter's parents were on the road for much of his early life and he was sent to live with his grandparents in Haverhill, Massachusetts, a move that provided more stability. His parents eventually divorced and young Peter returned to Rochester to live with his mother and her new husband, Al Weber, who was a sports editor of the Rochester - Times Union.　　Following his schooling at John Marshall High School in Rochester, Peter served in the United States Navy. He then turned his attention back to education and studied English and drama at the University of Houston in Houston. While performing in college plays, he started to apprentice at Houston's Alley Theatre, where he appeared in such productions as "Stalag 17", among others. He also had a talent for singing and performed in several clubs around the Houston area.