He was born in Palo Alto, California, and moved at the age of six with his family to Massachusetts.
While he is sometimes associated with the Asian-American jazz or avant-garde jazz movements, Ho himself was opposed to the use of term "jazz" to describe traditional African-American music because the word "jazz" was used pejoratively by white Americans to denigrate the music of African Americans. In his role as an activist, many of his works fuse the melodies of indigenous and traditional Asian and African musics, which as Ho would have said is the music of the majority of the world's people.