The R&B artist was picked to represent Germany in Stockholm, Sweden, but bosses at broadcaster ARD have opted to drop him and find another singer.
Naidoo has been accused of using anti-Semitic slurs and homophobic language in his songs.
In a press release, ARD executive Thomas Schreiber states, "Xavier Naidoo is a brilliant singer who is, according to my own opinion, neither racist nor homophobe. It was clear that his nomination would polarise opinions, but we were surprised about the negative response."
"The Eurovision Song Contest is a fun event, in which music and the understanding between European people should be the focus," Schreiber adds. "This characteristic must be kept at all costs. The ongoing discussion about Naidoo could harm the image of the Eurovision Song Contest."
In a statement on Naidoo's Facebook page, the singer has expressed how ARD bosses approached him to compete in the Eurovision Song Contest, adding he remains committed to tolerance and love.
Singer Mans Zelmerlow gave Sweden its sixth Eurovision Song Contest win in May (15), beating off Russia's entry from Polina Gagarina. As a result, next year's (16) event will be held in the Swedish capital.