The 'Blue Monday' band went on hiatus in the early 90s before reconvening in 1998 and releasing a further two albums, but Peter now says he should have stayed with his side project, Monaco.
He told live4ever.uk.com: "I left Monaco to go back to New Order, but in a lot of ways when I look back I think I would have been better off staying in Monaco because New Order was such a bitter ending.
"Maybe 1990 would have been a better year to leave New Order and end it, but you live and learn."
New Order - who formed in 1980 from the remaining members of JOY DIVISION after the death of singer Despite Ian Curtis - have never formally split up, although Peter left the band in 2006, and despite saying he would have preferred the group to have finished earlier, he liked the records they made in the last decade.
He added: "I'm very proud of `Get Ready', `Waiting for the Sirens Call' was OK, but I loved `Get Ready' and thought it was a great comeback."
Peter also cited the fact the bands 2005 album 'Waiting for the Sirens Call' flopped was a contributing factor to his decision to leave, blaming the fact it was one of the first albums to be widely illegally distributed online.
He added: "We were in the studio for three years putting that album together which cost £700,000 to record and we got nothing back from it. So if you were to ask me if I'd want to do it again you can guess The Answer."
Peter is currently touring Joy Division's 'Closer' album.