Billy insists they shared something special when they were together and both supported each other's musical aspirations.
In a HuffPost Live chat to promote his band's new album 'Monuments to an Elegy', he said: ''We connected very deeply and once it spun off and where we spun off - we've never had that same relationship again ... We were two artists, probably before their insanity hit, finding some sort of Zeitgeist between one another in terms of artwork and what we wanted to accomplish.''
Billy - who married artist Chris Fabian in 1993 but split from her in 1995 - is believed to have rekindled his romance with Courtney again in 1994 after Courtney was widowed following Kurt's death.
In 2006, he moved in with the Hole frontwoman and her daughter Frances Bean Cobain and had his own wing in her Hollywood Hills mansion, but they he angered her in 2008 by not attending Frances' 16th birthday party in 2008 and they fell out publicly in 2010 over a row about songs she included on an album without his permission.
The 47-year-old musician accepts Courtney has her flaws but he has always loved her for who she is and still cares deeply for her now.
Speaking to host Caroline Modarressy-Tehrani, he explained: ''She is who she is I love and accept her. It doesn't mean I like everything she says and does, nor do I want to spend a year with her in an elevator.''