The rock legend, 70, married Amanda Sheppard, who is 37 years his junior, in 2012 after a three-year romance, but the union crumbled and they divorced in 2014 after just two years as husband and wife.
Ferry has now revealed the pair's relationship ended because Amanda wanted to have a baby, and he felt he was too old to become a parent for a fifth time.
"It was a great combination and it's sad we had to go our different ways. We wanted different things, that's really the phrase," the singer tells Britain's Event magazine.
"She wanted a baby and I didn't. First of all I thought I could and then later I thought, it's not the right thing to do. It was a terrible, stressful time for both of us.
"I just thought it could have ended up really bad with me being too old to work. We so enjoyed each other's company apart from the stupid things that people would write in the papers about the age difference," he added.
Since his divorce from Amanda, the Love Is The Drug rocker has not rushed into a new romance and admits he has been single since the split.
"I have not seen anyone since Amanda so it is now quite a long time," the musician said.
Ferry has four sons from a previous marriage.