The 23-year-old musician briefly dated the '22' hitmaker in February last year and admits he found the press attention too much to bare.
Tom told Britain's GQ magazine: ''It was terrifying. I'd have to be very in love to be with someone that famous.''
The 'Another Love' singer claims he never had the opportunity to dress in smart outfits when he was growing up, but his love of fashion has grown since he performed live at the Burberry Prorsum Autumn/Winter 2013 show.
Tom explained:''I am interested in that classic British look - yet also that classic American look. Burberry did that for me, because when I played at their party last year it introduced me to fashion.
''I grew up in the suburbs of a small town on the south coast where the only opportunity I ever got to wear anything smart was a funeral, so I had never owned a piece of clothing worth more than £40.''
Tom now relishes the opportunity to buy expensive clothes, and revealed he even spends hundreds of pounds on multivitamins to keep him looking and feeling his best whilst on tour.