The 28-year-old singer - who rose to fame on Simon's UK talent show 'The X Factor' in 2009 - was shocked when his grandma went up to the music mogul to give him one of her creations to wear when he goes ''running'', something Olly is not sure he even does.
He said: ''She calls them Nana's Knits and she actually sold them on eBay. She goes up to Simon and says, 'All right Simon, here's a woolly hat. When you are running around Los Angeles getting fit, you might want to keep your head warm.'
''I said, 'Nan, it's not cold in Los Angeles and does Simon even go running?' ''
Olly could afford to buy all of his grandmother's hats now, as the 'Please Don't Let Me Go' singer recently became a millionaire thanks to his pop career and TV presenting jobs.
A source told The Sun newspaper: ''Olly is a real grafter - he has worked his socks off since he took part in 'The X Factor'.
''He is constantly juggling working on the 'Xtra Factor', touring and recording new music, so every penny is hard earned.''