The 'Dear Darlin' singer - who is supporting the former Take That star on his current stadium tour of Europe - is keen to start a family of his own after seeing how happy the first-time-father is with his 10-month-old daughter Theodora 'Teddy' Rose.
He told Britain's OK! magazine: ''Little Teddy is such a little bundle of joy. To see her little face around the place is brilliant and Robbie is so happy. He has a big smile on his face every day.
''I'm definitely broody but then I'm always broody. I'm probably the most sensitive man. I want to have lots of children, eventually.''
Olly, 29, has never had so much fun on tour and revealed Robbie is extremely supportive of him and great to hang out with backstage.
The cheeky singer said: ''It's been a brilliant tour so far and Robbie and me have been having an amazing time. I've actually been a support act a few times now but I've never had an experience like this.
''I've never had so much love from another artist before and it's so nice. Robbie's been a true gent on this tour and I have nothing but utter respect for him. It's great to be able to spend time with someone like him because he's such a laugh.''
However, the boys have yet to hit the town together since Robbie is too busy looking after Teddy and spending time with his wife Ayda Field, who are also accompanying him on his Take The Crown tour.
Olly added: ''Unfortunately, we haven't really been able to go out together yet because everything's been so hectic. Robbie's had his wife and his little daughter on tour, so us going out would have been a bit tricky.
''I think that might all change when we get to [notoriously hedonistic capital of the Netherlands] Amsterdam, though! I think that night will be the one!''
Perry appeared on James Corden's 'The Late Late Show' on Monday and addressed her long-standing rivalry with Swift.