The Les Miserables beauty called off her romance with the Welsh presenter earlier this year (13) and was subsequently linked to Dolce & Gabbana hunk Gandy.
Barks' brief relationship with Gandy reportedly finished last month (Jul13), but Johnson admits it was tough for him to see the actress in the arms of another man.
He tells Britain's The Sun, "Sam and I cared for each other and the decision to break up was hard. When you genuinely have love for somebody it is hard to walk away. But it hurt when she upgraded to David Gandy. F**k Gandy!
"They were photographed together about three weeks after we split up, which was difficult to take. That's why it's a closed door between the two of us.
"My first reaction was to go: 'Get off her'. Then I was just thinking: 'Not Gandy'. I've seen his adverts - the man's got abs of steel. Anybody who is honest would think: 'He's a handsome man, I get it.' But it still burns, it really does."
Johnson goes on to reveal the reasons behind his split from Barks, adding, "Sam is amazing - she is so talented, gorgeous and sweet. But she's at a point of her career when she's just got to go for it and strike while the iron is hot. And it gradually got a bit more stressful because we never saw each other. We finally made the call to end it while drunk in a restaurant at 3am. It was really sad."