The supermodel, 38, has reportedly banned him from accompanying them on their yacht holiday to Mallorca and St Tropez.
A close friend told the Daily Star newspaper: ''Sadie has been seeing James Gooding for the past year. He's famous for doing a kiss and tell on ex Kylie Minogue and cheating on her with Sophie Dahl.''
He spilled the beans after his relationship with Kylie broke down, confessing to cheating on her.
Apparently this past behavior has deemed him not good enough for Sadie, 48.
The source added: ''Kate didn't want him hanging around so she vetoed him from having yacht fun.''
Sadie will now join Kate and Fran Cutler in the sun without 27-year-old James.
Unlike Kate, Sadie's family appears to have embraced him.
The two spent the last few weeks in a villa in France with Sadie's mother and three sisters.
She also has three - Rafferty, 15, Iris, 11, and Rudy, nine, with second spouse Jude Law.