The 26-year-old reality TV star doesn't want to watch Kim, 33, and her fiancé Kanye West, 36, tie the knot in Paris, France, in May because he's unhappy with his appearance.
A source told RadarOnline: ''Rob is upset about his changing shape and truly feels like the black sheep of the family.''
The source added: ''Kim is his sister. He loves her. But he knows this will be all about Kim and as usual, Rob feels he would be like the third wheel.
''The reality of their situation is that Kim and Rob haven't been that close in years. Deep down, she thinks he is a total slacker and it's very hurtful to Rob.''
Meanwhile, Rob recently sought medical help on his weight and he was horrified after learning that he was at risk of developing diabetes and eventual organ failure.
However, the sock designer - whose father Robert Kardashian died aged just 59 from oesophageal cancer in 2003 - has agreed he needs to make changes to his lifestyle.
OK Go add another fantastic video to their repertoire with the release of The One Moment