The rock matriarch has made no secret of her dislike of the Australian pop star, who joined the panel of the U.K. talent show in 2007, three years after it debuted with Osbourne, Cowell and Louis Walsh on the panel.
Osbourne eventually quit the show the same year due her bad relationship with Minogue, and now she reveals the singer's fling with Cowell made her unbearable to be around on set.
In an excerpt from her new book, Unbreakable, which appears in Britain's The Sun, Osbourne writes, "When she joined X Factor as a judge in 2007, Simon fancied the pants off her. I get that. He's single, he's the boss and he can do what he wants. But during filming it was obvious to me that there was some sort of relationship going on between them. And the more it progressed, the worse it became between her and me. It was unbelievably bad.
"The closer she got to Simon, the more I felt she started telling the producers what to do. Suddenly, from being the new girl, she was saying, 'This isn't right'...
"Meanwhile the antics behind the screen had gone up a gear, with her sticking her tongue in Simon's ear and giggling like a blo**dy teenager as Louis Walsh and me stood there like a couple of gooseberries.
"She clearly felt s**gging Simon gave her vicarious power and it was so immature, not to mention deeply irritating to deal with."
Cowell and Minogue's relationship first came to light last year (12) in biography Sweet Revenge: The Intimate Life of Simon Cowell.