Filming on the movie musical was halted for a week when the British funnyman fell sick with a sore throat and was unable to complete his part as Thenardier.
Cohen has now revealed he offered to mime his lines so production could move forward, but The King's Speech director insisted on all the music being sung live in front of the cameras.
Cohen tells Deadline.com, "After a month I actually lost my voice. And I said to Hooper, and this is actually a testament to him, 'Fine, I'll just mime it. I've got the track of me singing and I'll just mime it.' And he said 'Absolutely not. It has to be sung live.'
"And so they shut down the movie. They shut it down for a week during which time I was forbidden from speaking. I was on voice rest."
During his time off, Cohen worked with a vocal coach who also trains Coldplay frontman Chris Martin, but admits his voice still sounded "croaky" when filming resumed.
He adds, "They brought in Chris Martin's vocal teacher; who's the head of voice at the Royal Academy of Music to train me up again in three days... But when you see the first take of (song) Master Of The House it sounds like I'm a drunk guy who's got a croaky voice but that actually was me with a very croaky voice. Tom Hooper found himself with Helena Bonham Carter (who plays Thenardier's wife) and a soft-voiced, stumbling English actor; it was like The King's Speech all over again."