The telly mogul had signed up to be a judge on the new series of X Factor Israel which is due to start filming over the summer.
However, Variety.com reports Simon has now pulled out of the role.
A spokesperson for the production company behind the show, Reshet, told the industry news site that Simon had cancelled his judging role “for his own reasons”.
A rep for the company previously told the site that Simon’s team had got in contact citing “legitimate concerns” over the violent clashes in Gaza between Israeli and Palestinian forces.
Around 242 people were killed in Gaza and 13 more died in Israel during 11 days of violence this month which came after weeks of mounting tensions between Israeli and Palestians in East Jerusalem.
The fighting ended with a ceasefire and talks continue in a bid to halt the clashes permanently.
Jewish News has also reported that Simon has pulled out of the show, with a source claiming the X Factor mogul felt “disappointed” not to be able to take part.
The inside added: “Simon won’t be going to Israel after all. I know it has already been announced that he was one of the judges, but for a number of reasons he just can’t be there in Israel to film the show now.
“Of course he is bitterly disappointed – but it was a decision he had to take.”
The Mirror has contacted a representative for Simon for comment.
X Factor Israel had originally been slated to start filming in late 2020, but production was pushed back until this summer.
The show is due to choose the entry for for the 2020 Eurovision Song Contest and the judging panel features 2018 Eurovision winner Netta Barzilai, as well as singers Aviv Geffen, Margalit Tzanani, Miri Mesika and Ran Danker.
It was due to have been Simon’s first time judging a different version of the show following stints on both the UK and US series.
After announcing his plans to join the Israeli panel, Simon said at the time: “I can barely wait to see what the Israelis have to offer.”
It’s been a tough year for the star as he was forced to cancel a number of TV appearance, including his stint judging Britain’s Got Talent, last year after breaking his back in a cycling accident.
Simon took a tumble from an electric bike and had to undergo spinal surgery.
He faced a long recovery but is now back up and walking again and ready to get back to work.