This article originally was published by the Spectator.

Have you ever wondered why so few moderate Muslim voices are heard in the public debate? I used to, until I started to defend my faith against its extremist defamers. I then found out that any Muslim who ventures into this arena to stand up against hardliners is subject to fierce and immediate character assassination.

The process is exposed in a Civitas pamphlet, out this month, entitled 'The No True Muslim' fallacy. It provides examples of the attempts to silence people like Sara Khan and Fiyaz Mughal by those who have appointed themselves as Islam's spokesmen. But I can offer another example: the reaction to a recent event at the last Tory party conference.

The discussion, hosted by Policy Exchange, was about the debate on so-called 'Islamophobia' – which, as I have described in The Spectator, is a phrase intended as a weapon to silence, intimidate or even criminalise anyone who scrutinises what extremists have to say. Honed by the ayatollahs in Iran as a form of thought policing, it has been endorsed by various (non-Muslim) political figures in Britain who doubtlessly mean to protect Muslims from attack, but are unwittingly passing a tool into the hands of pro-Islamist hardliners. As a Muslim who has travelled extensively in the Muslim majority world, living in Saudi Arabia and visiting Pakistan for over four decades, I have seen how the device is used to introduce and enforce lethal modern-day blasphemy laws. The idea that Islamophobia could at some point be introduced into British law is terrifying, especially for British Muslims who disagree with what these self-appointed spokesmen think.

This is why I agreed, with 48-hours notice, to fly from New York to the event in Manchester. The debate was robust and sensible, and recorded on video. You can watch it here, and read a transcript here. But what I had been unprepared for was the intellectually dishonest Twitterstorm which erupted almost immediately after the event. It follows a familiar pattern to the reaction (which Douglas Murray exposed) to Michael Gove's recent appearance at the German embassy: a speech is made; the speech is then wilfully misrepresented; and then a tsunami of digital outrage follows. The truth is crushed in the first few moments.

Baroness Warsi appeared in the pages of the Guardian to say that the very existence of the debate made her worried. 'When I first read reports on the gathering, I felt sick to my stomach at what went on. Later, as I listened to a recording of the event, I felt deep fear.' Why? She might have heard views that she disagrees with, but nausea is not normally a side-effect of this. 'Being named "Islamophobe of the year" was joked about by panellists as if it were a badge to be worn with pride,' she says. Given that Sadiq Khan, the (Muslim) Mayor of London, was shortlisted for this ridiculous award last year, what's not to joke about?

She denigrated my fellow panellists, who included Trevor Phillips (a former chairman of the Equality and Human Rights Commission) and human rights advocate Peter Tatchell, and went on in her piece to accuse us of 'dog whistling,' 'Muslim bashing,' and (inevitably) promoting 'Islamophobia'. Unwittingly, she made our point: anyone who challenges the hardliners falls prey to this label.

Not mentioned in the reporting of the recent Policy Exchange event was the hour of repeated statements made by each of the panellists and the moderator, deploring and mourning lethal anti-Muslim xenophobia – the more accurate term to describe discrimination of a Muslim individual for their membership of the Ummah (the Muslim brethren).

Instead, Sayeeda Warsi denigrated the panel as 'disingenuous', a 'panto', and accused us of 'shutting down' Muslims attending the event. No balancing quote was sought by the Guardian from me, nor were Transport minister Nusrat Ghani's extensive arguments about the defence of pluralist Muslims as well as other minorities targeted by Islamism included in the piece. Baroness Hussein-Ece of the Liberal Democrats also joined the pile-on. Strangely, she even discounted the remarks of the leader of the world's biggest Muslim organisation from Indonesia.

At great personal risk, I have travelled to witness and lend support to the de-radicalisation of Taliban child soldiers in the former badlands of north west Pakistan. More recently, I have worked with colleagues at the University of Duhok in Iraqi Kurdistan to support not only the Yazidi and Muslim and Christian survivors of Isis's occupation, but also the rehabilitation of Isis child soldiers who were enslaved and indoctrinated. Nowhere have I been labelled as Islamophobic. Unlike in Britain, in these societies Muslims have no problem with describing a jihadist as a jihadist and calling out extremist ideologies.

None of my actions could be considered Islamophobic – quite the opposite. But speaking about the Muslim victims of Islamism and the extremism of some Muslims in today's Britain is apparently enough for me to be denigrated and demonised as Islamophobic by high-profile Muslims in Britain. From a distance, this looks worrying. Close-up, as I found, it's terrifying.

I flew a 6,700 mile round trip in 24 hours to make one key fact patently clear to the Conservative party: be under no mistake about what is at risk here. They should see the 'Islamophobia' campaign for what it is: what the late Christopher Hitchens referred to as a 'cultural fatwa', where all discussion of Islam let alone Islamism becomes off limits, followed by the formal criminalisation of exactly such discourse. The reaction to our panel's discussion of the subject makes our case. In even considering whether to outlaw so-called 'Islamophobia' you only empower Muslims who agree with Baroness Warsi and those like her with a weapon to silence pluralist Muslims like me.