The former life member says his support of those voicing ‘trans-critical’ views is a matter of free speech – but others say it’s a question of what values the party supports