Who the heck do I think I am?

I wouldn't worry about it too much. FA is clearly #1 and so people will bash it for exactly that reason, without worrying too much about whether their complaints have any substance or not. Given that people seem to be making complaints that you're erring on both sides that you're doing both too much and too little about inclusion and political correctness at the expense of absolute personal freedom, you've probably got the balance about right.