Why is not wearing a tie considered impolite in some situations?

Don't wear a tie! It's their problem not yours. If they don't like it and they have the upper hand, have them show you the door. That is one place you don't need to be and anyone else can just go hang.