We are of course talking about the "rough men who would do us harm". It is incumbent upon every ethical person not to obey their orders! There is absolutely nothing wrong with defending against aggression. What is wrong (under a Non Aggression Principle ethical standard) is to initiate aggression or follow the orders of those who do.