Don’t live in the US, but in Sweden almost all WCs are gender neutral.
But the scenario is a cis gender male, walking in to a bathroom where he doesn’t belong. And you ask what is stopping him, well in the world where someone is asking you what gender you are, to allow you to go the bathroom; I guess that will be stopping you.
It’s a made up scenario
I don’t think you’re allowed to physically attack anyone at all, could be wrong though. Except in sports, any sport.