Jaron Lanier said this on a podcast
"Perpectual annoyance is the cost of freedom". (or something to this effect)
He goes on to explain that people are annoying, and that true freedom isn't about doing whatever you want and avoiding people. It's about what you're able to do, in communion- however annoying that may be- with people.
People who want to take the "politics" out of every day society tend to be the people who try to take the "people" out of society (at least, the ones they don't like).
So be wary of these types of people.
You share this world and this life, with people.
You can choose to go off into the mountains, you can have that kind of freedom. Nobody bothers you, and you don't get to bother anybody either.
But as humans, we need and want to be with each other, and while this is not the clearly demarcated and 100% "do as you please" freedom of living by yourself in the mountains, you can still live free in this setting.
How free that is, is up to your tolerance level for perpetual annoyance.