The problem is you're not just choosing for you. You're choosing for everyone you interact with. Wearing a mask protects them. Not wearing a mask puts them at risk.
"Sure, you're free to risk the lives of strangers you interact with throughout the day" is the position you're taking. As one of those potential strangers... no. You refusing to take steps to protect me means you are violating the non-aggression principle by putting my life at risk.
Yes, we should be focusing on helping one another: That means wearing masks, social distancing, washing your hands, and staying home when you can. That's what "focus on helping one another" means right now.
If you're not doing that, then you are, objectively, "not helping one another."
RE: Mask or No Mask: Fight Fear with Love