I don't mean to imply a fence is a barrier which cannot be defeated. A fence has the potential to reveal intention and add a psychological barrier. It implies you have an interest in privacy. To defeat said fence is to reveal violent, potentially harmful intentions. If intimidation is the goal they may stop at the gate. Once intention is interpreted preparation may commence.
Keep them at a distance, allow communication without confrontation.