Everything actually comes down to preferences as you rightly said. I attended a single sex boarding school myself. Despite it being single, we had guys that'll go to any length 'never to be caught unfresh'.

For me, the single sex environment wasn't the best growing up. For some reason, the absence of females was was overwhelmingly obvious especially on weekends, when there was no female in sight. Not that I actually wanted to do anything with the females, don't get it twisted, I just craved the feminine audience and got easily tired of being surrounded by boys all the time.
It also affected my relationship with the female gender when I got into uni. Took me a while to finally be at ease when talking to the opposite sex.