I think it's more likely people gathered in sizes larger than families to protect against rival groups. This explains why Neanderthals died out. They never organized and in spite of superior physicality and possibly intelligence, they were overwhelmed by superior numbers.