It does, but it doesn't talk to all peers just some peers. And plenty of peers are behind firewalls so not directly addressable: they can call you, you can't call them. So since you can't call them, you have no idea if they are there unless they decide to contact your node (which they won't do as long as they have their desired number of peers).