The number of matches you get depends on how many people in other guilds selected the same Fray. You will fight against everyone else who chose that particular Fray. So if you only have 2 matches and some people have more, it means not as many other guilds picked your Fray.

I expected this would happen in testing, as the number of QA testers is relatively small. Once we launch for real and Brawls are opened up to everyone, battles should be more evenly spread out.