A useful reminder:
Having more workers means more work being done, which should mean that society and everyone in it has more stuff and is better off.
If you find yourself in a situation where having more workers (such as women, or immigrants) somehow makes you WORSE off, then there's a major problem with your economic system. You should change it.
In case this comes across as being sexist or racist, whatever discriminatory label you wish to affix to pointing fingers at women or immigrants, read the statement again. The idea is not to be spoon fed but to think.