One other reason for taxes, is so that the government can intervene with the free market where there is an inefficient allocation of resources. A classic example is provision of utilities like electricity or water.
If a private company provided these services, then they would have a monopoly and charge what they like, which would be damaging to the economy. However competition is inefficient in this case, because why do we need 2 or 3 electricity grids. This is where governments are either the provider of these services (hence the taxes), or have pricing controls on the providers of these services.
RE: Why do taxes exist??