Crazy weed

This weed or plant grows very fast. This is from just this summer. These weeds started growing in the yard about 5 years ago, and every year the numbers have about doubled. I don't know what it is, but it has a dark red stalk, and when they get to this size, there are some dark blue berry's growing on it. I need to find out how to prevent these from growing back every year.