I was the same with olives and peppers.

Mushroom consumption is on the increase in this house as my youngest daughter loves them. I often use them in place of chicken if making a curry with one of those jars of curry sauce. The massive ones make a cracking alternative for a beef burger when cooked on the grill. I feel far less guilty about topping with cheese and bacon than if it was beef.