Victoria does has some nice whale watching, but mostly during the summer months. I've seen humpbacks and minke whales, not to mention orcas out of there.
The humpback whale show here is one of the best in the world though, highly suggested checking it out. Plus Hawaii is pretty nice in general :)