It most probably has to do with the fact that the museum, in most cases, will have more to offer for one to see and usually gives you solid information on whatever you are viewing.

Our time is essential to us, and we want to make the most of it. I could take multiple people with me to the museum, and each might find something they deem interesting. As for the temple, everyone that goes with, should like to visit temples.

Also, temples aren't as easily found as museums across the world, so take that in to consideration. Maybe there aren't any temples in your area that are worth a visit, thus you'd rather go to a museum.

Hope this helps.