Anytime 🙂
In my opinion, next to Union Square (where these beautiful buildings are located), a fantastic choice is a walking tour by the river.