Yes, the past is important. When the party go somewhere, this place didn't appear from nowhere. To merge a place with the plot and possible hooks coming from the PCs, that's the fine art of running a game. Most of the time I think about the best connections afterwards ;)