Yeah, I know exactly what you mean. Has happened to me many many times, especially in Prague and Krakow as they are both historical cities full of stunning medieval architecture :)