Yeah, the Matrix clients are not absolutely perfect yet, but they are working on it. Also, some people have problems with messages decrypting slowly too if their device is a bit bogged down with other cpu cycles. So when you load up a new encrypted chat or chatroom, it takes your cpu a bit of time to decrypt everything for you.