That is a disaster for the building, walls, but it's even bigger problem for you. You could develop asthma or other respiratory diseases. That is no joke.

Some people say things like "just turn on the air con, that will deal with the moisture" and yeah,I get that this is the case but I'm not trying to run my AC year round and have a massive electric bill buddy.

That would make it even worse. If there's mold, could cause even more harm.