Math was the worst for me too. And now I kinda really like it! I also find it rather simple and easy. I believe it's probably the easiest subject in school! Perhaps the problem is it isn't taught right, or you forget the material during summer and then you have these holes in your learning that make it increasingly difficult as the years go by. Or they teach you geometry for a really long time and by the time you're done you've forgotten all about algebra or something, so again there's holes in your learning.
With other subjects, it's easy to learn just enough to write the test and then forget, cos nothing builds up, nothing depends on anything else. But if you learn math just to pass the test and then forget it all, you're screwed. You can't just pick up the new stuff on the spot, and it becomes obvious that you were never really learning cos you were never really interested, whereas with other subjects this fact remains hidden cos you can just memorize the new material.
RE: The Mystery of Learning Mathematics - Why a lot of students Fear Mathematics