This update doesn't include new content, however, I managed to crank up the performance by about 100%.
The problem was that I used normal ArrayLists that got synchronized across every thread (cuz they all tried to access it) and I didn't know that synchronized basically halts the thread until no threads access it.
Anyway, I changed these Lists to ConcurrentLinkedQueues, which to my understanding are basically stacks that are thread-safe, so threads won't have to halt to access them.