"it was never meant to describe prejudice against all Semitic language speakers...I can see how it can be misleading."

It appears that term was deliberately coined to deceive people, because Jews or Judaism is a perfectly good word to create the term 'antijewish' or 'antijudaism', but using antisemite conceals the crimes committed by Jews against Semitic peoples by implying crimes against Semites are crimes against Jews. It is deliberately misleading with malevolent intent. No one nor nothing is more antisemitic than Israel. Ask any Semite in Palestine, Lebanon, or Syria who is harming them the most, and they will certainly inform us it is Israel.