Israel is a client state. If for example there was political will in US tomorrow for a settlement they'd have to accept it and BDS would have virtually no hand in it other than providing general pressure. Worrying about anti-semites is sort of like worrying about communists taking over america.
I don't think it's helpful to dismiss anti-semitism like that. For example, in the UK in 2014 anti-semitic attacks reached record levels, more than double where they were in 2013. However, Amnesty International rejected a motion to fight against anti-semitism in the UK, the only motion it rejected at the entire annual conference of the organisation. Also, as I mentioned, I have personally witnessed people using the BDS campaign to push blatant anti-semitism. I'm not inherently opposed to boycotts and sanctions, but there is far too little effort being put into ensuring that the BDS campaign does not become a vehicle for anti-semites.