It's easy to call people racist or xenophobic, but your opinion might change if you lived in an area that over the past couple of decades has gone from just mixed working class to seeing women every day in the full veil and hearing any language but English spoken in your local supermarket, plus jobs decimated and local community areas reduced to a string of betting shops. I think these people have real concerns, confusion and fears that aren't being addressed - they're making connections between what's happening on the news and to the economy and what they see in their daily lives.
This is true the same way that people who are against same-sex marriage or racial integration or the people who whooped and hollared at Enoch Powell 40 years ago are all responding to very real anxiety about the changing world. The classic divide between urban-rural or cosmopolitan-provincial. I agree that it's impossible to discard the role fear plays in governing these peoples' daily existence, but the response to that is not to mollycoddle the fearful at the expense of whatever outgroup is currently being scapegoated as they lead their daily life. It's all well and good to say "What about the men who suddenly sees women in a full veil", but he's suffering a far lesser offence than the woman wearing the veil who gets leered at, yelled at, or worse--used as an example of the evil forces conspiring to destroy the nation. The specifics of integration issues vary across time and space, but the core is the same: I would absolutely always want to take the side of kindness and fraternity than the side of fearfulness and bunkering. it's the same in every country. In America it's worries about taquerias and fruit guys speaking spanish and playing footie, in England it's people named Muhammed. I know of course that these people are not cartoonish racist fascists, they're simply isolated and fearful of a changing world.
The government can be blamed for its failure to provide for the economic prosperity and quality of life of the working poor, of course, but not for ignoring the racist calls to Make England English Again. Even if it causes tension and anxiety in the mean time, we need to encourage a spirit of cosmopolitan brotherhood.
I mean, the language stuff is a perfect point. Yes, there are many people who get anxious and angry when they hear people speaking non-English languages. And? A man walking down the road should be able to speak English. Or French. Or Spanish. Or Farsi. Or !Kung! Or even terrorist languages like Arabic or Mohammed. Stripping their dignity by saying they are somehow less human--or less a part of England--or less in any way because of that difference is giving into fear.
I think your post is telling because it dispenses with the pseudo-economic arguments and makes this what it really is: a referendum about how comfortable one is with the strange and foreign ways the country is changing. Glad we're no longer pretending.