Here's the thing. In ancient Japan or many other parts of the world, limited communication and travel made it difficult for information to spread widely. It's impossible for everyone (in that huge the open world ) to have seen Yasuke. Even if they had heard of him, a person from any other race, whether white or black ,would have seemed strange to them.
The Japanese (and even broader Asian societies at that time) were very xenophobic, and they held prejudices against other races. And yes even if they saw him every day, they would still have prejudices. It was a feudal society, so it's unrealistic to expect them to treat someone of a different race like modern society. A lots of Asia countries ( include Japan) used to refer to white people faces as ''devil faces'' "white demon" in the past, and some people still use this term even today. So expected they treated him like normal japanese man is unrealistic.
Idk why do I need to explain this... You should watch more movies or books about that era to see how they treated other races.