How? One may be more negative than the other but they are the exact same level of generalization.
I'd say by the definition of the term only. There are many layers to racism.
I do agree that it isn't fair to judge and entire race's attractiveness based on someone's limited knowledge and ignorance. Though saying someone is racist as a blanket term is kind of unfair. Though I do see the relation.