In your happy-go-lucky imagination of reality maybe.
In practice, it very much does.
Indeed. Ideally, such laws would not be needed and the social contract and societal consequences would do the work. However, we don't live in an ideal world and when most people hear about someone they know engaging in hate speech they either:
1.) Are too scared to speak up in any meaningful way and thus don't engage in any particular action.
2.) Are completely indifferent to the situation and feel it's not any of their business.
3.) Actively agree with such individuals.
Those who are all:
a.) concerned by such remarks
b.) have to courage to actually speak up and say something about them
c.) have the ability to inflict some degree of societal consequences that have the potential of actually changing behavior
are in far too few a number to be effectively in lobbying meaningful consequences. Thus, the necessity of the government/law needing to get involved, because the social contract by itself is clearly not doing much to stop it. The entire history of the United States is a case of point in that--things only change in the face of either violent action or laws/the courts getting involved and using their power to force a societal change. Till such an event occur, things either continue unabated or get worse. The "free market" approach to truly unabated free speech is a complete failure, and the entire history of the United States is the strongest evidence you could possibly find.