I'm just constantly shocked when white men think they get to dictate what's offensive or not after centuries of these other people being victims of it.
I think it's less that and more "I am the one who gets to ascribe intent to what I say, not you."
Offensiveness is a bit of a two-way street. I could say some of the most heinous things to you, but if you don't have a fuck to give, you might not be offended. Rather, you might be amused at my attempts to offend you. On the other end of the spectrum, I could say something innocuous like "niggardly" and you might take offense at it. It's not so simple as "X is an offensive word. On some level, you have to "allow" yourself to get offended (although "allow" implies a choice which I don't believe is often the case so it's probably not the best word to use there).
But there's absolutely no way to separate the two meanings when they are in use simultaneously, which means that even "non-homophobic" usages are still implicitly homophobic regardless of what the speaker intends. When Dark Octave calls only "someone of despicable character" a faggot, he is not working to separate it from its homophobic usage. Instead he is adding one more voice to the chorus that says that a "faggot" is a bad thing, and until "faggot" means something other than "gay man" - and we're nowhere close - then he's implicitly saying that being a gay man is a bad thing. As far as I'm concerned, even the supposedly non-homophobic usages are homophobic.
Let's take that to the extreme. Imagine two scenarios. In both cases, you're walking down the street at night and cross by a man who looks at you and yells "Faggot!" out of the blue.
In scenario 1, the guy is just an asshole. He is saying it with hurtful intent (not caring whether or not you're gay). That's clearly offensive.
In scenario 2, the guy has Tourette's which manifests itself through coprolalia (i.e. the involuntary uttering of obscene words or phrases). He had no control over saying what he did. That doesn't strike me as offensive. Granted, you wouldn't know of his affliction and so might understandably take offense at what he said in your ignorance of his condition.
But what's the difference between the two scenarios? The context is essentially the same. Same street, same time of day, same culture, same complicated history of the word's usage in our society, same understanding of the word's taboo nature, etc. I would argue that the only thing that is different is intent. In scenario 1, the man is clearly intending to offend you. In scenario 2, he is not (even if it's not known to you at the time). From what you said here, you'd say that the man in scenario 2 exhibited homophobic behavior even if what he said was completely beyond his control.
Like I said, it's an extreme case. However, if you concede that it is in principle possible to call someone a "faggot" without being homophobic about it (which I admit I don't know that you will concede that point), then I think it's not a terribly large leap to think that other instances are similarly not offensive in that way. After all, language is 100% context-driven. Certain words and phrases--even just sounds--might have more significance to me than you or vice versa. There are generational differences, regional differences, cultural differences, and racial differences (among others) in how we use words and language. For that reason, I feel like a rule that says "this is implicitly homophobic or offensive no matter what the context" is a bit heavy-handed.