Sure, I can agree with that, but I do think it does feel at times that because there are a handful of racist idiots out there ANYBODY who does oppose Obama for legitimate policy oriented reasons is accused of racism.
My favorite version of this is "well, I didn't hear you complaining about spending when Bush was president, you must have a particularly SPECIAL problem with Obama." sometimes this is framed as a political disagreement, sometimes it's a veiled accusation of racism. In reality there are a couple of non-racist responses. First, often the poster HAS complained about Bush in regards to spending, but anti-Bush voices often get lost in the chorus. Second, even if you generally felt spending was high under Bush (highest spending President since LBJ, just ridiculous on every level), Obama has taken it to yet another level thus it's fair to be even more upset with Obama.
Personally I think they've both been "bad" on a variety of issues I care about, but people often view my opposition to Obama's policies differently than they do my opposition to Bush's.