Many believers will argue that the harm done by religion isn't religion's fault. Many will point out all the wars, bigotry, fraud, oppression, quashing of science and medicine, and terrorizing of children done for reasons other than religion. And many will argue that, even when this stuff is done in the name of religion, it isn't really inspired by religion at all. It's inspired by greed, fear, selfishness, the hunger for power, the desire for control... all the things that lead people to do evil.
And they'll have a point. I'm not saying that religion is the root of all evil. I'm not arguing that a world without religion would be a blissful Utopia where everyone holds hands and chocolate flows in the streets. (And then we all die, because the chocolate is drowning us and we can't swim because we're holding hands.) I don't know of any atheist who'd argue that. I know that the impulses driving evil are deeply rooted in human nature, and religion is far from the only thing to inspire it.
I'm saying that religion provides a uniquely stubborn justification for evil. I'm saying that religion is uniquely armored against criticism, questioning, and self-correction... and that this armor protects it against the reality checks that act, to a limited degree and in the long run, to keep evil in check. I'm saying that religion takes the human impulses to evil, and cuts the brake line, and sends them careening down a hill and into the center of town.
Yes -- even moderate religion. Not to nearly the same degree as extreme religion, of course. If all religion were moderate, ecumenical, separate from government, supportive of science, and accepting of non-belief... well, atheists would still disagree with it, but most of us wouldn't much care.
But moderate religion still does harm. It still encourages people to believe in invisible beings, inaudible voices, intangible entities, undetectable forces, and events and judgments that happen after we die. And therefore, it still disables reality checks... making people more vulnerable to oppression, fraud, and abuse.