Yeah, Arcane seems to be kinda weak in this game. You get tons more attack by putting points into Strength or Skill, or a combination of both (for weapons that scale well with both stats, like Ludwig's Holy Blade). And if you look at the scaling for each weapon, you'll notice that most of them have (a low) scaling with Arcane. That scaling only takes effect when you slot in an Arcane/Fire/Bolt gem onto your weapon. Surprisingly, Ludwig's Holy Blade has great scaling with Arcane, after a couple upgrades. But unless you were making an Arcane build from the start, it is probably better to ignore that stat.
What you can do, and what I did, is buy a second Cleaver/Axe/whatever and put your Arcane/Fire/Bolt gems onto that. Now you can have a normal weapon and an elemental weapon (that will probably do less damage). Some enemies are very resistant to physical but are weak to Arcane/Fire/Bolt, namely the
and
in Chalice Dungeons.