I thought dual wielding had some 15% speed bonus or something.
It does. What I was getting at is that since the game goes Main Hand swing, Off-hand swing, Main Hand swing, Off-hand swing, etc. it's possible to end up in a scenario where you'd have been better off with a single weapon than dual wielding. To give an example, let's say you have two weapons: Your Main Hand with 1.5 Attacks per Second (.667 swing speed) and an off-hand with 1.2 Attacks per second (.833 swing speed):
t = 0.000: Main Hand swings
t = 0.833: Off-hand swings.
t = 1.500: Main Hand swings.
t = 2.333: Off-hand swings.
t = 3.000: Main Hand swings.
etc.
If your Off-hand is bad enough, you'd have been better off swinging your Main Hand only. Essentially, this just means that just throwing "any ol' weapon" into your off-hand won't increase your damage. It needs to at least be comparable to your Main Hand weapon since you're alternating which one is attacking.
It also means there's the possibility that swapping your off-hand for a better DPS one isn't ALWAYS the best case scenario but that's mostly due to other things (e.g., old off-hand had +10 STR which gave more damage than replacing it with a +0 STR Off-hand that has a listed 5 DPS increase). But this is getting a bit obsessive at this point probably.
Basically, if you plan to Dual Wield, just check your listed DPS when you change weapons. or just not care as it's most likely not a huge deal.