Gah, very disappointed in the performance

... to be absolutely clear I'm not meeting their minimum spec. I have my trusty i5-3570K and 16GB of RAM, so okay there. But I have a 560Ti, not a 750Ti. The cards are really not that much difference performance wise, and normally I do not have any problems. I'm not crazy and expecting miracles, I went in expecting 720p everything set low/off. I have 1GB VRAM vs 2GB, so I have the textures set to "low". I figure this should be "ok" because at 1080p high quality textures the game is set to use 1.6GB VRAM.
That's what I'm running, but it's very poor. The starting area right at spawn (I assume due to the fog?) was 12-14 fps. So bad it kicked me out of the game to disable online mode.
Afterwards, the steam FPS counter says 22-30 which is what I was expecting, but I can't believe the game is actually playing at that rate. Everything I do feels like my character is pushing through mud. I managed to beat the first boss, but I wouldn't say it was pretty.
I know the 560Ti has got to go, but I'm waiting for Pascal, I can't afford to buy two video cards this year simply due to one game. I was expecting it to be slow, but not terrible, not after I played through all of Witcher 3 in 720p and most other details on (high textures) at 50+ fps.