It always makes more sense to do it on the GPU, unless you've sunk otherwise wasted money into a failed CPU design.
depends what you're doing.
You can't do *everything* on the GPU. Well, maybe you could, but then you're taking time away from it doing other things. You could almost not bother with a decent CPU, just something good enough to handle basic housekeeping, and let a GPGPU do everything else. But I can't help thinking that wouldn't give the best results.
modern multcore, multithreaded CPUs have plenty of grunt for tasks that free up the GPU to do other, more graphically focused things.