It's all about the games. So far, neither handheld has proven they warrant my dollars. Just a bunch of lazy ports and 'lite' versions of existing console franchises.
Stuff like video encoding and hax manga viewing, which seems to get a lot of attention here isn't really worthwhile for me to own a PSP, not to mention the massive capital outlays I need just get get the unit plus all the unmentioned extras, like memory sticks that cost as much as the unit itself and coming from a person who uses SD card.
As for the DS, the games, well most of them, are just plain boring and uninspiring. There's a few niche touchy type games that's going to get the hardcore Nintendo/otaku/quirky crowd excited, but nothing like what Mario 64 or Ocarina was for the N64 and looking ahead, there's not much yet known about the line-up to really get me excited. Animal Crossing might do it for me, but that's if it's not another revision of the same game Nintendo has peddled so far and it actually offers new stuff to do and maybe on-line play. A lot of ifs.