There are some things that change based on the selected race, actually. Don't know about gender or fate selection.
Well, at least that. Right now I just keep getting annoyed with people insulting Almar WHEN I AM ONE.
I'm not a gluten for punishment, so I rarely turn up the difficulty, but I never turn it down. I like to play my games as the developer intended them to be.
I would figure if there are difficulty settings then the developers intended them to be played according to the skill level of the person.