The previous responses covered a lot of what I wanted to mention, but I'll delve into some more detail on what I consider to be the bigger offenders.
- The rhythms used on Normal could get surprisingly tricky, and could be difficult for beginners unfamiliar with playing music: off-beat entrances, longer eighth-note strings (> 3 notes), syncopation, triplets, etc. Look at all the hate for Clockwork Clown!
- The note patterns used are relatively more complex for that stage of the game, in terms of positional arrangement and repeated use of notes on the screen.
For example, the following would be straightforward with 3 notes, going from left to right:
But if it were six notes, going from left to right, then right to left, it'd be difficult to react to without knowing / remembering that it's coming.
Another example:
Without knowing that the notes will come in at a clockwise order and without catching what the starting note is, this pattern can be a struggle for novices, particularly if they are on an eighth note rhythm.
I've also missed entrances fairly often when the start of a sequence appears far from where the previous one ended, particularly when the MV is rather colorful / bright. This may be more a personal problem though, playing on the PS3 version and sitting somewhat close to a larger screen. It would be great if there were an option to dim the MV though.
While these play to the unique presentation of Project Diva's notes, they are also relatively more dishonest for rhythm gaming in that they can make easy rhythms harder to parse.
- Link notes. There are typically four cues to catch the correct timing for a note: the music, the note flying in from off-screen to overlap with the receptacle, the timer on the receptacle, and the relative spacing of the receptacle with the ones around it. Depending on the player, they rely on a combination of those to varying degrees. I feel like link notes get particular flak since it robs people of two of those.