Some Famicom games don't work well with PAL consoles because of the 50hz output. I know for example Quinty just crashes when you start a world. And some games can have graphical glitches like Kid Dracula (still playable though) and Wai Wai World 2 (can't see half the screen) on PAL consoles. And yes games that use special sound chips like Castlevania III won't play the enhanced audio unless you modify the console and adapter.
On an NTSC NES every game will work fine, but the enhanced audio is still a problem.
Thanks for the input. Personally I'm gonna get that AV Fami eventually, but my friend said he had an extra adapter I could try, since I have a few Famicom carts lying around (I actually picked them up at a flea market, how strange is that, this is Finland we're talking about), just to try it out.