Fallout 2 and Fallout 4 are tied in the running for me.
Fallout 2, when I played it back in the day, felt so tonally bizarre. It clearly thought that it was a lot funnier than it actually was, and that really hurt the game. It didn't take itself seriously enough to be enjoyable. While Fallout 1 definitely had plenty of jokes and dark humor, it was still creepy and heady enough that you could get lost in its world. Fallout 2 never gave me that sort of feeling. Or maybe I never got the chance, what with all the constant crashing.
Hated it.
Fallout 4 feels like STALKER-lite in its gameplay (not terrible), and in its presentation - hell, I don't even know, man. It's so bland and milquetoast and strange. Its fixation on settlement building felt completely and totally wrong. Its fixation on the dog companion felt completely and totally wrong. All of it felt wrong. It was tonally discordant and offputting throughout. It did have its moments. Nick Valentine was a good companion (at least in my opinion). Honestly, so was Codsworth. Guess I just have a thing for robots. That, or the person/person(s) responsible for writing the robots did a better job than whoever did the other companions.
There are a ton of franchises I can think of that have a black sheep in them I dread to play, but I'm thinking about Fallout because I recently reinstalled F4, and was shocked by how mehhh it made me feel, right from the start.