BioShock 2 only had two faults, in my eyes:
1) The game starts out very, very slowly. The early areas are a bit bland and forgettable.
2) The scenery outside the windows wasn't as pretty as BioShock 1; usually, it was foggy blue water and a sparse kelp forest.
However, unlike BioShock 1, BioShock 2 only continues to accelerate, slowly but surely, and it reaches a high top speed. Barring the absence of a Fort Frolic equivalent (although Fontaine Futuristics comes close), BioShock 2 tells the superior story to BioShock 1. And it has superior gameplay, as well.
Where does Infinite factor into all of this? In terms of the narrative, Infinite is the best of the three, imo. It starts strong like BioShock 1, it ends strong like BioShock 2, and it's consistently interesting throughout. With a single focused ending and no banal morality system, Infinite has the narrative strengths of its predecessors, and none of their weaknesses.
In terms of gameplay, the controls feel better in Infinite, in terms of sprinting and turning and jumping -- just the right amoung of tilt and bobbing. The gunplay feels greatly refined, as well.
However, the combat doesn't encourage the same twisted creativity as BioShock 2 and, to a lesser degree, BioShock 1. It's just as good, in a different way, with more focus on navigating the environment, from the skyrails to the tear transformations. But I think I preferred rigging the entire environment in my favor: turrets and sentry bots, mines and trip-wires, and so on.