Only XF3 is really worth using for most of the cast. Of XF1 is used in the neutral, smart players burn their own XF to be safe (the gods do this consistently). Once XF3 is burned, the game becomes less thoughtful for the player using it because everything is so much safer. For example, Skull players can go ham with Meteor Smash. For other characters, like Arthur, it creates zero depth situations like Gold Lance chip infinites.
Basically, X-Factor adds close to nothing in the play-by-play of the game because X-Factor counters itself, and once it is activated, the game changes in such a way that tends to remove thoughtful considerations. If Wolverine pops XF3, most players will do only one thing: run away. Maybe try to block. The neutral is dead, though. A few characters like Doom might get out to attack, but they will just spam Photon Shot until the duration is over (zero depth, again).
TAC infinites reduced XF depth even more, because almost everyone runs a TAC infinite character second, so there is no incentive to burn XF for early kills, either. If the game had lower damage opportunities, it might add something more.