"gyro" vs "IR" is the exact same difference as "analog sticks" vs "mouse"
Inside-out positional tracking is 3D mouse tracking, you are selecting a point in space. IMU tracking doesn't have any reference for absolute position, it doesn't know where you are in space. All it can measure is amount of acceleration. You work under the assumption that the math is absolute and will arrive back at the origin with the appropriate negated motions, but it doesn't work like that in practice. To arrive at position using an IMU, you perform integration of your acceleration to arrive at velocity, and then you do integration of your velocity over time to arrive at position. The problem with arriving at velocity is that one must account for gravity as a constant force in the equation, which means you need to constantly know which direction is down. And IMUs poll at much higher rate than the magnometer, meaning you can actually misread which direction is down, that will throw off your calculation entirely. These errors are cumulative, meaning that, in very short time, the errors ramp up into infinity in any direction.
Games like Uncharted: Golden Abyss and Splatoon prevent this from happening from making sure your IMU readings don't ramp up in any given direction wildly. That alone accounts for the discrepancy in their accuracy.
Just like mouse vs stick, you are controlling a point in space vs acceleration to a point in space.
And, again, in any outside-in positional tracking or inside-out positional tracking solution, an integrated IMU is present. None of those solutions are purely optical; purely optical solutions are extremely latent. They are achieved using a weighted sensor fusion with the IMU readings. It's impossible for an IMU to be "more accurate" than inside-out or outside-in positional tracking, because those trackings are IMU trackings.