IQ refers to a bunch of features of a game's visuals. Games with great IQ will usually have a very good anti-aliasing solution (to eliminate jaggies), a high level of anisotropic filtering (to ensure textures remain clear and sharp from any angle) and will often run at the highest resolution available on the platform they appear on (so in the current generation, 1080p).
Lots of things can contribute to a perceived "bad IQ". No or poor anti-aliasing and anisotropic filtering, low resolution output, tech designed to upscale lower resolutions to 1080p, etc. Some people might stretch to include screen-tearing, poorly implemented effects (such as motion blur) and stuff like incorrect frame-pacing as well.