It's true, but then, it's all about the design of the game, this in itself is just an effect. In Unfinished Swan, you basically only use it to navigate the environment, and that's it, so this explains the need for introducing other mechanics once you've got that one figured out. There's arguably an aesthetic component as well, but if you're not into painting your surrounding, this has no effect on your path/play.
The idea I had back then involved temperature scales - color would actually represent a combination of type of texture to the touch, and temperature, and you were meant to "feel" your way around your environment by touching (not throwing) in front of you. Touching surfaces that were too hot or cold would damage your sensibility and you would subsequently lose some of your perception power, having to rely on longer presses to get more of the environment revealed. Anyway, those are the bare basics of the idea, but I still think you can build a whole varied design around that single effect and have it work for quite some time