Anyone is allowed to have an opinion on the matter... There's no "wrong".
I personally think that it's the most annoying thing on a TV set... I could accept bad contrast, bleeding, even SD and 30fps but not a 40ms lag. It probably has to do with some of the games I play... When I bought my TV set, I was playing video golf on both TV and PSP, and before I found the "game mode" on my TV, the 30ms lag was just feeling awfully wrong. I though I had made a bad purchase (I bought a revised edition without the lag information, which was available only for the previous model).
That's the reason lag is my first filter when looking for a new TV, and unfortunately recent sets have often very high lag (40-50+). For this reason, I think the data should be as available in feature sheets as contrast ratio, resolution, etc.
I'm perfectly fine with people saying they don't care, though. I think it's quite close to the 30/60 fps discussion.