Oh it definitely can be worse. The majority of high end 4k TV's are about 45ms up to 110ms in input lag and that is in Game Mode, even some high end 1080p sets also suffer from poor input lag. Unless you are in competitive MP and play as partially a profession, 40ms is not too bad. Different people have different perceptions of input lag, so it can vary person to person. I generally have had sets that range from 20ms to 38ms and I have been fine.
A friend bought this set for his kids and he is also into calibrating TV sets as a side job, if he allows it, I will ask him to let me post his calibration settings for this set so that those that bought the set can use them.