I think it also gets hit for not actually doing that much new, just a lot of stuff bigger. I think the blueprint stuff is the most significant new feature and WHO CARES. Everything else is just a mix of cool stuff from the first two games, plus the showcase events that desperately need a rethink. Australia is a pretty neat setting, though.
I also feel like certain genres just never get a ton of respect for GOTY, and racing is absolutely in that list.
Plus, just to go back to the PC version for a second: when one of the big things about the genre is extremely responsive controls and split-second timing, and you completely ruin that by putting out a PC port with wildly erratic performance and stuttering, you have RUINED YOUR GAME. I legit don't understand why the Forza Horizon 3 stuff doesn't get nearly the outrage Arkham Knight got for its PC port.