While I also doubt highly that the polycount has anything to do with overall game size, this specific part is completely wrong, a low polycount model would be under 300 or so kb, where as a large high detail model can often be +50mb, partly because most models contain not just their high polycount version but also the various lod level versions of each model too, when not separately stored as an external file, in reality the polycount is simply lower because the engine cannot handle the higher polycount environment on the hardware it's running on.
Now it's either an engine issue or a hardware performance issue, I'd be inclined to say a bit of both since the game runs like a dog regardless.