if you create a real set, you have craftsmen, you have people who know woodworking, you have carpenters, you have painters, you have all these skilled artisans working together. then the lighting guy comes in does his thing, director of photography, etc. the lighting and camerawork they set up is seamless, instantly conveying a real place. then actors come in, they act in that real place, react to the geometry of the set, etc.
compare that to a CGI set. the actor has no idea what he is looking at, because he is against a giant green wall talking to a ping pong ball. so things like improvising or using the set naturally are out of the question for the most part. then the lighting is 100% artificially re-created in a computer, and at least at THIS point in time, the human eye can still instantly tell a fake image from a real one because we instinctively know how light works. also yes you have craftsmen but we are talking a digital arts student with 4 years of experience imitating a carpenter (or traditionally trained matte painter) is no replacement for someone who has done work in the tangible, physical world.
you can go on and on through all the elements. CGI indeed uses physics and models to approximate things like light and gravity, but they are no substitute for the real thing. when we look at a real set, we feel more immersed, like it could be a real place, because in fact, it is a real place.
plus there are a lot of elements of digital filmmaking that act as shortcuts. if something is messed up, rather than do it again until you get it right, they can now just remove things in post. or do things like when George Lucas would edit together multiple takes in a single frame. when you do something like that, you remove the context of the actor's motions, the artificial nature is instantly communicated to the audience, who is kept at a distance.