The higher level your gems get the higher the greater rift requirement to get an upgrade on it. Plus the end game is all about pushing your character to its limit. If you are just grinding for gear then do the GR difficulty that gives best difficulty to bloodshard ratio for you.
Thanks for this. I just came back after a hiatus and GRs are new to me. Was wondering the same thing.