The retail price is irrelevant. It's what the market is willing to pay that matters. Unfortunately, the used market isn't willing to pay much if anything for the Xbox Series Consoles.
I have a PS5, but unless you really want its exclusives then I don't see why you shouldn't keep your Xbox and play the heck out of it. It also has the most comfortable controller.