The difference in performance between quality presets can be up to 2X, the lower the upscaling ratio is - the heavier is the upscaling process.
The bigger difference is stemming the output resolution though which is why we see games upscaling from 1080p to 1440p only and not to 4K.
This is for FSR2 (FSR3 SR is similar albeit a bit heavier overall):
gpuopen.com
It's similar with FSR4 but it's up to 4X as heavy vs FSR3 even when running on dedicated ML h/w of RDNA4.
To answer your question of "If you upscale 1080p to 1440p does it have the same cost as upscaling 1080p to 4k since they are both the same internal resolution?" the answer is "no, upscaling to 4K would be considerably more expensive".
6700XT is at 0.9ms when upscaling 1080p to 1440p and at 1.7ms when upscaling 1080p to 4K.
What is a bit counter intuitive is that upscaling from a lower internal resolution would in fact be cheaper than upscaling from a higher one to the same target resolution.
Same 6700XT is at 1.7ms when upscaling 1080p to 4K but is at 2.0ms when upscaling from 1440p to 4K.