I just realised that you can't delete a 3DS game from the SD card without also deleting its saves. Way to discourage people from getting games digitally.
Apparently it is possible to extract the save file from the SD card, and back it up, although it requires a little bit of digging.