4GB is fine for many use cases, but not alland if you go over that, you'll start writing to disk, which is where the 5400 rpm drive will hurt you. Having more RAM means that some of the disadvantages of the drive are mitigated. On the Air, you've got a fast SSD to write to, so the slowdown will be much less.
In the new minis, the hard drives are upgradable and even a base model can take a PCIe SSD in the futurethey did however make it a bit harder to do so. The other alternative is getting a cheap external USB3 external drive and SSDthat way you don't have to crack open the mini, and you could even create your own fusion drive using the two.
The RAM, meanwhile, is soldered and except for those with specialized kits, essentially irreplaceable. So if you only have the money for one upgrade, boost the RAM as much as you can.
A guy at MacRumors put together a fairly decent questionnaire to help you figure out how much RAM you'd need:
http://forums.macrumors.com/showthread.php?t=1756865