Yes, absolutely, the ONE fan that you DO want connected is your CPU cooler fan, it would be the most important BY FAR. You do not want to remove or unpower the fan connected to the CPU heat sink, as even though you don't OC, CPU's are still expected to have heat drawn from the heat sink for efficient heat exchange, as your CPU even at stock clocks is going to get pretty hot. You have two fans in the front and one in the back with that case I believe, so if you have to disconnect a fan it would be one of the fans in the front.
The case itself is designed for efficient air flow, with air moving over the motherboard drawing heat from the components. I would suggest, if you have do disconnect one, that it be the fan at the top, as the fan at the bottom is going to be pulling air across your GPU.
I'm going to assume when you said you connected the third fan to the PSU you connected it to the four pin molex floppy power plug. This will cause that fan to move at max RPM the entire time which is not all that bad. It makes sense that the fan does not make this sound all the time, as bad (loose) bearings would cause the fan to rattle only part of the time if it's not sever, as it shakes itself in and out of smooth movement.
Two things though, make sure the fans are secured tight to the chassis, as your sound might be a micro rattle from a loose connection, and do as I suggested earlier and feel you HD mount for vibration, the one thing you want to make sure of is that the sound is not coming from your HD. I would suggest running the machine with the cover off for a while to try to locate where it is coming from, as in the front or the back of the case.