If you are consistently having lag, it's probably your connection. I assume most of the people here are smooth for the majority of the time. I play every day, at different times of day, always four-player with randoms "for fun" and maybe one out of ten matches have any sort of lag, and its never unplayable.
I have just the average cable modem connection here. Nothing fancy. I have no suggestions other than definitely look closely at your router. Almost all of the routers out there are junk: linksys, dlink, netgear, etc
my friends and I have tried them over the years and they all are bad in one aspect or another. Update your firmware and hope for the best.