In that sense, sure, but "go around and do whatever" is ultimately something I consider to be moreso free-roaming. "Sandbox" is a pretty wide-ranging word and there's no one definition of it. I personally think the "do whatever you want" part is more powerful and meaningful when you can do things the game never expected or intended you to do, and that's what I think a sandbox game is. GTA3 (especially San Andreas) was filled with that kind of stuff. GTA4 dropped the ball when it came to that (though it did do other things much better).
I was never remotely trying to insult Mafia by saying that I don't think it's a sandbox game, by the way. I just don't think it has all that much in common with GTA and its clones beyond a few broad generalizations. I think it was a much better game, too.