If J Stars Victory VS was made with all the rivalries and pairings people would expect... it'd have an incredibly boring cast. It'd basically all be Goku & Vegita repeated with different skins over and over again!
Votes, Seiyuu availability / cross character voicework, and cross-series similarities between characters should all matter into the roster picks.
... and in a vain attempt to pretend this comment has a purpose being in this thread, I'd love to see a T(oonami)-Stars Victory VS.
Roger Smith VS Sym-Bionic Titan VS Gundam Wing VS Hamtaro VS Kirito, GO! This is also the only way a Gene Starwind VS Space Dandy VS Vash VS Spike battle would ever actually occur. Thundercats VS Tenchi? Samurai Jack VS Kenshin?
Of course, LORD SESSHOMARU would be the one true winner. Opening Narrated by "The Big Guy", Closing theme
Hiro No Tsuki, punctuated by a *BANG*.