Right - depends what you're going for. Bigger track or interesting track?
"The Cross" lets you build interesting tracks with less overall table space needed. If you don't use the rails in the middle, you've got a way of doing 'cross over' tracks too.
"Expansion 1" is all about "the jump" - it's two small ramps & connectors for a bridge over another piece of track. I tend to use it as two separate 'ramps' over a gap or something.
"Expansion 2" has a bunch of 45 degree curves instead of the usual 90, plus more curves + straights.
I like expansion 1 for the jumps - it's always a winner with the kids & a great spot for me to handicap myself with them

Expansion 2 definitely adds the most flexibility. Throwing in the relatively cheap 'the cross' helps with layout space & flexibility IMO. I've ignored the long straights completely.
With the base game, you've got two types of pieces to construct with - a straight piece and a 90 degree turn. An extra base set gives you the capability of
longer tracks, but it's always going to be the same fundamental mix of straights & turns. Throwing in an expansion allows you a bunch of flexibility.
So yeah, depends what you're wanting out of it. I got a big lot all at once off an BGG auction & added the second base set later after it was a hit.