The damage multiplier exists in every mode except straight TvT, but it still takes a while to kick-in for Swap mode since the two boards seem to have independent timers. (Standard Puyo matches go on for 96 seconds before margin time kicks in; Swap mode's timer appears to be roughly double that.) So you're still looking at ~3 minutes before scrub chains start to become threatening, whereas a normal fighting game would have already reached time-over in ~90 seconds for a multi-round game or ~180 seconds for a single-round game like Marvel or Injustice. It's still strictly better than just TvT, but the long timer and the possibility of using one of your two boards to tank garbage mean that matches can still drag on for a while at certain skill levels.
The best way to deal with it is to just make sure that seeding is as thorough as possible.