Which particular couch potato one is this? It doesn't sound like Canadian Couch Potato since I don't recall him having any XEF. Is it one of those earlier MoneySense ones? If so, I recall those ones having a ridiculous amount allocated to bonds, which will indeed kill your returns.
Mine is Canadian Couch Potato focused, with a mix of VCN and VXC/XAW, 0% bonds. This is what it looks like so far
Dashed black line is the S&P 500 and the dashed aquamarine-ish line is the MCSI World Index. The other 2 are my TFSA and RRSP. My (and your) index funds aren't indexed to the S&P 500 so I wouldn't expect them to be the same, but they're still tracking fairly closely. This is a cumulative view so you can see that I am fairly handily beating out the SP500.
Maybe you just unluckily had bad timing? What kind of time period are we talking here?
In any case, an SP500 fund would be a bit redundant as all those companies will be included in your other funds.