Here is a link to a graph of the percentages of people who voted in midterms and presidential elections:
http://www.fairvote.org/voter_turnout#voter_turnout_101
It makes more sense to look at percentages than absolute numbers of voters because the U.S. population continues to increase.
I can't embed the graph directly, but it shows that turnout for Presidential elections bounces around between about 50% and 65% in the post-war period, with a low of 51.7% in 1996 and a high of 63.8% in 1960. In general, it appears that turnout in Presidential elections stayed relatively high during the 1950's and 1960's, and then fell after 1968. Turnout remained below 60% during the 1970's, 1980's, and 1990's, until it rose to 60% in 2004 (from 54% in 2000) and then to 61.6% in 2008. After that, turnout dropped to 58.2% in 2012, and if I were a betting man, I would wager that turnout will drop further in 2016.
Turnout for midterms follows a similar pattern to turnout for Presidential elections, just reduced by around 20 percent.
Whether those changes over time are significant I guess is debatable. I would note, however, that periods of relatively high turnout in the post-war period seem to coincide with organization around and advancement of interests usually associated with the left. Periods of relatively lower turnout seem to coincide with periods of right wing dominance and left wing disorganization. For example, its easy to see where the New Deal coalition broke up in the aftermath of the disastrous Chicago 68 convention.
Edit: I don't think Democrats primarying Obama would have been a good idea of would have solved this problem. It is a problem, though. I don't recall primarying Obama ever being seriously discussed, though - just the usual griping,