Support NeoGAF

[CosmicQueso]

First time OP. Be gentle.

I'm no scientist, but I like data. I also like having discussions about things using data. So, given the last few days have been a little weird, I wanted to do a deep dive in data to answer a couple questions:

• Are games of higher reviewed quality now than they were during the last gen?
• Which platforms have the games of the highest reviewed quality?
• Have disc based game release counts really gone down since last gen?
• Will games get better (at least in terms of review scores) soon?

So, I did a few things. I went to GameRankings and scrubbed all the data I could for PS4, PS3, Xone, X360, WiiU and the Wii. I averaged the scores if there were multiple versions of the same game, and I ensured that each game had a proper release date. I then time aligned these data, did some basic comparisons with math and stuff, and then made some pretty charts. I only used Disc based games (sorry, left the downloadable stuff out, which is probably both good and bad).

Here's what I found:

• Release Counts are Significantly Lower vs Last Gen – 104 Packaged games were rated across PS4, Xone and Wii U over the first 8 months in market for each platform, down 32% versus the 154 titles rated over the first 8 months each of PS3, X360 and Wii.
• However, the number of 80+ releases has remained flat vs last gen – 35 titles in the current generation have scored an 80 or higher, the same number over the first 8 months of the prior cycle.
• Average Quality has improved slightly – The average PS4/Xone/WiiU game has scored a 71, while during the first 8 months of last cycle the average PS4/X360/Wii game averaged a 69.
• Quality scores should improve in 2015 – 95 titles released in year 3 of the prior cycle scored 80 or higher, while during the first two years only 35 titles reached that benchmark. We should expect more 80+ releases coming in 2015 than we’ve seen to date.

Here are data charts looking at the first 8 months of each cycle (Basically November through June):

Really, too many fun things to count in here, and some surprises. Check out the release counts, the average review scores, and the 26 Wii games rated under 60 that launched.​

And here are some charts highlighting the Current and Last Gen releases in terms of review score:

Way more releases in the prior cycle, with Wii trailing in quality. This gen, however, its the Xbox One titles that are trailing in reviewed quality.

And to the Platform comparisons:

Microsoft:

Sony:

Nintendo:

So, will things get better? Are we going to get better games soon? I think the answer is yes.

I took a look at the titles released in year 3 of each platform's life, and compared to what came out prior.

What becomes clear is that something changes in year 3... developers get better, more releases come out, other "stuff". Whatever the reasons, a marked improvement in quality happened in year 3 of the last cycle. So, if that trend holds, we'll get games that are reviewed more highly next year than we have so far.

Check it out:

Microsoft:

Sony:

Nintendo:

So, to answer my own questions:

• Are games of higher reviewed quality now than they were during the last gen? On average, yes. But the MAX isn't as high as last gen, and the MIN is lower. So, kinda, I guess?
• Which platforms have the games of the highest reviewed quality? The PS4 (over the first 8 months of each platform's life)
• Have disc based game release counts really gone down since last gen? OMG yes. A lot.
• Will games get better (at least in terms of review scores) soon? They should.

Disclaimers:

• I'm not a scientist
• I used GameRankings because it was way easier than trying Metacritic
• I own all 6 Current and Past gen consoles
• The Xbox One is my fave, followed by the Wii U and the PS4, but I love them all equally, like children
• Any errors in the data and math are my fault, let me know where I'm wrong
• I love everyone, all of you, all of gaming, everything is hugs and flowers and let's all be friends
• I subscribe to PS+, XBLA, EA Access, I love competition and think the stronger and more competitive the market is, the better it'll be for everyone

 

[SwiftDeath]

I find it kind of surprising that the Wii U retail only releases seem to have a significantly lower review score average than comparable PS4/XB1 average. There must be some retail shovelware or something for Wii U that's bringing down their generally well received 1st party efforts?

Ah maybe it's simply the sheer number of releases bringing down the average.

 

[CosmicQueso]

I find it kind of surprising that the Wii U retail only releases seem to have a significantly lower review score average than comparable PS4/XB1 average. There must be some retail shovelware or something for Wii U that's bringing down their generally well received 1st party efforts?

Ah maybe it's simply the sheer number of releases bringing down the average.

Well that, and the 13 titles released so far that have rated below 60. At the top end of the scale, however, the Wii U is really solid, and much better than the Wii.

 

[SwiftDeath]

Well that, and the 13 titles released so far that have rated below 60. At the top end of the scale, however, the Wii U is really solid, and much better than the Wii.

What Wii U game got a 16% anyways? Likely brought down the whole group

 

[jschreier]

Interesting data. Kudos on putting in all this work. But one factor you didn't mention is that review websites have generally gotten harsher with scoring over the years. It's much harder for a game to get a 9 or 10 now than it was in 2007 or 2008. (Yet another reason review scores are meaningless and should be abolished forever.)

 

[CosmicQueso]

What Wii U game got a 16% anyways? Likely brought down the whole group

Family Party: 30 Great Games Obstacle Arcade with 5 reviews

Interesting data. Kudos on putting in all this work. But one factor you didn't mention is that review websites have generally gotten harsher with scoring over the years. It's much harder for a game to get a 9 or 10 now than it was in 2007 or 2008. (Yet another reason review scores are meaningless and should be abolished forever.)

Thanks! And while your second point seems to definitely be true now (harder for games to get higher scores now than then), do you think there's any credence to the idea that devs will improve, thereby making better games by 2015 perhaps, which would then lead to higher scores? Seems like in the last gen, scores definitely improved once devs figured out the new boxes a bit?

 

[Weng]

Increasing scores later in the generation don't mean that the games that get released are actually (factually? objectively? subjectively?) better than launch titles. Reviewers could just be convinced a game that releases later has to be somehow better and therefore tend to give higher scores and also could be more cautious giving higher scores to launch titles because they assume the games didn't exhaust the platforms potential.

While it is true that later games often have "better graphics" than launch titles, could a mere improvement of graphics make up the difference in scores alone?

How would the games get rated if they were all reviewed after the end of the consoles lifespan and without the reviewer knowing when they released?

 

[CosmicQueso]

Increasing scores later in the generation don't mean that the games that get released are actually (factually? objectively? subjectively?) better than launch titles.

Sure, review scores are just aggregated data points of quantitative values assigned to qualitative things. They are flawed and imperfect to be sure.

Of all of the mess I looked at, the sharp reduction in release count is what was most surprising/concerning.

You go into NPD threads and it's a cluster of CONSOLE IS DOOMED. This might be the case, but it would be from dev/pubs abandoning the platforms, not from consumer demand for content on those systems.

 

[LOCK]

I wish we had the numbers to determine if sales averages are higher now due to lower releases. In other words are the games still selling average, better, or lower across the board.

 

[lightning2k3]

Great work. The wii really had a crap ton of shovelware at launch...

 

[Blanquito]

It's interesting information, and does provide lots of value, but at the same time I think it needs to be emphasized even more that this is disk-only titles.

Considering that we even have big publishers like Ubisoft releasing digital-only titles (such as Child of Light and Valiant Hearts) in addition to all the indie games being released for consoles, it may not provide as much useful information as originally assumed. (and I realize that including those titles would mean more work for you, so I'm not implying that you need to do it.)

Especially since Sony has gone on record as stating that they foresaw the shortage of game releases in the first year and really pushed to get more indies games on PS4 (which generally do not have disk based releases).

I mean, even just looking at Metacritic top PS4 games I see roughly ~17 games that have a score higher than 80 and are digital only releases. That's not an insignificant number.

Despite all of what I've said, I do really appreciate the work. I love seeing the data and the charts. Thanks CosmicCheese!

 

[maxcriden]

Fascinating OP, thanks for all of your hard work. I definitely agree with Jason that review scores have become unquestionably harsher.

 

[Ishan]

since people in the other thread are discussing this already and this has a lot of data. Hmm interesting pointabout reviews getting harsher also tho. But that would also mean games have gotten significantly better as they are already better on average scores in a similar time frame.

 

[Carlisle]

Awesome OP. Great work.

I find it kind of surprising that the Wii U retail only releases seem to have a significantly lower review score average than comparable PS4/XB1 average. There must be some retail shovelware or something for Wii U that's bringing down their generally well received 1st party efforts?

Ah maybe it's simply the sheer number of releases bringing down the average.

I thought it was odd too, so I did a cursory check on Metacritic, counting the games for each platform that scored in the Green (75 and above).

Wii U: 58
PS4: 57
Xone: 28

So while the Wii U has more crap on the system dragging its average down, it also has a comparable amount of high scoring titles to the PS4.