• Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.

Review Score Bias? A statistical analysis of IGN, Gamespot, and 1UP.

Status
Not open for further replies.
B

beef3483

Member
Wow, so somebody already beat me to the punch. Too bad, I guess. And he even has better stats. Hurry up, somebody kill this thread and put me out of my misery.
 
E

Enk

makes good threads.
Ashcrazyman.gif

Ow my head.
 
B

Bildi

Member
beef3483 said:
If this thread hasn't already devolved into Caturday, I edited for clarity.
Click to expand...
It's a bit easier to read like this:
Code: 
[B]Top 10 WII Games[/B]
[B]Average Difference from the Mean[/B]

IGN) 1.83   Gamespot) 0.63   1UP) -8.07


[B]Top 10 Xbox 360 Games[/B]
[B]Average Difference from the Mean[/B]

IGN) 0.43   Gamespot) -0.23  1UP) 1.21


[B]Top 10 PS3 Games[/B]
[B]Average Difference from the Mean[/B]

IGN) 1.63   Gamespot) -3.67  1UP) 0.77
 
B

beef3483

Member
underfooter said:
I didn't know 1up gave Madden '08 on wii a 4.5. I played 07 and it was fine, is 08' some kind of HUGE donwgrade?
Click to expand...

1up was the most serious offender as far as Wii scores were concerned. For those top 10, they scored them, on average, 8.07 lower than Gamerankings average.
 
Z

ZealousD

Makes world leading predictions like "The sun will rise tomorrow"
underfooter said:
I didn't know 1up gave Madden '08 on wii a 4.5. I played 07 and it was fine, is 08' some kind of HUGE donwgrade?
Click to expand...

They scored 07 really low as well. Something like a 6.0.

The "1Up Bias" doesn't extend to every Wii game, even though the statistics are hugely stacked again them. They score both Madden games, SSX Blur, Trauma Center, Scarface, and Manhunt 2 way below the norm, but most of their other reviews are at least somewhat close to the mark.
 
B

beef3483

Member
ZealousD said:
They scored 07 really low as well. Something like a 6.0.

The big problem with 1UP is that the huge difference in scores is a result of just a couple games. Namely both Madden games, SSX Blur, Trauma Center, and Scarface.
Click to expand...

But they also consistently underrate each individual game as well. Only Zelda and Mercury Meltdown did they score higher than the gamerankings average. That's 2/10.
 
B

Bildi

Member
beef3483 said:
Yes it is, except that I cut off the negative sign before 1UPs Average difference, in the Wii column. That was when I edited.
Click to expand...
Yeah sorry, I noticed that and fixed my post.

8.07 lower is a pretty damn big margin. And it's interesting that Gamespot scores the PS3 so low - I had no idea.
 
D

DDayton

(more a nerd than a geek)
Bildi said:
It's a bit easier to read like this:
Code: 
[B]Top 10 WII Games[/B]
[B]Average Difference from the Mean[/B]

IGN) 1.83   Gamespot) 0.63   1UP) -8.07


[B]Top 10 Xbox 360 Games[/B]
[B]Average Difference from the Mean[/B]

IGN) 0.43   Gamespot) -0.23  1UP) 1.21


[B]Top 10 PS3 Games[/B]
[B]Average Difference from the Mean[/B]

IGN) 1.63   Gamespot) -3.67  1UP) 0.77
Click to expand...

See, that's actually interesting and easy to read. It doesn't really MEAN anything, but at least the numbers are presented in a easily understood manner.

Now, crunch the numbers for a larger range of games, and we might be looking at something...vaguely.
 
B

beef3483

Member
DavidDayton said:
See, that's actually interesting and easy to read. It doesn't really MEAN anything, but at least the numbers are presented in a easily understood manner.

Now, crunch the numbers for a larger range of games, and we might be looking at something...vaguely.
Click to expand...

This guy actually did better than I did:

http://www.neogaf.com/forum/showthread.php?t=202412

He calculated with a larger range and even used more measures of central tendency. He used the top 25 for each system from Metacritic.

If I had known that somebody had already done this(and gotten a similar response), I wouldn't have bothered.
 
J

Jtrizzy

Member
CrushDance said:
So:

1up=Microsoft

IGN=Nintendo

Where's Sony?
Click to expand...

The PS3's software library is surprisingly well liked given the claims made by just about everyone that software is sparse. One of the reasons why those claims are likely still made even when the software is well liked is because there are an awful lot of multiplatform and annual release/sports games relative to the other two consoles.
Click to expand...

That ad is fucking hilarious.
 
K

kassatsu

Banned
Bildi said:
Could you maybe make it more readable? Something easy such as "the average scores are

IGN 9.5
1Up 9.3
Gamespot 7.7"

Or a graph? With nice colours.


Maybe use the code tags. That might help.
Click to expand...
.
 
M

mrklaw

MrArseFace
I only kept reading this thread because I thought people were beign harsh on the OP - whats wrong with trying to see if there are trends in review scores across formats/magazines.

then I saw crushed's posts. I'm in tears here. why are they so fucking funny?

Crushed is my new hero. He's been on song recently but this just tops it off.
 
N

neojubei

Will drop pants for Sony.
Although I do believe there are some bias in all of us including reviewers, this topic cannot prove any of that. I find it best to buy the games you want than harp on reviews.
 
NekoFever

NekoFever

Member
This study is flawed. It doesn't take into account the fact that all the review sites are teh bias against PS3 and Wii games due to Micro$oft moneyhats. Everyone knows this.
 
S

Skiptastic

Member
Mohonky said:
No fucking way......did the marketing team think this one through before putting that shit out?
Click to expand...

It's not real. Read the paragraph below Lickable and it becomes painfully obvious.

PS. Caturday. Serious Business pic plz?

PPS. I want to see IGN Dreamcast vs. the statistical mean. :lol
 
S

Stumpokapow

listen to the mad man
Skiptastic said:
PPS. I want to see IGN Dreamcast vs. the statistical mean. :lol
Click to expand...

MetaCritic Dreamcast Top 25 Average: 90.8
IGN Dreamcast Top 25 Average: 93.32
IGN Average deviation from MetaCritic average: +2.52

Reviews more than 5 points off the MC average:
NBA2k1 - MC 93 - IGN 88
Tennis 2k2 - MC 90 - IGN 96
Samba - MC 89 - IGN 95
Sonic Adventure 2 - MC 89 - IGN 94
Shenmue - MC 89 - IGN 97
Daytona USA - MC 86 - IGN 93
 
S

Skiptastic

Member
NekoFever said:
IGN Game Boy would be more interesting. If you think 1Up throws 10s about like confetti...
Click to expand...

But wasn't IGN GB rounded to the nearest integer? Like, if it was 9.5, they rounded up to 10?

Actually, looking at that data Stumpokapow posted, IGN DC doesn't seem that bad, which is funny seeing as how they have the running joke that they overrated everything.

NBA2K1 should actually have been higher, Samba de Amigo DEFINITELY deserved the 9.5...but the remaining ones were overrated. :(
 
T

TheRagnCajun

Member
IGN most consistent confirmed. Well no this annalysis doesn't confirm anything, but thats been my personal thoughts on the matter for years. I don't care that they score too high, they're consistent and fair.
 
E

esbern

Junior Member
i remember when i was like 13 i wanted to find out what system to buy so i went to the back pages of EGM or Gamefan or something where they kept a log of all the recent reviews. I started recording the scores to find the average, and got bored after about 15 minutes. you guys that care about review scores need a damn hobby.
 
K

kay

Member
6b9iyqa.jpg


IGN confirmed biased compared to IGN





or these are reviews and people have different opinions you should take with a grain of salt.
 
Status
Not open for further replies.

Similar threads

EDGE #393 reviews and GOTY AWARDS
Opinion News Hype 2
77
7K
DenchDeckard replied
whats your top 10 games of 2023 ?
2
68
4K
ManaByte replied
Dexerto's top 100 games of all time
Social Opinion 2 3
125
5K
_Idea_of_Evil_ replied
Famitsu Review Scores: Issue 1835 | Like a Dragon: Infinite Wealth awarded a perfect score
Review
44
3K
Miyazaki’s Slave replied
I don’t understand why more reviewers don’t use a 5 star system
2
50
2K
Holammer replied
Top Bottom