Since there is so much bickering about console bias. I figured I would do an analysis to try and see if there is any truth to peoples preconcieved notions. I chose IGN, Gamespot, and 1UP because thier reviews seem to be the most discussed.
Usually when comparing game review scores to assess a site bias, they only compare one score to another. I knew this was a faulty way. If there is truly a bias, you have to find and assess patterns which requires looking at more than one games score compared to another games score. So I set rules for a system, and then I compiled data.
How it works is that I've have taken the top ten games according to Gamerankings, of each system, with a minimum of fifteen reviews. I then took the score from the three sites and calculated the difference from Gamerankings mean (average). At the end, I calculated the average distance from the mean for those top 10 games.
It's not perfect, but I set it up in a way that is not preferential to any system. The reason I chose the top ten games for each system was because these are the games most talked about and discussed and also because, according to a great many reviewers, these are the sytems best games. So it's not a comparison for all the games in each consoles library, just the best games, according to Gamerankings.
I chose Gamerankings because it is one of the only sites to have a large enough compiled database of scores. Also, bias implies a deviation from others opinions, so it was necessary to use Game rankings to draw comparisons.
I chose a minimum of fifteen reviews for each game just because that seemed about right. This also means that games like Super Mario Galaxy, Zac and Wiki, Virtua Fighter 5 (Xbox 360), Mass Effect, and Uncharted won't be included. Obviously, this is because they do not yet have enough reviews.
I would also encourage people to use this as a way to find the best games for each system.
The scores from left to right read:
1) Gamerankings(GR) 2)IGN 3) IGNs difference from GR 4) Gamespot 5) Gamespots difference from GR 6) 1UP 7) 1ups difference from GR
Top 10 WII Games
Zelda: Twilight Princess
1) 94.2 2) 95 3) 0.8 4) 88 5)-6.2 6)100 7)5.8
Resident Evil: Wii Edition
1)91.2 2) 90 3) -1.2 4) 91 5) -0.2 6) 80 7) -11.2
Metroid Prime Corruption
1)90.5 2) 95 3) 4.5 4) 85 5) -5 6) 90 7) -0.5
Super Paper Mario
1) 85.3 2) 89 3) 3.7 4) 88 5) 2.7 6) 80 7) -5.3
Madden NFL 07
1) 81.8 2) 85 3) 3.2 4) 84 5) 2.2 7)65 7)-16.8
WarioWare: Smooth Moves
1)81.7 2) 82 3) 0.3 4) 91 5) 9.3 6) 80 7)-1.7
Trauma Center
1)80.5 2)80 3) -0.5 4) 80 5) -0.5 6) 65 7)-15.5
Mario Strikers Charged
1) 79.5 2) 83 3) 3.5 4) 75 5) -4.5 6) 75 7) -4.5
Madden NFL 08
1) 78.2 2) 85 3) 6.8 4) 80 5) 1.8 6) 45 7) -33.2
Mercury Meltdown Rev
1) 77.8 2) 75 3)- 2.8 4)85 5) 7.2 6) 80 7) 2.2
Average Difference from the Mean
IGN) 1.83 Gamespot) 0.63 1UP)-8.07
Top 10 Xbox 360 Games
From left to right it reads:
1) Gamerankings(GR) 2)IGN 3) IGNs difference from GR 4) Gamespot 5) Gamespots difference from GR 6) 1UP 7) 1ups difference from GR
The Orange Box
1) 96.1 2) 95 3) -1.1 4)95 5) -1.1 6) 100 7) 3.9
Bioshock
1) 95.3 2) 97 3) 1.7 4) 90 5) -5.3 6) 100 7) 4.7
The Elder Scrolls VI
1) 94.1 2) 93 3)-1.1 4) 96 5) 1.9 6) 91 7) -3.1
Gears of War
1) 93.8 2) 94 3) 0.2 4) 96 5) 2.2 6) 100 7) 6.2
Halo 3
1) 93.3 2) 95 3) 1.7 4) 95 5) 1.7 6) 100 7) 6.7
Guitar Hero II
1) 92.1 2) 94 3) 1.9 4) 89 5) -3.1 6) 90 7)-2.1
Tom Clancy's Ghost Recon AW
1) 90.6 2) 92 3) 1.4 4) 92 5) 1.4 6) 90 7)-0.6
Call of Duty 2
1) 90 2) 90 3) 0 4) 88 5) -2 6) 90 7) 0
Forza Motorsport 2
1) 89.8 2) 89 3) -0.8 4) 92 5) 2.2 6) 85 7) -4.8
Average Difference from the Mean
IGN) 0.433333333 Gamespot) -0.233333333 1UP) 1.211111111
Top 10 PS3 Games
From left to right it reads:
1) Gamerankings(GR) 2)IGN 3) IGNs difference from GR 4) Gamespot 5) Gamespots difference from GR 6) 1UP 7) 1ups difference from GR
The Elder Scrolls VI
1) 93 2) 92 3)-1 4)95 5) 2 6) 95 7)2
Ratchet & Clank Future
1) 90.7 2) 94 3) 3.3 4) 75 5) -15.7 6) 85 7) -5.7
Tom Clancy's Ghost Recon AW2
1) 87.5 2) 83 3) -4.5 4) 85 5) -2.5 6)80 7) -7.5
Resistance: Fall of Man
1) 87.3 2)91 3) 3.7 4) 86 5) -1.3 6)85 7)-2.3
Virtua Fighter 5
1) 86.9 2) 88 3)1.1 4) 81 5) -5.9 6)95 7)8.1
Ninja Gaiden Sigma
1)86.8 2)93 3)6.2 4) 90 5) 3.2 6) 90 7)3.2
Skate
1)86.4 2)88 3)1.6 4)70 5)-16.4 6)90 7)3.6
NHL 08
1)85.7 2)86 3)0.3 4)85 5)-0.7 6)No Review
Tom Clancy's Rainbow Six Vegas
1)85.5 2)87 3)1.5 4)90 5)4.5 6)85 7)-0.5
NBA Street Homecourt
1)83.9 2)88 3) 4.1 4)80 5)-3.9 6) 90 7) 6.1
Average Difference from the Mean
IGN) 1.63 Gamespot) -3.67 1UP) 0.777777778
To sum it up:
Usually when comparing game review scores to assess a site bias, they only compare one score to another. I knew this was a faulty way. If there is truly a bias, you have to find and assess patterns which requires looking at more than one games score compared to another games score. So I set rules for a system, and then I compiled data.
How it works is that I've have taken the top ten games according to Gamerankings, of each system, with a minimum of fifteen reviews. I then took the score from the three sites and calculated the difference from Gamerankings mean (average). At the end, I calculated the average distance from the mean for those top 10 games.
It's not perfect, but I set it up in a way that is not preferential to any system. The reason I chose the top ten games for each system was because these are the games most talked about and discussed and also because, according to a great many reviewers, these are the sytems best games. So it's not a comparison for all the games in each consoles library, just the best games, according to Gamerankings.
I chose Gamerankings because it is one of the only sites to have a large enough compiled database of scores. Also, bias implies a deviation from others opinions, so it was necessary to use Game rankings to draw comparisons.
I chose a minimum of fifteen reviews for each game just because that seemed about right. This also means that games like Super Mario Galaxy, Zac and Wiki, Virtua Fighter 5 (Xbox 360), Mass Effect, and Uncharted won't be included. Obviously, this is because they do not yet have enough reviews.
I would also encourage people to use this as a way to find the best games for each system.
The scores from left to right read:
1) Gamerankings(GR) 2)IGN 3) IGNs difference from GR 4) Gamespot 5) Gamespots difference from GR 6) 1UP 7) 1ups difference from GR
Top 10 WII Games
Zelda: Twilight Princess
1) 94.2 2) 95 3) 0.8 4) 88 5)-6.2 6)100 7)5.8
Resident Evil: Wii Edition
1)91.2 2) 90 3) -1.2 4) 91 5) -0.2 6) 80 7) -11.2
Metroid Prime Corruption
1)90.5 2) 95 3) 4.5 4) 85 5) -5 6) 90 7) -0.5
Super Paper Mario
1) 85.3 2) 89 3) 3.7 4) 88 5) 2.7 6) 80 7) -5.3
Madden NFL 07
1) 81.8 2) 85 3) 3.2 4) 84 5) 2.2 7)65 7)-16.8
WarioWare: Smooth Moves
1)81.7 2) 82 3) 0.3 4) 91 5) 9.3 6) 80 7)-1.7
Trauma Center
1)80.5 2)80 3) -0.5 4) 80 5) -0.5 6) 65 7)-15.5
Mario Strikers Charged
1) 79.5 2) 83 3) 3.5 4) 75 5) -4.5 6) 75 7) -4.5
Madden NFL 08
1) 78.2 2) 85 3) 6.8 4) 80 5) 1.8 6) 45 7) -33.2
Mercury Meltdown Rev
1) 77.8 2) 75 3)- 2.8 4)85 5) 7.2 6) 80 7) 2.2
Average Difference from the Mean
IGN) 1.83 Gamespot) 0.63 1UP)-8.07
Top 10 Xbox 360 Games
From left to right it reads:
1) Gamerankings(GR) 2)IGN 3) IGNs difference from GR 4) Gamespot 5) Gamespots difference from GR 6) 1UP 7) 1ups difference from GR
The Orange Box
1) 96.1 2) 95 3) -1.1 4)95 5) -1.1 6) 100 7) 3.9
Bioshock
1) 95.3 2) 97 3) 1.7 4) 90 5) -5.3 6) 100 7) 4.7
The Elder Scrolls VI
1) 94.1 2) 93 3)-1.1 4) 96 5) 1.9 6) 91 7) -3.1
Gears of War
1) 93.8 2) 94 3) 0.2 4) 96 5) 2.2 6) 100 7) 6.2
Halo 3
1) 93.3 2) 95 3) 1.7 4) 95 5) 1.7 6) 100 7) 6.7
Guitar Hero II
1) 92.1 2) 94 3) 1.9 4) 89 5) -3.1 6) 90 7)-2.1
Tom Clancy's Ghost Recon AW
1) 90.6 2) 92 3) 1.4 4) 92 5) 1.4 6) 90 7)-0.6
Call of Duty 2
1) 90 2) 90 3) 0 4) 88 5) -2 6) 90 7) 0
Forza Motorsport 2
1) 89.8 2) 89 3) -0.8 4) 92 5) 2.2 6) 85 7) -4.8
Average Difference from the Mean
IGN) 0.433333333 Gamespot) -0.233333333 1UP) 1.211111111
Top 10 PS3 Games
From left to right it reads:
1) Gamerankings(GR) 2)IGN 3) IGNs difference from GR 4) Gamespot 5) Gamespots difference from GR 6) 1UP 7) 1ups difference from GR
The Elder Scrolls VI
1) 93 2) 92 3)-1 4)95 5) 2 6) 95 7)2
Ratchet & Clank Future
1) 90.7 2) 94 3) 3.3 4) 75 5) -15.7 6) 85 7) -5.7
Tom Clancy's Ghost Recon AW2
1) 87.5 2) 83 3) -4.5 4) 85 5) -2.5 6)80 7) -7.5
Resistance: Fall of Man
1) 87.3 2)91 3) 3.7 4) 86 5) -1.3 6)85 7)-2.3
Virtua Fighter 5
1) 86.9 2) 88 3)1.1 4) 81 5) -5.9 6)95 7)8.1
Ninja Gaiden Sigma
1)86.8 2)93 3)6.2 4) 90 5) 3.2 6) 90 7)3.2
Skate
1)86.4 2)88 3)1.6 4)70 5)-16.4 6)90 7)3.6
NHL 08
1)85.7 2)86 3)0.3 4)85 5)-0.7 6)No Review
Tom Clancy's Rainbow Six Vegas
1)85.5 2)87 3)1.5 4)90 5)4.5 6)85 7)-0.5
NBA Street Homecourt
1)83.9 2)88 3) 4.1 4)80 5)-3.9 6) 90 7) 6.1
Average Difference from the Mean
IGN) 1.63 Gamespot) -3.67 1UP) 0.777777778
To sum it up:
Code:
[B]Top 10 WII Games[/B]
[B]Average Difference from the Mean[/B]
IGN) 1.83 Gamespot) 0.63 1UP) -8.07
[B]Top 10 Xbox 360 Games[/B]
[B]Average Difference from the Mean[/B]
IGN) 0.43 Gamespot) -0.23 1UP) 1.21
[B]Top 10 PS3 Games[/B]
[B]Average Difference from the Mean[/B]
IGN) 1.63 Gamespot) -3.67 1UP) 0.77
