Support NeoGAF

[Mononoke]

Imagine that 500 console games came out per year. How many can you play in a year? Maybe 25 or 30 if you play a ton of games, maybe 50-75 if you basically do nothing other than play games, right?

It would stand to reason that in your process of selecting games, you'd apply some sort of sorting measure for choosing one over the other. Probably this sorting measure is your own level of interest. For many people, this closely tracks with the level of polish--they don't want to put up with rough edges, they just want really "amazing" "experiences". Personally I like flawed gems, I like trying different kinds of experiences. But for a lot of people, they really want that blockbuster experience with games, just like with movies.

So if you imagine that in this reality, 50 amazing games came out every year (the top 10% of what's released), it's pretty unlikely you'd be playing the stuff that comes out as "pretty good, not a waste of your time, but there are better options out there", right? Now recognize that the vast majority of gamers play 5-10 console games a year at most.

It's okay if maybe your tastes are a little more narrow, which leads you to deep-diving genres you like while ignoring high quality games in genres you don't like, or if you have only a few consoles and are stuck to what's available on the hardware. It's also okay if you play so many games that you like trying all sorts of things. But for the vast majority of people, writing off games with average scores in the 7s is in fact a good strategy to maximize their gaming time and money value.



Because it's not a problem for the vast majority of readers or the people posting the threads. It is a problem for the people complaining, who have an easy solution to solve the problem. Likewise, if you don't like Metacritic, ignore the Metascore. If your experience is that it tracks consensus very well and that's useful to you, use it. This is not a difficult problem.

Well said. However, aren't critics supposed to be judging games on an artistic level? Comparing them to their contemporaries, analyzing and putting context to them culturally?

I feel like the games industry's critics are largely driven by the consumer aspect (what is a good value for the gamers money), whereas other mediums are more academic based in their criticisms. I dunno. I just feel like reviews should be more than putting a value to the game (time/money), and should focus more on how the game stacks up to other games in its genre, and how it works on an artistic level.

 

[DeaviL]

Misuse and misreading of the score system, combined with no one ever reading the full text reviews.
Also, reviewers being humans who can have biases.

I like my reviews with:
- Positives
- Negatives
- To be considered.

Rather than a score.

What dev was it that missed their bonus for getting 84 not 85 on metacritic?

Fallout New Vegas / Obsidian

 

[Sanke__]

Just think of it as an A-F scale
because thats they way it is
Its not unusual at all

And a 10/10 does not have to mean a game is literally perfect/ the single best game ever

 

[tokkun]

A string of 1000 random numbers has no worth, but reducing it down to a single digit is useless and offensive. This is my point.

If a score needs text to justify it, then the score is not sufficient, whereas text can exist without a score alone. Therefore, the text is superior in all aspects.

No, the text is inferior in one very critical aspect: ease of consumption.

A review without a score is a bit like a newspaper article without a headline. Sure, someone who only reads the headlines will not get the same level of nuance as the person who reads the newspaper cover-to-cover. However there are a lot of people who do not have enough time or impetus to read the full article. For them, being able to skim the headlines in 30 seconds is a lot better than nothing.

Sometimes less is more.

 

[Principate]

Because reviews are now part of most videogames marketing , you just need to see them as adds.

Long boring adds btw.

That's the same with movie reviews and many forms of entertainment. Humans like to use a comparison tool even if it's not a particularly accurate representation.

I think the only real issue with the review scale is that your more likely to book reviews and movie reviews use more of the scale than game reviews. I'm not going o say generally as whole games journalists are less professional etc than these journalists (but chances are you get a higher skew in academic persons due to the respect of the medium as seen in every profession), but I will say I'd prefer they use the whole scale more.

It makes it easier compare and catalogue scores than if everything is in the same bracket with only slight difference.

 

[TheRedSnifit]

Just think of it as an A-F scale
because thats they way it is
Its not unusual at all

And a 10/10 does not have to mean a game is literally perfect/ the single best game ever

I don't get this, though. I mean, movie reviews usually follow a scale where 5 is average, not 7. Why is an A-F scale exclusive to video games?

 

[VindicatorZ]

I tend to want every game I play to be amazing. So if I give a game a 7, it means there's some disappointment there. If I walk out of a movie and give a movie a 7, it means the movie was decent but has flaws. So to me, usually anything below an 8 is a disappointment. Doesn't mean they aren't worth playing, but they aren't incredible either. And who cares about the difference between a 4 and 6. Maybe the best way is a simple Thumbs Up or Down, who knows.

 

[hardcastlemccormick]

No, the text is inferior in one very critical aspect: ease of consumption.

A review without a score is a bit like a newspaper article without a headline. Sure, someone who only reads the headlines will not get the same level of nuance as the person who reads the newspaper cover-to-cover. However there are a lot of people who do not have enough time or impetus to read the full article. For them, being able to skim the headlines in 30 seconds is a lot better than nothing.

Sometimes less is more.

One character is beyond the point at which that trade-off between ease of consumption and communicated information becomes well out of whack.

 

[Ataru]

Your game got a 8 lol floooooop

Some rather beloved games have metacritic scores less than 85, including Mario Kart 64, Ogre Battle 64, Baten Kaitos, Skies of Arcadia, Xenogears, Final Fantasy Tactics, Suikoden II, and Lunar Complete.

 

[Shalashaska]

Well since it hasn't been posted yet:

Maybe it's inflation related.

 

[AtomicShroom]

Miss back when reviews used to cover technical aspects of games...

Nowadays reviews really never talk about technical shortcomings like input lag, stuttery framerates, lack of v-sync, etc... it's like today's game reviewers are oblivious to this.

 

[Chainsawkitten]

Well since it hasn't been posted yet:

Maybe it's inflation related.

So it's actually just that we're using a logarithmic scale.

 

[Principate]

Well said. However, aren't critics supposed to be judging games on an artistic level? Comparing them to their contemporaries, analyzing and putting context to them culturally?

I feel like the games industry's critics are largely driven by the consumer aspect (what is a good value for the gamers money), whereas other mediums are more academic based in their criticisms. I dunno. I just feel like reviews should be more than putting a value to the game (time/money), and should focus more on how the game stacks up to other games in its genre, and how it works on an artistic level.

Games aren't the same as books and movies. So you can't only apply the same methods of evaluations. Games are at their very nature interactive experiences. This changes them completely from a traditional book or a movie.

That's why I have an issue with simply comparing the artistic merits of games. Visual style is important presentation of events is important narrative is important but how you choose to play a game is something entirely different to these things. There is a back and forth in games that isn't possible in a movie or a book.

It would be like if you did an analyses of chess. The story is non existent, the presentation plain but clear, but there's something about it that's kept people playing it for years. It's a response factor that doesn't exist generally in the other mediums. A game doesn't need a story or a outstanding visual presentation to be an effective game. It just needs to challenge your thought process in some manner.

That's why a scientific and mathematical approach to the review of a game is just as valid as a literary approach.

 

[Nibiru]

Ign and the like give out so many 9's and 10's it's skewed the whole grading system.

 

[hawk2025]

Use this formula, and plug in the review score as X:


10* (x - 6) / 4



You're welcome!

 

[Sanke__]

I don't get this, though. I mean, movie reviews usually follow a scale where 5 is average, not 7. Why is an A-F scale exclusive to video games?

I would say the normal scale for movies is 1-5 and yes they do use the whole scale

I would say video games are rated differently because the ratings are so heavily based on technical aspects
More like a test in school i guess

 

[tokkun]

One character is beyond the point at which that trade-off between ease of consumption and communicated information becomes well out of whack.

When I go to look up the current weather, I don't need a 10 page meteorological report. Sometimes I just need a rough idea of whether or not I should take a jacket.

I think you are making too many assumptions about how other people use reviews. If there are people for whom a number is all the information they want, then what's the problem? The full text is still there for those who want it.

Sometimes even single digit scores from multiple reviews is more information than I want. That's why I'm glad Metacritic exists.

 

[Rezae]

The 5 star system seems the best IMO.

Most good to great games are 4 stars, with the rare GOTY caliber game getting a 5 star. Most average games would be 3, stinkers are 2, trash is 1.


In a perfect world free of click-bait, no "score" would be assigned and simply read the content of the review, with perhaps some + and - bullet points at the end.

 

[Ravidrath]

Ha ha - I swear hearing GameFan anecdotes as an adult is like discovering Santa Claus isn't real over and over again.

Haha.

Halverson would make a pretty good Santa Claus, now that you mention it...

But, yeah... having roomed with a Play editor and sharing a cube with a Gamefan editor, I've heard a lot. Nothing would surprise me any more. And I can't imagine either comes close to the insanity of the real thing.

The 5 star system seems the best IMO.

I think this is widely regarded in the gaming media as the best system, but industry PR won't let them do it because there isn't enough granularity for them to get good Metacritic scores.

For example, one outlet I used to write for switched to a five star system. PR balked and complained, and so they added a 4.5 "Editor's Choice" thing, so PR could have their their 90%-equivalent score.

 

[Azull]

Its one of the things that I believe kotaku got right. The yes, no, or maybe system is so much more preferable since I believe that it forces the person to read why its a yes, no, or maybe. Well I hope it does at least .

 

[Eusis]

I mean the 6-10 scale. It is such a bizarre scale. Actual percentages don't seem to correlate properly? A 7/8 is mediocre?

Perception that people have and the fact it really IS a huge deal if some game with a massive hype machine ends up mainly just getting 7s and 8s rather than, say, 8s and 9s and a few 10s. A lot of pretty good games get 7s and 8s attached to them, some even go with a lot of 6s and 7s, it's just a game like Watch Dogs was mega hyped by Ubisoft so if it comes out a little flat it can be seen as a big deal, even though outside of that context it'd probably be considered a very good game, and without that hype machine a very pleasant surprise.

Though some people ARE idiots and will see a game that wasn't built up to be a company's flagship title or is an indie title get 7s and 8s and go "WTF IS THIS SHIT NOT BUYING."

Ha ha - I swear hearing GameFan anecdotes as an adult is like discovering Santa Claus isn't real over and over again.

I remember when that was well regarded too. In hindsight I think all those people were younger, and were just happy to have a magazine that more seriously covered niche stuff. Or because everyone got great reviews they liked that their favorite unknown games got great reviews, whatever.

Its one of the things that I believe kotaku got right. The yes, no, or maybe system is so much more preferable since I believe that it forces the person to read why its a yes, no, or maybe. Well I hope it does at least .

To be honest I tend to find the final paragraph is usually the only part of a review I really need. I just won't get in depth mechanics explanations without playing the game, or want to be spoiled even on dumb minor things, whatever. If I'm really interested either in the game and am on the fense or just the review I'm more likely to read it all.

 

[Willectro]

I could really argue either way about reviews. When I think back to when I was younger and could only rent (or very rarely buy) 1 game for a weekend (3 days) every few weeks after hours of chores only to get stuck with a turd, I'm glad the reviews are around.

But the industry has become much more complicated and I find that reviews often struggle to covey their intent and whether or not they are being objective when readers (and metacritic) are focusing on scores and single statements. I generally follow a few, usually like-minded, writers and only check out metacritic for a comparison. I find some writers are very knowledgeable about a specific series or genre and therefore can't review from the mass-market perspective, but conversely others review typically "hardcore" games and complain about complexity, difficulty, and accessibility. But ultimately reviews are (hopefully) opinions.

I really enjoy Rock Paper Shotgun's "Wot I Think" articles. They are witty and I often agree with their opinions. I give them 95% out of 5 stars.

 

[Principate]

Use this formula, and plug in the review score as X:


10* (x - 6) / 4



You're welcome!

It's a pretty good model of the modern game review but I still feel there needs to be an offset for a score of 10 in the review scale in this equation.

 

[PandaPedinte]

Read the review, ignore the score. Magically unbroken.

I've been doing this for quite some time, reading what the reviewer liked and disliked about a game is a lot better as a insight than deciding to buy a game based only an arbitrary number, and lets be honest some people don't even take their time to read the review just skim through it to see the score it received.

 

[PseudoViper]

money and mindless(robot) readers.

 

[MattKeil]

I don't know that it did "start" with this. I definitely remember magazines doing this before the "Official" era (e.g. Amstrad Action, CVG, Sega Power).

I honestly have no idea what causes it. It seems like some kind of psychological barrier that prevents you from accepting that 5/10 is average. It just seems so low.

It's based on grading in school. 50% or less is considered failing. So me terrible game that fails to work would be a 5 or under. A "B " level game is a 7-7.9. An "A" game is 80-89 and an "A+" game is 90-100. It's just the way we were graded in school.

It's this, pure and simple. Goes way back before anyone even thought of accusing game reviewers of being bribed or paid off.

 

[old]

Want to continue to get free copies for reviews? Want to continue getting invited to behind-closed-door showings and junkets? Want to continue receiving ad money to keep the site running? They know what you want and you know what they want.

The beauty of it is nothing can be proved. It's all unspoken with winks and nods. The publisher never said you have to give a game an 8 or better. They just coincidentally didn't invite site X (who gave their last game a 6) to the junket to review the next game. You can't definitively prove why one way or the other.

 

[Orayn]

Want to continue to get free copies for reviews? Want to continue getting invited to behind-closed-door showings and junkets? Want to continue receiving ad money to keep the site running? They know what you want and you know what they want.

The beauty of it is nothing can be proved. It's all unspoken with winks and nods. The publisher never said you have to give a game an 8 or better. They just coincidentally didn't invite site X (who gave their last game a 6) to the junket to review the next game. You can't definitively prove why one way or the other.

Pretty much this. Publishers have the entire system rigged, everyone is at their mercy and nobody is willing to admit it. The lack of transparency makes it impossible to put any level of trust in game reviews.

I would go so far as to say that game reviews are completely non-indicative of the user experience. Depending on publisher relations, something that gets a 1/10 could easily be a better than than a "11/10 GOTY MASTERPIECE OF THE FOREVER" because it's impossible to how far pubs will go to ensure good PR for their multi-million dollar AAA projects and there's no reason to believe that game journos won't play along every step of the way.

 

[PK Gaming]

It's based on the American school system. A C, which is called average but is really kind of bad, is a 70-79.

Seriously?

So are Bs 80s, and As 90s?

 

[DaBuddaDa]

Seriously?

So are Bs 80s, and As 90s?

The most common are:

90-100 A
80-90 B
70-80 C
60-70 D
60 below F

or slightly more strict:

93-100 A
85-93 B
78-85 C
70-78 D
Below 70 F

 

[amar212]

Pretty much this. Publishers have the entire system rigged, everyone is at their mercy and nobody is willing to admit it. The lack of transparency makes it impossible to put any level of trust in game reviews.

Bribery isn't the guilt of the one which offers.

It is guilt of the one which accepts.

As I said many times before, blame major online media outlets, not publishers.

 

[robotzombie]

I always hate this argument. I don't understand why people expect the entire scale to be evenly weighted in terms of where reviews fall.

Here's the deal: Videogames aren't randomly created in a vacuum. People intentionally create videogames with the intention that they are good. People try to make good video games. For the most part, this will usually lead to games having some semblance of fun involved as that is the intention of their creation. Now, while of course that doesn't automatically mean that a game is going to be anywhere near good, the general trend is going to weigh heavily towards the good side of the scale than the bad. There's no one on the other side of the equation either; i.e. intentionally trying to make shitty games (insert your dumb sarcastic/anecdotal counterpoint here).

Everyone says, "why don't they use the whole scale!" because they don't often see less than a 6-7 review. But I'd suggest that they don't often see that because there aren't all that many cases that warrant much lower than a 6. The vast majority of games are at least "decent" because the vast majority of games aim to be excellent. People intend to create great stuff; it may not always work out, but it does weigh more heavily to the positive side.

Again, if video games were created in some chaotic random fashion, then yes, we should be seeing a lot more 1s and 2s and 3s, but overall, games are generally not bad.

 

[Orayn]

Bribery isn't the guilt of the one which offers.

It is guilt of the one which accepts.

As I said many times before, blame major online media outlets, not publishers.

I blame everyone: The publishers for creating this unhealthy environment, and the press for participating and ensuring that it'll never go away.

 

[mcfishstick]

Well since it hasn't been posted yet:

Maybe it's inflation related.

This is the problem with number scores, people have just got it into their head that 10/10 is great and anything below is clickbaiting etc.

The second problem with reviews is very few actually mention mechanics and graphics in a meaningful way. They'll mention clunky controls or beautiful graphics but rarely ever analyse a game's mechanics in comparison to others in the genre in a meaningful manner.

 

[Azull]

To be honest I tend to find the final paragraph is usually the only part of a review I really need. I just won't get in depth mechanics explanations without playing the game, or want to be spoiled even on dumb minor things, whatever. If I'm really interested either in the game and am on the fense or just the review I'm more likely to read it all.

Which is great, because I feel people usually look at the score and just assume its either good or bad without reading anything. This is of course just anecdotal.

 

[QaaQer]

Reminds me of the article I read that showed average grades have increased over the last 50 years (i.e. way more As given out), dress sizes have gotten bigger (i.e. a size 8 today is bigger than a size 8 in the past), men's pant sizes aren't always the proper measurement (i.e. a 32 waist often times is actually a 34), & some other stuff.

 

[Phades]

It's a pretty good model of the modern game review but I still feel there needs to be an offset for a score of 10 in the review scale in this equation.

9* (x - 6) / 4

This seems to be a "close enough" substitute, as nothing should be rated "perfect", or "flawless" experience. There is no such thing, as everything has some flaw in it somewhere.

 

[PK Gaming]

The most common are:

90-100 A
80-90 B
70-80 C
60-70 D
60 below F

or slightly more strict:

93-100 A
85-93 B
78-85 C
70-78 D
Below 70 F

...

That's seriously screwed up

 

[Crazy_Sunshine]

I think it started to break as soon as someone put an arbitrary number behind someone's opinion.

 

[Orayn]

...

That's seriously screwed up

That's about what it is for high school, but it's not universal. I've had a lot of college classes that grade on a curve and set the class average anywhere from a C+ to a B, or use a fixed scale where a passing grade (C- or higher) starts as low as 50% due to difficulty.

 

[Nymphae]

9* (x - 6) / 4

This seems to be a "close enough" substitute, as nothing should be rated "perfect", or "flawless" experience. There is no such thing, as everything has some flaw in it somewhere.

Again I point to Tetris. The game had zero flaws. None whatsoever. You might be able to say oh it'd be cool if it had this or that feature, but base Tetris is a flawless game. I don't see how you could score this game less than 10 based on the merits of the game, the only thing that could knock it down is subjective experience (you get bored with it, don't enjoy the limited music options, want more colour, etc.)

 

[jaundicejuice]

There are a couple of reasons why I like Rock Paper Shotguns approach to reviewing games with their Wot I Think's. There's no score assigned, no positive and negative bullet points to skim, no tldr blurb...you kinda just have to read the whole thing.

As for why scores are skewed the way they are, it's a buyers market. The less time and money people have to sink into games, the pickier they're going to be about what they spend that time and money on. If you only buy three or four games a year, you're probably only going to want to play the "best" of the best.

 

[PK Gaming]

That's about what it is for high school, but it's not universal. I've had a lot of college classes that grade on a curve and set the class average anywhere from a C+ to a B, or use a fixed scale where a passing grade (C- or higher) starts as low as 50% due to difficulty.

Interesting

I've gone my entire life with the standard 80 = A, 70 = B, 60 = C, 50 = D scale, so I never got bent out of shape when a game I liked got 7. I always thought it was incredibly strange when people would freak out over scores like 7 or 8, but it kind of makes sense now (though most vitriol directed review scores is still pretty unwarranted though)

 

[Kamina]

Well, a game with a rating 0f 5/10 or lower is supposedly 50%+ shit. Or at least thats what we are led to believe.
I for my part prefer a reviews arguments to evaluate my own score, rather than blindly following one.

 

[Dr Dogg]

There are a couple of reasons why I like Rock Paper Shotguns approach to reviewing games with their Wot I Think's. There's no score assigned, no positive and negative bullet points to skim, no tldr blurb...you kinda just have to read the whole thing.

As for why scores are skewed the way they are, it's a buyers market. The less time and money people have to sink into games, the pickier they're going to be about what they spend that time and money on. If you only buy three or four games a year, you're probably only going to want to play the "best" of the best.

Way back in the olden days before RPS was better known they used to do round table Wot I Think's which sometimes ended up with varying opinions from each Jim, John, Kieron et al. There's a great programme on BBC 2 called The Culture Show which reviews various forms of media (radio plays, cinema, live music etc) as a panel and actually debate over the work as a group. Personally I find this approach gives a better overall view that just a singular opinion.

 

[commedieu]

Its our fault. Put too much value/traffic/debate around numerical scores to fight the system war.

 

[QaaQer]

And what's this about a conspiracy? Game reviewer choose high scores because they don't want to upset people?.

Have you seen the venom? Chick got death threats over his Halo IV review.

 

[TVC 15]

The internet is somewhat to blame for this, IT aggregates data to together, how hard would it have been in the 90's do the same thing with print based media. Pubs can simply gleam statistics more readily and easily, bung it in some spreadsheets and... WHALA

This is why games have hit a milieu creatively because data is so readily available, every bodies looking for the 'perfect formula' for that hit game. High dev costs, high ad cost, higher risk, less affordability for failure.

This is why market economics always fail in creative fields, because creative endeavors essentially become commoditised, creativity does not work well with homogenization.

Online gaming reviews are the very victims of the internet platform and mass of data online. Pubs and reviews are just stuck in one massive feedback loop, and a part of each others problem.

There was shit reviews in the 90's but with a very different kind of stench.

This post also stinks.

 

[Ataru]

But, yeah... having roomed with a Play editor and sharing a cube with a Gamefan editor, I've heard a lot. Nothing would surprise me any more. And I can't imagine either comes close to the insanity of the real thing.

I miss GameFan. For all their flaws, they were always a fun read. Plus they were big into import gaming and always had tons of pictures.

I preferred them to NextGen, which seemed dry and boring in comparison.

 

[flatearthpandas]

The only reason I mentioned the hundreds of people working on a game was to emphasize how much information is packed into even the shittiest game, not that hundreds of people make a game "worth" more.

We agree that reviews are a necessary way to communicate a game to those who have not played it. I gained a good-enough understanding by reading Wolfenstein's recent reviews, and in the end I will likely buy it because what they describe is something I would enjoy quite a bit from my tastes in FPS games.

Where I think I may not be expressing myself correctly is in your sentence:

I agree with this! The issue is that review scores cannot meaningfully share impressions. Review text can, if it's sufficiently long. But if the text is already doing a far, far better job than the score is, then the score is unnecessary.

I guess I don't associate the number of people or amount of code with the experience and so misunderstood what you were saying. That, coupled with the idea that a number is "offensively reductive" implied to me that there was a certain length of description that a game "deserved." What I understand now is that you meant only that it was insufficient to summarize an experience so large.

I'm not sure I agree necessarily, but I don't know that it's a point worth pursuing. I'm not trying to argue that scores should exist in isolation to review text, but I do think a score is something we can still justify having. I said earlier we can all comprehend and categorize our experiences into categories as simple as "worth it/not worth it" and "hate/dislike/ambivalent/like/love" I think we can perfectly share impressions with just as much brevity. Hence a number scale.

And while we generally need review text to justify and explain our impressions, I don't think that makes the text more important if we are able to encapsulate those impressions into a number. The text is the body while the number is the conclusion. The body is necessary if we want everyone to understand how we got to our conclusion, but the conclusion is still arguably the most important part. Of course the text will have a written conclusion, but if that conclusion can be transcribed into an easily consumed number, why not do it?

The OP's problem as I understand it, is why do we have such a disconnect between the text and the number of a review? Why can the text say "mediocre, mediocre, mediocre" and the number be at the upper end of the scale? Removing the number solves the disconnect, but I don't think a number scale is inherently a problem if it can accurately represent the conclusion of the review. And no matter how much text we give to the body of the review, I think the conclusion can only end in so many ways.