I purposely didn't bring up multiplayer reviews because those are all just complete garbage. Writing an accurate review of multiplayer when the multiplayer environment is not live and under real conditions is just impossible.
The divergence of reviews and public perception
- Thread starter Margalis
- Start date
I purposely didn't bring up multiplayer reviews because those are all just complete garbage. Writing an accurate review of multiplayer when the multiplayer environment is not live and under real conditions is just impossible.
Trent Strong
Banned
I'll never trust another review of a Rockstar game, that's for sure. They all get good reviews no matter how shitty they are.
I also think "vocal minority" is a nonsense argument.
No, I agree -- but time would help that.
I know who this vocal minority is supposed to represent. My beef is with the way it's used in arguments.
And such comments are usually based on an assumption backed up by sales figures.
That's about right. New Vegas wasn't significantly more buggy than either Fallout 3 or Skyrim, despite being much more complex and prone to scripting bugs to begin with. But Bethesda is a critic/fan darling, and Obsidian is still largely looked at as a pale imitation of a "real" AAA RPG studio.
Yes, it's actually true. I think you misunderstand my larger point. It's not about a single review or how that review is written, but rather a culture. You can still find gaming reviews which give a new big title a lower than expected score, or ones which actually take the time to critique something. But it is not widespread and not an expected part of the culture.
If movie reviews only largely came from magazines like Empire and Entertainment Weekly, or online sites catering specifically to the special interests of mainstream audiences who watch movies, then the culture of reviewing a movie would be very different. There would also not be any "more professional" standard to be held to or higher standards which they can be compared to.
As such, reviewers will have to rely on their own instincts to write and score movies based on:
- generating more interest and readership for their publication
- retaining their good relations with movie studios and distributors so they can continue to get previews and exclusive access to early screenings so they can continue operating the way they do
- retaining their readership by ensuring that they print what people want to read, and not turn off fans who might simply turn to other publications instead if they are the ones saying what they want to hear
That's what I'm trying to illustrate here. Not that Empire and Entertainment Weekly in particular have crappy reviews in particular, but that if the nature of such publications are all that make up movie reviews, then we would see a similar systemic problem.
Going to the ME3 example, perhaps the case is that critics have been trained, in a sense, to simply not have the same knee-jerk reactions to specific aspects of games and look more at the big picture. That is, I would hope, a "professional" reviewer's responsibility.
Now, I haven't reached the ending yet -- I'm about to -- so I'm kind of halfway talking out of my ass. But at the same time, I acknowledge that the game I have spent 50 hours playing has been a good one. It's not mindblowing, and personally I would even now rate it below its current Metascore, but unquestionably it was a very good game... with some flaws.
I'd like to believe that even after completion, I will still be able to recognize that I enjoyed these 50 hours quite thoroughly, and there's not a shitty ending in the world that can take that away from me.
This is obviously just one example out of many, but that's my sentiment here.
Now, I haven't reached the ending yet -- I'm about to -- so I'm kind of halfway talking out of my ass. But at the same time, I acknowledge that the game I have spent 50 hours playing has been a good one. It's not mindblowing, and personally I would even now rate it below its current Metascore, but unquestionably it was a very good game... with some flaws.
I'd like to believe that even after completion, I will still be able to recognize that I enjoyed these 50 hours quite thoroughly, and there's not a shitty ending in the world that can take that away from me.
This is obviously just one example out of many, but that's my sentiment here.
bigtroyjon
Member
Where is all this evidence? Millions of people bought the game and there are what, thousands of complaints out there? 1% maybe has expressed their outrage? You might be right but vocal minority comes into play because we only hear opinions from a tiny minority of the overall gaming public. As I said up above, the minority and majority can agree but there isn't any evidence of the public, majority opinion, since most of the people buying games don't give an opinion.
Oh, you know, just 83% of unhappy gamers (aka around 12,000 people on the poll below alone) who have played the game:
http://news.cnet.com/8301-17938_105...-should-bioware-change-mass-effect-3s-ending/
I think there are several reasons:
-The medium itself isn't entirely mature and gaming media reflects that, in fact much of the gaming media's target audience isn't particularly mature, they just want to see an arbitrary number at the end which reinforces their own opinion.
-Games are often difficult to assess properly in limited time, reviewers can't dedicate hours and hours on each game to really come to grips with the mechanics and look into every aspect of the game. What we often get is a review which appears to only scratch the surface. It doesn't help that reviewers are held to deadlines which ultimately affects the quality of the review.
-Many outlets rely heavily on developers and publishers giving them previews, advance copies, etc, there's a bit of an "I'll scratch your back" kind of deal going on in many cases.
-There are an absolute litany of samey reviewers who cover the same ground, approach a game from exactly the same perspective, who look for a shopping list of features in games, who produce carbon copy reviews, there's barely any diversity. The quality of the prose in so many cases is utterly dismal, so many reviewers have a difficult time actually properly expressing what makes a game great.
-The medium itself isn't entirely mature and gaming media reflects that, in fact much of the gaming media's target audience isn't particularly mature, they just want to see an arbitrary number at the end which reinforces their own opinion.
-Games are often difficult to assess properly in limited time, reviewers can't dedicate hours and hours on each game to really come to grips with the mechanics and look into every aspect of the game. What we often get is a review which appears to only scratch the surface. It doesn't help that reviewers are held to deadlines which ultimately affects the quality of the review.
-Many outlets rely heavily on developers and publishers giving them previews, advance copies, etc, there's a bit of an "I'll scratch your back" kind of deal going on in many cases.
-There are an absolute litany of samey reviewers who cover the same ground, approach a game from exactly the same perspective, who look for a shopping list of features in games, who produce carbon copy reviews, there's barely any diversity. The quality of the prose in so many cases is utterly dismal, so many reviewers have a difficult time actually properly expressing what makes a game great.
This.
Sit back, relax, give up on the enthusiast press, and wait for the Times / AV Club / Wired / Forbes(!?) / whoever to treat games and the industry as thoroughly as they do film and Hollywood.
In the meantime, the current gang will switch from pretensions of independence to a full-on embrace of their role as fluff-slingers, and will make themselves useful with more and earlier access to media and staff.
Perfect Cha0s
Member
12,000 out of what, 3.5 million? The 3.5M is reported shipped, so lets low-ball to 2 million copies sold. Even then, 12,000 is just 0.6% of players.
You're free to extrapolate, but don't act like 12,000 is more than a minority sample.
Also, I would bet that most reviewers don't finish the games they review, or as mentioned before, they go for the "big picture" of the game.
In ME3's case, the bad ending doesn't negate what was for most people, a very enjoyable, memorable experience.
Did you fail to read the poll? 12,000 out of 15,000 respondents. AKA those who took the poll. It's not anybody's mistake if those 'silent majority' didn't vote. And where are these 'silent majority' anyway? Up to what point are you going to hammer out the silent majority argument? How would you propose we do a poll that covers ALL of the users? I posted another poll from BSN which had 60,000 people voting for a changed ending too:
I know right?
You start working with reliable numbers to within 5% at a few hundred responses, given the entire population of the US. If you want to attack the poll, attack the sample composition rather than the sample size.
The 'vocal minority' argument is always so pointless to make. How are we meant to prove or disprove it? You actually want a survey of a population? Those two words are just thrown out so often to try and debase an opposing opinion, they're pretty much useless now. Anyone who uses it in their arguments are actually decreasing the quality of their posts.
Red Blaster
Member
Fucking statistics, how does it work
Here's the problem with the "vocal minority" statement. The very nature of this statement is based on accepting that if a person is not vocally expressing this opinion, that the person probably disagrees with it but doesn't care to make his voice heard.
For example, if a bunch of people complain about how shitty Game XYZ is, and the "vocal minority" argument is invoked, it is accepted that it means to state that only a small number of people who played Game XYZ dislike it, and they are the ones who are complaining the loudest. Examples are then drawn out that out of the "millions of people" who bought Game XYZ, only "a few thousand" are complaining about it on the internet.
But the flaw in this argument is that this is factually wrong. Just because a person is not voting on a poll, or not complaining about it online, or not signing petitions about something, does not mean that they DISAGREE with the position. They could agree that Game XYZ is a piece of shit, but just not care enough to take an active stand about it. If a friend asks them about it, they would say the game sucked, but outside of that they don't talk about it because they have better things to do.
Why is the presumption of people who throw the "vocal minority" argument out always that if you are not complaining about something vocally, you are grouped with the people who probably liked it but are keeping silent? Is there a good argument around this?
Let's say millions of people did buy ME3, and let's say 5% of them are so outraged by the ending that they are making a stand online to do all sorts of vocal stuff. Are they the vocal minority? Or the vocal majority? How many of the other 95% -like- the ending? How many of them also dislike the ending? Can anyone provide definitive figures? If not, why is it assumed that most of the other 95% are on the opposite side?
For example, if a bunch of people complain about how shitty Game XYZ is, and the "vocal minority" argument is invoked, it is accepted that it means to state that only a small number of people who played Game XYZ dislike it, and they are the ones who are complaining the loudest. Examples are then drawn out that out of the "millions of people" who bought Game XYZ, only "a few thousand" are complaining about it on the internet.
But the flaw in this argument is that this is factually wrong. Just because a person is not voting on a poll, or not complaining about it online, or not signing petitions about something, does not mean that they DISAGREE with the position. They could agree that Game XYZ is a piece of shit, but just not care enough to take an active stand about it. If a friend asks them about it, they would say the game sucked, but outside of that they don't talk about it because they have better things to do.
Why is the presumption of people who throw the "vocal minority" argument out always that if you are not complaining about something vocally, you are grouped with the people who probably liked it but are keeping silent? Is there a good argument around this?
Let's say millions of people did buy ME3, and let's say 5% of them are so outraged by the ending that they are making a stand online to do all sorts of vocal stuff. Are they the vocal minority? Or the vocal majority? How many of the other 95% -like- the ending? How many of them also dislike the ending? Can anyone provide definitive figures? If not, why is it assumed that most of the other 95% are on the opposite side?
Perfect Cha0s
Member
Did you fail to read the poll? 12,000 out of 15,000 respondents. AKA those who took the poll. It's not anybody's mistake if those 'silent majority' didn't vote. And where are these 'silent majority' anyway? Up to what point are you going to hammer out the silent majority argument? How would you propose we do a poll that covers ALL of the users? I posted another poll from BSN which had 60,000 people voting for a changed ending too:
I know right?
I'm not attacking the poll, I'm saying that we can't hold it up and go "HOLY BUTTS YOU GUYS EVERYONE HATES ME3 ENDING!"
Also, I'd say of those 95%, a substantial amount probably havent even finished the game, or even intend to.
The opposite is also true.
The sample isn't randomized; therefore, it's non-representative. However, if magically the sample was randomly selected, 12,000 respondents is more than enough.
Failure to read again. Did anyone actually say that?
The Technomancer
card-carrying scientician
Can we figure out what the sample population has in common and possibly make inferences from the data in light of that?
Perfect Cha0s
Member
Oh, I know. I wasn't intending for my post to challenge the poll, just to say that we (obviously) can't say for sure, and that, anecdotally, many people are saying "Loved the game, hated the ending," and thus, the review scores could arguably hold.
Reviewers attacking fans for wanting a different ending is a whole 'nother ball game.
Replicant said:
See post.
Edit: Just for the record, I don't like the ending to ME3, at all, and would like to see it changed.
It's not representative of the whole playerbase but it is representative of the ones on the BSN at least. We should start a poll in the spoiler thread or something to confirm what GAF users who've finished the game think.
You mean what the sample has in common with a hypothetical random sample? As in, does this sample of 12,000 have any merit? Yes. You would have to do some limited random sampling to use as a baseline to compare against the non-random sample. If the results from the two groups (random and non-random) are quasi-equivalent, they would be labeled as such. The results would still be weak in design, but, yes, we would learn something from the data about the population as whole.
Right, but I think the argument here is that the BSN players nor the GAF players are representative of Joe Nobody out there in 'Merica -- which is true. What's funny is that Bioware records a lot of telemetry data and it would be relatively easy for them to send out a poll via PSN/XBL and publish the results.
Edit: Beaten. Perfect Chaos read my mind.
Perfect Cha0s
Member
It would be bold for BioWare to put a mandatory poll into ME3, that came up when you started the game, but even then, many people have now moved on to other games, and they'd still be missing the people who don't have internet connectivity.
That's fine, the people with Internet would be more than enough for a sample.
The Technomancer
card-carrying scientician
Eh, actually I meant what those polled had in common, to try and understand what distinguishes this sample from a truly random sample. I think we have similar ideas though.
IISANDERII
Member
You can look back a little further to Mass Effect 2. With all the great reviews, year end accolades and the might of the EA marketing machine, you'd think it was the #1 selling game that year when in actuality it barely made the top 30.
The vocal minority says it's wonderful, the silent majority didn't waste its breath.
The vocal minority says it's wonderful, the silent majority didn't waste its breath.
I still think the poll holds some value. Even if its only a tiny sample size of the entire audience. Sure its only 60,000 out of millions, but I'd say its safe to say that a high concentration of these 60,000 are ME's most passionate fans. These are the fans who are fully invested into the storyline, their characters. These are the fans who will buy all the merchandise, the limited Collectors Editions, and write the emails to the devs. If the ending really needed defending, you can be damn sure the fans that would defend it would find their way onto the polls scattered around the net and vote. Vocal minority? sure, but they're also your biggest fans.
Oh, yes. Now I understand; that part is incredibly easy. You'd simply frame the population you're interested in as those who responded to the poll. A hypothetical survey could go like this:
Code:
1. I enjoyed the ending of Mass Effect 3 (1 - strongly disagree; 7 - strongly agree)
2. I would consider myself a 'core gamer.' (1 - strongly disagree; 7 - strongly agree)
3. I am usually happy with the games I purchase. (1 - strongly disagree; 7 - strongly agree)Etc. etc.
Depending on how many topics a poll may cover (from amount of disposable income to time dedicated to playing every week) you could easy reach conclusions about who these people are and what they have in common. You can't statistically conclude causation (what makes these people dislike the ending) from survey data alone, but you could most definitely find insight.
While I agree the poll does hold some value, being passionate doesn't impart additional value. Your point makes it seem like passionate fans' views hold more value than the average customers'.
:science: time!
Margins of error for pretty much any polling are conducted to 95% confidence, so that's what I'll be using here. That means that on average, one time out of twenty the actual answer will be outside the margin. Not necessarily a lot; when you see a political poll with 3% MoE, that's 60-40 or something, the actual election being 56-44 is outside that margin even though it's still a solid win for the guy the pollsters picked.
Now, there's a 85-point gap there between "hated it" and "naw, it's okay". That means we need a 43% margin of error to not be 19-in-20 sure we asked enough people.
How do we figure out how many people is "enough"?
For that 95% confidence...
Sample size = 1.96 squared, then multiplied our best guess at the actual amount of haters times 1 minus that same number. If we assume it's a 50-50 chance, 1.96^2 * .5*.5.
Take that entire mess, and divide it by the margin of error squared; we need .43.
(1.96^2) * (.5 * .5)
over
(.43^2)
3.8416 * .25
over
.1849
.9604
over
.1849
Rounding the answer up, we learn that unless the people who hate ME3 hate it so bad they're stuffing the ballot box, we need to ask a grand total of six random guys to be pretty sure that if five of them don't like the ending, most people who bought ME3 didn't like it either.
The more you know!
Well... from a developer's standpoint, if you were a community manager looking for advice and input from the public to improve your game, who would you look to?
Also, if ME3 had been an exclusive console title, the poll would likely also attract tons of haters in addition to the passionate fans.
Perfect Cha0s
Member
What's funny and/or sad is that I'd bet that Bioware would never do a poll like that, because they've shown many times throughout this debacle that they're afraid to swallow their pride, at least publicly.
However, I feel that I've contributed to pulling this thread slightly off-topic.. sooo..
The general divergence in review scores and public opinion of some games is the result of multiple compounding factors. Frankly, reviewers aren't immune to hype. That makes for a poor source of objective analysis, but there it is. Often, reviewers aren't as professional as they should be.
Reviewers, many times, aren't representative of the groups that will put their reviews under the most scrutiny, or the ones that will heed their reviews when considering buying a game (and the two are usually different groups.) I would think the former is more so true than the latter, but holding up a slew of high review scores can sway some people, regardless of what games the reviewers like, or why they gave the score they did.
I think it would be really interesting to see the divergence between people who tear reviews apart, and the people that actually give a shit what reviewers say.
ME3 shipped 3 million copies. Say all those get sold. What are you saying our sample size needs to be? To be reasonably substantive, wouldn't your sample size need to be a few hundred at least? Say, given the standard 95%.
It's okay Jett. Fuck your logic. I'm just going to quote your tag and post hatersgonehate.gifI made a some what similar thread last year(you're much nicer than me though), used LA Noire and MGS4 as examples of reviewers overreacting, overhyping and ultimately ignoring every single flaw those two games had, using their own quotes. My choice of words for the my thread title was not the best though.
Most game reviewers are not critics. They care little about actually crafting a critical analysis of whatever it is they're reviewing. They probably don't even know how. I don't know about you but there's not a single guy in this industry whose reviews I honestly trust and follow.
I made a some what similar thread last year(you're much nicer than me though ), used LA Noire and MGS4 as examples of reviewers overreacting, overhyping and ultimately ignoring every single flaw those two games had, using their own quotes. My choice of words for the my thread title was not the best though.
Most game reviewers are not critics. They care little about actually crafting a critical analysis of whatever it is they're reviewing. They probably don't even know how. I don't know about you but there's not a single guy in this industry whose reviews I honestly trust and follow.
), used LA Noire and MGS4 as examples of reviewers overreacting, overhyping and ultimately ignoring every single flaw those two games had, using their own quotes. My choice of words for the my thread title was not the best though. Most game reviewers are not critics. They care little about actually crafting a critical analysis of whatever it is they're reviewing. They probably don't even know how. I don't know about you but there's not a single guy in this industry whose reviews I honestly trust and follow.
/end fucking thread.I made a some what similar thread last year(you're much nicer than me though
Most game reviewers are not critics. They care little about actually crafting a critical analysis of whatever it is they're reviewing. They probably don't even know how. I don't know about you but there's not a single guy in this industry whose reviews I honestly trust and follow.
Well said sir. Wish I didn't have to agree.
Perfect Cha0s
Member
Seeing the fervor the ending has generated, that is probably part of the situation.
Looking for advice comes after tallying and processing opinions. And developers would like to know if majority liked their game, passionate or not.
SolidSnakex
Member
Didn't you mention that you liked Mark's reviews from CGR?
Similar threads
- Poll