• Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.

Media Create Sales: Feb. 8 - 14, 2010

Stumpokapow

listen to the mad man
The article misquotes Nippon Ichi's revenue as declining 98% when in reality their profit declined 98%.

If I have an orange juice company, and my operating costs are ten million dollars a year, and I make ten million dollars, my profit is 0. This does not mean my revenue is 0. In this particular case they're even dumber, because they link their own article on the subject which gets it right.

Games journalism is frankly terrible as it relates to business issues. Conflating profit and revenue, shipped units and sold units, claiming games bombed when they didn't or didn't when they did, citing ZG ZHARTZ. That's the tip of the iceberg... the quality of games writing in this area is just embarrassing.
 

Dalthien

Member
Stumpokapow said:
The article misquotes Nippon Ichi's revenue as declining 98% when in reality their profit declined 98%.
Yeah, the article was wrong about the revenue decline. But they recently updated their forecasts so that they are expecting some pretty sizable (for a small company, such as Nippon Ichi) losses this year.

But yeah - games journalism is pretty damned awful across the board. You certainly won't get an argument out of me on that. I would be hesitant to even use the word 'journalism'.
 
Enterbrain (Famitsu) will start 'tracking' digital downloads in Spring:
http://www.andriasang.com/e/blog/2010/02/22/enterbrain_download_sales/
Enterbrain, publisher of the Famitsu line of magazines, appears to have a means of tracking download game sales. Earlier this month, the company announced plans to begin a data service that tracks online game sales. The service will kick off in the Spring and will initially track sales through Wii Shopping Channel, DSi Shop, PlayStation Store, Xbox Live Marketplace, and Apple's App Store.

Current retail sales services base their tallies off sales information provided by partner retailers. Entebrain's new service will be based off purchase data from 300,000 gamers.

Enterbrain will offer the weekly and monthly sales data to members of its f-ism ("Famitsu Intelligence Strategic Marketing") marketing service as an option costing around ¥200,000 per month.

According to Enterbrain this is the first domestic download sales data service.
Maybe its me, but if they do a top10 by units, wouldn't it normally be only Apple apps in it? Cheap, popular ones I mean.


BishopLamont said:
Thanks for that, I was wondering what to compare this game to. Looks like I'm overestimating many games, I can change my prediction right, just for this game? :lol
Anyone can change their predictions up to the deadline. Just edit the post. I'll collect them when deadline is up.
 

duckroll

Member
If they're using a sample of 300,000 gamers to extrapolate sales estimates for a bunch of different online distribution platforms, not only do I expect the data to be extremely inaccurate, but I also expect it to be pretty much useless. The method they are using is similar to how TV ratings are generated, but the nature of actual sales vs TV viewership habits is very different and I don't really have any faith in this at all.
 

Dalthien

Member
Kurosaki Ichigo said:
Enterbrain (Famitsu) will start 'tracking' digital downloads in Spring:
http://www.andriasang.com/e/blog/2010/02/22/enterbrain_download_sales/

Maybe its me, but if they do a top10 by units, wouldn't it normally be only Apple apps in it? Cheap, popular ones I mean.
It depends who they actually get in their sample group. But since the sample group will only be 300k, it really doesn't even seem worthwhile. With many tens of millions of gamers on the platforms that it will be tracking, 300k is way too small to make any meaningful extrapolations from the sample. It would be way too easy for the sample group to skew in one direction or another and provide completely bogus results. Small sample sizes only have a chance of providing a reasonable estimate if they are a truly random and representative sample. Unless Famitsu has access to the actual online sales numbers from Nintendo, Sony, etc., they would have no way of knowing how their small sample size compares to the full online marketplace. And if they had those actual numbers from the platform holders, then there's no point for the sample size in the first place.

Seems really pointless to me - at least as described in that little blurb.

Edit - ha ha, duckroll beat me to it!
 

m3k

Member
Stumpokapow said:
The article misquotes Nippon Ichi's revenue as declining 98% when in reality their profit declined 98%.

If I have an orange juice company, and my operating costs are ten million dollars a year, and I make ten million dollars, my profit is 0. This does not mean my revenue is 0. In this particular case they're even dumber, because they link their own article on the subject which gets it right.

Games journalism is frankly terrible as it relates to business issues. Conflating profit and revenue, shipped units and sold units, claiming games bombed when they didn't or didn't when they did, citing ZG ZHARTZ. That's the tip of the iceberg... the quality of games writing in this area is just embarrassing.

your posts are the best, just saying
 

C-Jo

Member
Jeez, you weren't kidding about a shot in the dark. Here's my sure to be embarrassing attempt:

[NDS] Super Robot Wars OG: Saga Endless Frontier Exceed (Namco Bandai) - 90k
[NDS] Shiren the Wanderer 4: Kami no Hitomi to Akuma no Heso (Spike) - 40k
[NDS] Estpolis: The Lands Cursed by the Gods (Square Enix) - 35k
[NDS] Higurashi no Nakukoru ni Kizuna: 4th Chapter - Kizuna (Alchemist) - 10k
[PSP] Hot Shots Tennis Portable (SCE) - 50k
[PSP] Kenka Bancho 4: Ichinen Sensou (Spike) - 15k
[PSP] BlazBlue Portable (Arc System Works) - 17k
[PSP] Power Pro Success Legends (Konami) - 100k
 
Erm, 300k is by far a large enough sample size to extrapolate from, provided it's a truly random sample.

Really, statistically you can get by with a sample size of 30 people. 300k is massive.
 

duckroll

Member
Segata Sanshiro said:
Erm, 300k is by far a large enough sample size to extrapolate from, provided it's a truly random sample.

Really, statistically you can get by with a sample size of 30 people. 300k is massive.

My contention is not with the size of the sample, but by the type of sample. Sampling a store's sales is very different from sampling a gamer's buying habits.
 

Dalthien

Member
Segata Sanshiro said:
Erm, 300k is by far a large enough sample size to extrapolate from, provided it's a truly random sample.
The highlighted part of your statement is the important part.

Without actual numbers to provide a comparison with the sample group - there's no way of knowing if the sample group is a truly random sample.

There's a reason these trackers try to cover a large percentage of retailers. They don't track 1% of retailers and then try to extrapolate that out to the entire industry, because there's no way of knowing if that 1% would be a truly random and representative sample of the industry. So they try to cover 65% or whatever numbers they end up tracking.

Likewise, trying to use 1% of gamers to cover the entire digital industry is equally fraught with disaster.

Small sample sizes can work reasonably well in politics because you can make determined efforts to get a random sample, and then you can compare the results of your sample to actual recent voting patterns to verify that your sample falls more or less in line with the population at large.
 

Chris1964

Sales-Age Genius
Segata Sanshiro said:
Erm, 300k is by far a large enough sample size to extrapolate from, provided it's a truly random sample.

Really, statistically you can get by with a sample size of 30 people. 300k is massive.
300k isn't massive. I think Enterbrain tracks 50% and Media Create 60-65% of Japanese retailers (Media Works less than Enterbrain) and even with such big sample there are very often big differences. Now with a sample of only 300k choosen from gamers and not retailers I really doubt the results will be representative of reality.
 
Segata Sanshiro said:
Erm, 300k is by far a large enough sample size to extrapolate from, provided it's a truly random sample.

Really, statistically you can get by with a sample size of 30 people. 300k is massive.

300k isn't that big, especially if it's covering all platforms (In which case, the sample isn't really random if all these consumers own all platforms. If they don't and the 300k is split up across all platforms, then the sample size will be extremely small)

Not to mention that they're tracking consumers and not retailers, making it subject to personal bias and taste.

Nonetheless, it's at least a first step. Hopefully we'll see the sample size grow over time. Maybe if we're lucky, some of the figures that misrepresent a platform (by appearing to sell far less than they really do) will encourage platform holders to give out the real information in order to counteract the bad PR :D
 

Stumpokapow

listen to the mad man
Nuclear Muffin said:
300k isn't that big, especially if it's covering all platforms (In which case, the sample isn't really random if all these consumers own all platforms. If they don't and the 300k is split up across all platforms, then the sample size will be extremely small)

Not to mention that they're tracking consumers and not retailers, making it subject to personal bias and taste.

You are aware that for binary yes/no questions, a sample size of 350 people is enough to estimate the preferences of 120,000,000 people (+-5%, 19 times out of 20), right? Sampling is a lot more powerful than people think it is.
 

RJT

Member
Dalthien said:
The highlighted part of your statement is the important part.

Without actual numbers to provide a comparison with the sample group - there's no way of knowing if the sample group is a truly random sample.

There's a reason these trackers try to cover a large percentage of retailers. They don't track 1% of retailers and then try to extrapolate that out to the entire industry, because there's no way of knowing if that 1% would be a truly random and representative sample of the industry. So they try to cover 65% or whatever numbers they end up tracking.

Likewise, trying to use 1% of gamers to cover the entire digital industry is equally fraught with disaster.

Small sample sizes can work reasonably well in politics because you can make determined efforts to get a random sample, and then you can compare the results of your sample to actual recent voting patterns to verify that your sample falls more or less in line with the population at large.
They can ask the same people about their retail purchase habits and then check with their retail data. That way they can make sure their sample is representative of gamers who buy at stores.

I think we can safely assume that if a sample is representative of retail console buyers, it is also representative of online console buyers. This may not be applied to PC sales, I'd reckon (since every game can be purchased online, a lot of people don't buy retail), but I think it's fair for Wii/PS3 at least.
 

Stumpokapow

listen to the mad man
RJT said:
I think we can safely assume that if a sample is representative of retail console buyers, it is also representative of online console buyers.

Not a chance. Even the worldwide BEST CASE SCENARIO (Call of Duty and Halo map packs, where you can buy redemption cards in stores and so there's no barrier whatsoever to get the content) would still reveal an attach rate that suggests only the most technically avid consumers get in on digital downloads.
 

Jonnyram

Member
Apparently Softbank was selling FFXIII for 2222 yen because today's date is Heisei 22 (2010) / 2 / 22. Yeah, noone on 2ch quite understands how 22222 translates to 2222 or why FFXIII was picked either, but hey, if you wanted a cheap copy of FFXIII, today was the day.
 

Koren

Member
Stumpokapow said:
You are aware that for binary yes/no questions, a sample size of 350 people is enough to estimate the preferences of 120,000,000 people (+-5%, 19 times out of 20), right? Sampling is a lot more powerful than people think it is.
It is, assuming in the 350 sample you get enough 'yes' and enough 'no'.

Take 20 million people (that six time less than you number). A game sold, for example, 40k (so 1 out of 500). Buying a game is a yes/not question, isn't it?

Now, take a perfect random sample of 350 people. You get :
- 0 games bought 50% of the time -> 0k sold
- 1 game bought 35% of the time -> 55k sold
- 2 games bought 12% of the time -> 110k sold
...
- 4 games bought 0.5% of the time -> 220k sold
etc.

So in this example, one out of two tries, you get 0, one out of three, you're quite right (with a 40% error), one out of six, you far overestimate (3x and more) the sales.

Of course, if you have 6 million 'bought' and 14 millions 'not bought', you'll be really close to 105 'bought', and the error margin will be far smaller, and in line with your +- 5% most of the time.



On a market with 10 million customers, if you want to be able to measure sales at a ~1-2k level, 100-200k is a bare minimum if the subset is random (the esperance of the result will be 10-40, so a variance around 3-6, which is still 15-30% of error as standard deviation). With a clever subset, you can theorically reduce the minimum, though, as long as you know the composition of the whole market with respect to the subset.
 
Jonnyram said:
It's selling out because it's been woefully undershipped. In saying that, I mean retailer orders were pretty low - it's an adventure game, new IP, and a mature title; would you risk buying in a shitload of copies?
Me, I'd play it safe and just buy two dozen display cases.
Stumpokapow said:
In order from least recent release to most recent release:

Shiren DS (Sega) - 79k week one
Shiren 3 (Sega) - 58k week one
Shiren DS 2 (Sega) - 47k week one
Shiren 3 PSP (Spike) - 14k week one

...
There's definitely a downward trend looking at it that way, but we could also conclude that "DS > Not DS" and "New > Not New", both of which would work in Shiren 4's favor.
 

duckroll

Member
JoshuaJSlone said:
There's definitely a downward trend looking at it that way, but we could also conclude that "DS > Not DS" and "New > Not New", both of which would work in Shiren 4's favor.

No, it's definitely a downward trend. Chronologically:

Shiren (SFC) - Over 200k
Shiren GB (GB) - Over 100k
Shiren GB2 (GBC) - 172k
Shiren 2 (N64) - 283k
Shiren (DS) - 195k [remake of Shiren]
Shiren 3 (Wii) - 97k
Shiren DS2 (DS) - 85k [remake of Shiren GB2]

Unless you want to argue that N64 > DS, or that a remake of Shiren 1 is somehow newer than Shiren 3...
 

RJT

Member
Stumpokapow said:
Not a chance. Even the worldwide BEST CASE SCENARIO (Call of Duty and Halo map packs, where you can buy redemption cards in stores and so there's no barrier whatsoever to get the content) would still reveal an attach rate that suggests only the most technically avid consumers get in on digital downloads.
So?

Remember, they're asking a random samples of 300k consumers (they can validate their randomness by asking what retail games they bought and checking with the retail data) what they bought online. They're not using their retail behavior to extrapolate their online behavior, they're using it to validate the randomness of the sample.
 
duckroll said:
No, it's definitely a downward trend. Chronologically:

Shiren (SFC) - Over 200k
Shiren GB (GB) - Over 100k
Shiren GB2 (GBC) - 172k
Shiren 2 (N64) - 283k
Shiren (DS) - 195k [remake of Shiren]
Shiren 3 (Wii) - 97k
Shiren DS2 (DS) - 85k [remake of Shiren GB2]

Unless you want to argue that N64 > DS, or that a remake of Shiren 1 is somehow newer than Shiren 3...
Well, I'd say the 6 year gap between N64 and DS releases makes looking at them in the same trend as 2006-2010 Shiren releases some trouble.

Good point on how Shiren DS 2's LTD comparison to the first DS release was much more lacking than just looking at the first weeks indicates, though. Without knowing much about the history of the series, though, I'm wondering how much of that is some general Shiren trend or brand tiredness, and how much is just that people don't care as much about a remake of a portable game from the time when portables definitely got the second class games.
 

duckroll

Member
JoshuaJSlone said:
Well, I'd say the 6 year gap between N64 and DS releases makes looking at them in the same trend as 2006-2010 Shiren releases some trouble.

Good point on how Shiren DS 2's LTD comparison to the first DS release was much more lacking than just looking at the first weeks indicates, though. Without knowing much about the history of the series, though, I'm wondering how much of that is some general Shiren trend or brand tiredness, and how much is just that people don't care as much about a remake of a portable game from the time when portables definitely got the second class games.

Well don't worry, after Shiren 4 bombs, we can discuss how much of that is some general Shiren trend or brand tiredness, and how much of it is just that people don't care as much about an entry with super crappy art and weird DBZ style attacks. Not to mention, it's being published by Spike now. :)
 

donny2112

Member
Stumpokapow said:
binary yes/no questions,

And that is where the comparison breaks down. With hundreds/thousands of choices, it's not the same situation as Gallop poll survey for political questions.

Entebrain's new service will be based off purchase data from 300,000 gamers.

And that right there guarantees it won't be a random sample. Anyone who considers themselves a gamer probably wouldn't waste points on a software like "My Aquarium" (make your own aquarium to look at on the TV) for WiiWare. However, that was the #7 game on WiiWare in the U.S. 14 months after its release.

It'll be interesting to see, if any results ever get released, but it can't accurately cover the whole download market. Even NPD has major problems with at least the low end of the spectrum for retail, and they have ~60% of the retail market covered. They might be okay at showing numbers for the Top few games on each service, though, and it is better to start somewhere than not to start, at all. :)
 

jcm

Member
donny2112 said:
And that is where the comparison breaks down. With hundreds/thousands of choices, it's not the same situation as Gallop poll survey for political questions.



And that right there guarantees it won't be a random sample. Anyone who considers themselves a gamer probably wouldn't waste points on a software like "My Aquarium" (make your own aquarium to look at on the TV) for WiiWare. However, that was the #7 game on WiiWare in the U.S. 14 months after its release.

It'll be interesting to see, if any results ever get released, but it can't accurately cover the whole download market. Even NPD has major problems with at least the low end of the spectrum for retail, and they have ~60% of the retail market covered. They might be okay at showing numbers for the Top few games on each service, though, and it is better to start somewhere than not to start, at all. :)

I don't think you can count on that writeup to be accurate regarding their sampling method. Here's the original press release. I can't read Japanese, but google translate suggests they're using random stratified sampling, which, if designed correctly, corrects for the problem you mention.

This is in no way similar to what NPD/Enterbrain/MC normally do, by the way, so comparing the percentages is completely meaningless.
 
Dalthien said:
Likewise, trying to use 1% of gamers to cover the entire digital industry is equally fraught with disaster.

Chris1964 said:
300k isn't massive.

Nuclear Muffin said:
300k isn't that big, especially if it's covering all platforms

facepalm.gif


Please, if you're going to comment on this, do so from a solid knowledge of sampling and polling. While there are numerous factors that can affect the legitimacy of a measure like the one described in this PR, the sample size itself is absolutely not a problem. As a comparison, the Nielsen television ratings system for the United States utilizes a sample size of just 25,000 households. Complex polling that produces quite detailed and accurate results is regularly performed using sample sizes an order of magnitude smaller than the supposed sample described here.

Now, what's worth being concerned about is the form of selection being used to isolate the individuals polled here, the potential for self-selection bias, the methodology used to gather information (individuals do not always reliably report their own purchases and consumption), etc. and without detailed knowledge of the survey's methodology we can't be certain any of these things will be done "correctly" -- but 300k respondents is absolutely enough people to use as a sample for a well-designed survey of this type.

jcm said:
This is in no way similar to what NPD/Enterbrain/MC normally do, by the way, so comparing the percentages is completely meaningless.

Exactly. Producing a fine-grained model of sales at retail (including information like sale price) is a very different process from producing a coarse-grained estimate of relative sales levels for DD titles to end consumers and the differing sampling methodologies illustrate that.

(It's worth noting that part of the issue with the sample for retail models like NPD and MC is specifically that they can't take a random sample -- they have specific retail partners they receive data from and have to estimate completely for other individual chains and retailers, even though various factors may cause sales figures at these non-sampled retailers to vary significantly.)
 

doicare

Member
After playing some Lost In Nightmares i think now is a good a time as any for my resident evil 5 alternative edition week 1 prediction:

resident evil 5 A.E. - 140,000
 

Stumpokapow

listen to the mad man
doicare said:
After playing some Lost In Nightmares i think now is a good a time as any for my resident evil 5 alternative edition week 1 prediction:

resident evil 5 A.E. - 140,000

You think the first week of a rerelease of RE5 is going to 50% of the first week of the original release of RE5? Really?
 

donny2112

Member
charlequin said:
As a comparison, the Nielsen television ratings system for the United States utilizes a sample size of just 25,000 households.

And when Nielsen tries to get down to individual games played most on PC, they often get it quite incorrect. :p The greater number of options, the more coverage you need to get a representative sample. I think it would be great for overall numbers (e.g. # of downloads for a whole system), but getting down to results on particular games should get more and more problematic the lower the absolute number of sales goes. A top ten should be interesting, though, and like I said, it's better to start somewhere than not to start, at all.
 

doicare

Member
Stumpokapow said:
You think the first week of a rerelease of RE5 is going to 50% of the first week of the original release of RE5? Really?
I can see why people would have doubts but i think everybody will be plesently surprised with 'how well' this game does, it should be the number 1 selling game of the week.
 
doicare said:
I can see why people would have doubts but i think everybody will be plesently surprised with 'how well' this game does, it should be the number 1 selling game of the week.

I'm going to go with 50k, and I think that would be a great success for a quick re-release.
 

Dalthien

Member
charlequin said:
Please, if you're going to comment on this, do so from a solid knowledge of sampling and polling.
Charlequin, please don't automatically assume that I don't have a solid knowledge of sampling and polling.

I've done a fair bit of work on political polling, and I've completed more than one undergraduate course in statistics. There are certainly others with more expertise than me, but I'm definitely not some neophyte in the area.

300k is a phenomenal sample size (and far, far more than would ever be necessary) if you can be sure that the sample size is truly random and representative of the full set of people.

You then went on to list a number of possible problems in generating a truly random sample that is representative of the full set. Without assurances that the selected sample size is truly random and representative of the full set, then a 300k sample size becomes incredibly small and worthless when the full set is 100x greater in size.
 
doicare said:
I can see why people would have doubts but i think everybody will be plesently surprised with 'how well' this game does, it should be the number 1 selling game of the week.
Didn't seem like there was much buzz about it at all in Akiba on the weekend, but who knows I guess.
 

jj984jj

He's a pretty swell guy in my books anyway.
Predictions:
[NDS] Super Robot Wars OG: Saga Endless Frontier Exceed (Namco Bandai) - 74k
[NDS] Shiren the Wanderer 4: Kami no Hitomi to Akuma no Heso (Spike) - 24k
[NDS] Estpolis: The Lands Cursed by the Gods (Square Enix) - 30k
[NDS] Higurashi no Nakukoru ni Kizuna: 4th Chapter - Kizuna (Alchemist) - 19k
[PSP] Hot Shots Tennis Portable (SCE) - 48k
[PSP] Kenka Bancho 4: Ichinen Sensou (Spike) - 56k
[PSP] BlazBlue Portable (Arc System Works) - 20k
[PSP] Power Pro Success Legends (Konami) - 26k
 

Chris1964

Sales-Age Genius
doicare said:
After playing some Lost In Nightmares i think now is a good a time as any for my resident evil 5 alternative edition week 1 prediction:

resident evil 5 A.E. - 140,000
There is no point of predicting after you 've seen comgnet weekly report.
 

cvxfreak

Member
Jonnyram said:
Apparently Softbank was selling FFXIII for 2222 yen because today's date is Heisei 22 (2010) / 2 / 22. Yeah, noone on 2ch quite understands how 22222 translates to 2222 or why FFXIII was picked either, but hey, if you wanted a cheap copy of FFXIII, today was the day.

Softbank sells games now? ;)
 

Nirolak

Mrgrgr
Apparently Namco is recalling Tales of Graces.

Kotaku said:
Namco Bandai has recalled Wii title Tales of Graces for being too buggy.

The company has confirmed ten deflects in the title and has set up an exchange program for those who have purchased the title. Bugs include the inability for players to acquire certain items, music glitches, walking in place, freezing and more. Buggy bug buggity bug bug. Namco Bandai states that the company is revising its quality control system.

Going on sale December 2009, Tales of Graces is the 12th title in the Tales series. It will be released in Europe later this year.
Source: http://kotaku.com/5477866/tales-of-graces-recalled

Is this the first major Wii title to be recalled in Japan? I'm not remembering any others right now.

Here's the actual Japanese page also in case anyone wanted to use it: http://www.bandainamcogames.co.jp/cs_support/info/tog.html
 

duckroll

Member
Nirolak said:
Apparently Namco is recalling Tales of Graces.


Source: http://kotaku.com/5477866/tales-of-graces-recalled

Is this the first major Wii title to be recalled in Japan? I'm not remembering any others right now.

Here's the actual Japanese page also in case anyone wanted to use it: http://www.bandainamcogames.co.jp/cs_support/info/tog.html

They are not recalling the game. They are offering exchanges via retailers in-store now. They pressed corrected discs over a month ago, and customers could email customer support to arrange exchanges back then. I already posted about this previously in the ToG thread.
 

Nirolak

Mrgrgr
duckroll said:
They are not recalling the game. They are offering exchanges via retailers in-store now. They pressed corrected discs over a month ago, and customers could email customer support to arrange exchanges back then. I already posted about this previously in the ToG thread.
Ah I see, my bad then.
 

Nirolak

Mrgrgr
Man God said:
They are also claiming a western release for the game... :lol
Yeah I thought that sounded odd. I haven't been following Tales of Graces that closely as you might be able to tell. :lol
 
duckroll said:
They are not recalling the game. They are offering exchanges via retailers in-store now. They pressed corrected discs over a month ago, and customers could email customer support to arrange exchanges back then. I already posted about this previously in the ToG thread.

That desperate to get new copies out there huh? :lol
 

onken

Member
A 300k sample size is going to give us a hell of a lot better indication of sales that then the handful-a-year press releases for individual titles we currently get, I'm all for it.
 

Koren

Member
Dalthien said:
300k is a phenomenal sample size (and far, far more than would ever be necessary) if you can be sure that the sample size is truly random and representative of the full set of people.
It's definitively NOT phenomenal. It's a bare minimum if you want to know sales at thousands/tens of thousands resolution (and I think many DD releases are not hundred thousands sellers). See explainations above.

Again, the basic idea is simple: if you ask people in the set "did you buy game X", you need enough "yes" so that you can estimate the total sales.

If a game sold 2k to a population of 20M system owners (1 out of 10.000) and your set is 100k, you'll get (assuming the set is random), a mean of 10 positive answers. With 10 answers, the incertaincy is still high. Should you have a subset of 20k owners, the mean is 2 positive answers, and the variance (and % error) is insane.

Of course, if a game sell 500k, there's no problem with a 2k set... But are sales in the x.xxx range or in the xxx.xxx range? So if it's 300k owners of different kind of devices (DSi, PSP, Wii, PS3, iPhones), don't expect any precise results under 10k sales.


And, I'd like to stress the fact that random set is not optimal. You'd rather choose a set with a decent size of each kind of consumers, and know how many kind of consumers you have in the whole population. A minority in an inhomogeneous population can be a majority of buyers of a given title, so you'd want a big chunk of people in the minority so that the estimation is correct, even if the minority itself represent a very low percentage of the total population.

Of course, you can correct the results with multipliers (e.g. multiply estimations by 2.5 for popular shmups because shmups fans will all buy the new R-Type, and the shmup fans are a very minority in the random set) But you'll still need a larger set, and the multipliers are difficult to estimate (and the correction non-linear)
 
Top Bottom