The NPD Group registered a blip in videogame industry news last week with a press release based on its “Gamer Segmentation 2012” report. Alarming numbers, including a 5% loss of gamers in the United States, appeared in media outlets all over the internet. But how did the NPD Group arrive at this determination?

According to the press release:

(T)here are an estimated 211.5 million gamers in the U.S. that play video games, a 5 percent decline, or loss of close to 12 million gamers, when compared to 2011.

The report then goes on to discuss the year-over-year changes in size of the six gamer groups outlined by the NPD. Two groups (the Mobile Gamers and Digital Gamers) experienced growth, while the other four groups (Core Gamers, Family+Kid Gamers, Light PC Gamers and Avid PC Gamers) all shrank, most noticeably in the Family+Kid group.

The NPD Group

I haven’t seen the NPD Group’s “Gamer Segmentation 2012: See the Face of Gaming” report, and I suspect that most if not all media outlets that reported these alarming conclusions haven’t seen it either. That’s because this report is a market research product that is for sale. A one-page ad can be found here. The NPD Group is self-characterized as “dedicated to equipping companies like yours with the consumer market research, point-of-sale data, and expert analysis required for creating winning business strategies.” While the videogame industry is just one of the many market research areas that the NPD covers, it’s this edifice of the company, the “Group,” that I’ve followed for years and have to critique.

It Must Be Raining

Several publicly traded companies such as EA, Take-Two and GameStop release sales numbers regularly in reports for investors and prospective investors. From this information, it’s well known that traditional sales of videogames and hardware are down this year when compared with last year. The drop-off in traditional game sales isn’t surprising and can be attributed to a variety of factors.

The lack of hardware sales would seem to be alarming, but this is misleading. The 360/PS3/Wii generation is in its seventh year, and we haven’t even had a hardware revision like a new “Slim” model for any of them this year. The new 3DS XL was just launched, but the 3DS and PSVita sales are a sub aspect of the industry (not the life blood). Many analysts are quick to point out that a new console from Sony or Microsoft would translate to huge hardware sales, but for what benefit? Microsoft is still raking in money from existing 360 owners in the form of Xbox Live. Launching a new console just to attract those same users seems like something that can wait. Console hardware often begins as a money loser, and retailers like Best Buy and Gamestop have next to no markup on the consoles themselves. The segment of gaming that is most sickly according to the NPD is the Family+Kid segment, which will be getting the new Wii U this holiday season.

A Digital Reality

While the industry may seem to suffer from something akin to millions of gamers dropping off the map this year, the reality is that the NPD Group cannot really be taken seriously in its analysis. While the Group can chase around public numbers for boxed game sales, it’s left to estimate digital sales. Microsoft, Sony, Nintendo, Apple, Google, etc. all have a brand new industry unto itself with digital game sales. (Xbox Live fees and Xbox dashboard ads alone are jaw droppers.)

Take Valve, for example. With its Steam platform, Valve predates (2004) most other digital media platforms and has inspired a horde of competitors on the PC alone. Valve is a private company, and discloses nothing to the NPD.

This is how I imagine any exchanges between the NPD group and Valve’s managing director, Gabe Newell, would proceed:

“Hi, you have reached Gabe Newell’s voicemail. I can’t take your call right now, because I just had a pizza delivered… to my yacht. If our positions were reversed and you had a choice between enjoying the pizza that was just delivered to your yacht or having a stupid phone exchange, which would you choose? Thought so. Peace out.”

Valve is valued between two and four billion dollars, and there’s a huge disconnect between its dollars-and-cents business and the NPD’s reports.

In a Vacuum of Numbers

If the movers and the shakers of the videogame industry don’t deign to share any of their closely guarded sales numbers (never mind budgets) with the NPD, how can the NPD declare a 5% drop (12 million gamers) from last year? Simple. The NPD Group conducted…

“an online survey… fielded in March 2012 to members of NPD’s online panel. Responses for kids under age 13, as well as some kids in the 13-15 age range, were captured through surrogate reporting (where the parent answers on behalf of the child). The data is weighted and representative of the U.S. population ages 2 and older. The study is based on 8,488 completed surveys.”

I have a significant appreciation for survey-based statistics, and have spent time analyzing data with SPSS. Nevertheless, I can’t take the NPD Group’s surveys as evidence of a real loss of 12 million gamers in the U.S. simply because the NPD arrived at a 5% difference in numbers between the survey this year and last year.

That the report defines six separate groups of gamers creates concerns over how those distinctions were achieved. How is an “Avid PC Gamer” kept distinct from a “Core Gamer” or a “Digital Gamer”? Aren’t all “Mobile Gamers” also Digital Gamers? How can a person fall into to one of these categories this year, but not next?

From a consumer analysis point of view, I might argue that there are really only two groups: those people who buy their own games, and those who are dependent on other people (like parents) to buy those games for them. Even focusing on those two groups ignores the huge volume of business done with free-to-play models. What if a kid’s parents buy him $400 in iTunes cards this year, which he doesn’t spend until next year? In other industries, these issues are normally filtered by the hard data of new car sales numbers or box office receipts, for example.

As digital game sales and services have ascended, the credibility of the NPD Group with regard to the videogame industry has reached an all-time low. The NPD’s declaration of a loss of 12 million U.S. gamers based off its surveys is embarrassing. I’m embarrassed first by the press release, and secondly that it was so widely reported as hard data. Perhaps the videogame industry should try to stay more in line with other industries that readily supply media with nuanced data sets? Sadly, the normal tendency for companies to consider any public disclosure as a leak has the side effect of leaving various people grasping at straws to draw conclusions, no matter how fundamentally unsound.