Dubious Quality has some thoughts on whether game reviews are disproportionately favorable. He gathered some data on review scores from major sites like 1UP and GameSpot and finds that 50-60 percent were positive. He questions whether the games are really that good, with the subtext being that game review sites are under pressure to deliver positive reviews because they rely heavily on advertising from game companies. There’s no question that business concerns sometimes influence coverage, and industry insiders have gone on record many times confirming instances where that has happened.
However, there might be an additional factor that skews the results of such analysis. That is simply that the sample of games reviewed is probably not representative of all games out there. Take film. Ebert and most major newspapers don’t review all movies, they just review the handful their readers are most likely to be interested in this week.
By the same token, gaming sites review titles that that their readers are most likely to consider playing. Ebert doesn’t review direct-to-video “Little Mermaid 16: Ariel’s Facelift”, and most game sites don’t review “Lawnmower Tycoon” or “Diaper-changing Madness!”.
For example, there are supposed to be 1,000 games (retail and downloadable) out on Xbox 360 by the end of this year. If you figure that 200 games have yet to hit this year, there should be about 800 already out. Yet Metacritic only shows 449 reviews. Here’s the breakdown:
100-90: 14 games
89-80: 91 games
79-70: 123 games
69-60: 95 games
below 60: 122 games
If you take anything below 70 as unacceptable, then 48 percent of the game reviewed fall into that category. If you count 80 and above as where your gaming dollar is best spent, then 23 percent of games are worthy of your consideration. That suggests that as a whole, the industry-wide curve is more of what you’d expect — a very few GREAT games, a good selection of GOOD ones, a big band of MEDIOCRE ones, and a whole buttload of BAD ones.
Finally, I’d point out the obvious point that, as much as reviewers try to quantify their thinking with scores, it’s still an inherently subjective art. There are games I enjoyed immensely that only scored in the mid-70s on Metacritic, and there are some that won universal praise and high scores that I just didn’t get. Same with movies.
In the end, I wonder if we’ll see some sort of shakeout in the review industry where a handful of guys and gals elevate to become the Roger Eberts of the medium. I don’t agree with everything Ebert says, but he’s a consistent enough personality that you can read his reviews and decide for yourself whether or not you’d like a movie, regardless of the direction his thumb is pointing.
