Game Reviewers are Lazy

[Scott Tortorice]

Found this interesting:

AIAS President: 'Game Reviewers Are Lazy'

Speaking on the current state of video game journalism and criticism, Academy of Interactive Arts & Sciences president Joseph Olin has expressed his belief that "game reviewers are lazy" due to a reliance on scores and the desire to review a game quickly.

"When I just see a score, whether it's a Metacritic score or 5 stars or 4 thumbs, that doesn't tell me anything," Olin told Shacknews during an extensive interview, to be published in full at a later date. "I am never surprised when there's as much as a 40% or 50% variance between Metacritic numbers and user numbers."

"My pet peeve is that game reviewers are lazy," he said. "Not all, but in terms of the reviews [something like] 'This game isn't as good because let's compare it to that game over there and that game was great.' Who gives a, you know, bleep?"

He also went on to say:

"How can you review a game, how can you give a comment about a game like Grand Theft Auto IV, that has 40-plus hours or more of gameplay, if you've only spent 2 1/2 to 3 hours playing it," Olin asked, describing his query as a "challenge" to the industry.

"It would be like reviewing a movie but only seeing the opening, first reel. I don't think that's fair, or is it accurate," he explained.

Also a good point. But, as I see it, the only way to get out a timely review of a large game is if the devs give review copies or access to MMOs weeks in advance of release. Of course, weeks before release usually means weeks of bug hunting and fixes for the devs, thus the review will not reflect the actual release copy. This issue is certainly a conundrum with no easy solution.

Not only that, but while this above point is a fair one, I think it is usually pretty obvious if a game is good or not within a few hours of gameplay. If a game doesn't dazzle the player quickly, how many gamers are going to stick around for hours more in the hopes it gets better? Not many. And why should they?

 

[dwardzala]

Reviewers, in my experience, do not value the same things a game that I find important. Many are dazzled by the eye candy, cool explosions and sound effects - which are nice, but what I am really interested in is deep strategic game play. Now, granted, an FPS or MMO is not my cup of tea (maybe these can be reviewed after a couple of hours) but for more strategic games, a couple of hours usually doesn't give a good impression of the depth of the game. It will identify one that isn't very deep.

 

[Scott Tortorice]

Reviewers, in my experience, do not value the same things a game that I find important. Many are dazzled by the eye candy, cool explosions and sound effects - which are nice, but what I am really interested in is deep strategic game play.

That's true, but to be fair, I cannot remember the last time I read a review where the reviewer said the graphics were great but the game was awful, and the proceeded to give the game a good score. On the contrary, I have read reviews where games with good graphics get savaged anyway because the gameplay is not there. I think most reviewers treat good graphics as icing on the cake, but not the cake itself.

 

[Dr Zaius]

I agree with Scott. Graphics get talked about a lot precisely because they are so important. And it bears mentioning that "graphics" is a pretty broad term that can mean different things to different people. The interface, for example, is part of the overall graphical presentation of a game.

As I have said before, the human body has five senses: smell, taste, feel, sound, and sight. If you are playing games using the first 3, you may want to consider counseling. And audio, while important, is certainly not the heart of software design. Which leaves sight.

The primary method by which a human being gets information from a computer software application is via its graphical presentation. Little wonder reviewers spend so much time talking about what a game looks like and its overall production values.

Of course, graphical presentation is less important in, say, an office spreadsheet application. And if you're happy with a game that looks like a spreadsheet, more power to you. It goes without saying that movies are all about the story and the special effects and "graphics" are simply eye candy. But anyone calling for a return to claymation and black and white film is probably not qualified to write movie reviews for general consumption.

 

[jayedub7423]

I have wondered sometimes when reading a review in a magazine if the reviewer actually played the same game I did.

I actually worked for a publisher once for little over a month a long while back, and the head of marketing actually told me that he believes that most reviews found in magazines are created from the press release information given by the publisher. Needless to say, that company does not exist anymore.

 

[Scott Tortorice]

The primary method by which a human being gets information from a computer software application is via its graphical presentation. Little wonder reviewers spend so much time talking about what a game looks like and its overall production values.

Too true. My favorite example of this is World in Conflict. Let's be honest, WiC is not all that innovative for an RTS. Sure, there is no base building, something that was initially considered a radical design decision (which, in and of itself, is a commentary on the gaming industry), but the gameplay is pretty standard for a RTS. So what sets it apart from its brethren? The graphics. The game is a visual wonder of modern warfare and despite all the protestations to the contrary, I suspect the average wargamer would sell his soul if there was a grog game that had its graphics engine. After all, isn't that what set the original CM apart from other games? Graphics count in gaming, just as they do in movies.

That is not to say that solid gameplay is not a factor as well. After all, I'm a chessplayer. A chessplayer. Heck, we only have a two color palette! But the gameplay is there and that is what makes the game. Good gameplay is vital. You can have all the wiz-bang graphics in the world, but if the game is not fun, who cares? Who wants to be bored with stunning visuals? Who wants to sit through the gaming equivalent of Battlefield Earth? Graphics are the icing on the delicious cake. If the cake isn't appetizing, nobody cares about the icing.

And it bears mentioning that "graphics" is a pretty broad term that can mean different things to different people. The interface, for example, is part of the overall graphical presentation of a game.

Also true. After all, DEFCON does not have stunning visuals but was critically acclaimed nonetheless for its "minimalist" approach. The original CM won accolades from numerous sources too. Sins of the Solar Empire does not have cutting-edge visuals, but has done well too. GalCiv2?

I think the ultimate point is that the graphics have to fit the genre.

 

[Dr Zaius]

Also true. After all, DEFCON does not have stunning visuals but was critically acclaimed nonetheless for its "minimalist" approach.

True. But the production values in DEFCON are pretty good, even though it's not the kind of game that needs a fancy 3D engine.

The original CM won accolades from numerous sources too.

Quite so. In this case the "eye candy" was more than just eye candy. This design changed the way war gamers view games by helping them better visualize the events on the battlefield. If Combat Mission were reduced to a 2D game with crappy icons, it just wouldn't be the same game. And anyone who thinks grogs don't care about graphics need look no further than the huge number of high quality mods that were offered to re-texture the game's vehicles and buildings. Great effort on the part of the community, but that should have been the standard out of the box.

Sins of the Solar Empire does not have cutting-edge visuals, but has done well too. GalCiv2?

Sins is not cutting edge, but it's not mediocre either. It's production values are middle of the road.

I think the ultimate point is that the graphics have to fit the genre.

Agreed. That said, war gaming is the one genre where production values are unacceptably bad as a general rule. As a player, you may be willing to give these games a pass because you like the core gameplay. Fair enough. But a reviewer needs to be more critical and call it like it is.

Bottom line: reviews should not be written with genre fan boys in mind, they should be written for the consumption of the general public.

 

[dwardzala]

Agreed. That said, war gaming is the one genre where production values are unacceptably bad as a general rule. As a player, you may be willing to give these games a pass because you like the core gameplay. Fair enough. But a reviewer needs to be more critical and call it like it is.

Bottom line: reviews should not be written with genre fan boys in mind, they should be written for the consumption of the general public.

Well said.

The problem I run into with game reviews is that the reviewer's definition of deep strategic game play and my definition don't usually match. I suspose it has to do with the fact that most gamers agree on what looks/sounds cool, but there are broad differences of opinion on what good core gameplay is (at least in the more strategy oriented genres). Unfortunately, for me, that reduces the usefulness of most reviews.

 

[Dr Zaius]

The problem I run into with game reviews is that the reviewer's definition of deep strategic game play and my definition don't usually match. I suspose it has to do with the fact that most gamers agree on what looks/sounds cool, but there are broad differences of opinion on what good core gameplay is (at least in the more strategy oriented genres). Unfortunately, for me, that reduces the usefulness of most reviews.

I hear what you're saying. And the answer to that for some gamers is to seek out reviews on specialty sites. But in my own experience, many of these sites and their reviews seem to suffer from fanboyism that infects the articles. That, and the fact that many of these type of gamers suffer from pronounced tunnel vision due to the fact they spend most of their time in the bubble of their chosen niche, thus their reviews tend to lack any type of relevant context with the wider game community. YMMV on that.

In my humble opinion, there is no such thing as a completely fair, accurate or "correct" review. Although reviewers should certainly keep all of those things in the forefront when writing the article, at the end of the day a review boils down to one person's opinion. If you, the reader, derive useful information about the relatvie value of a game even though you disagree with the author's final conclusion, then that's probably a decent review. The way I see it, the review's true purpose is to provide the reader with information about the game, not to convince anyone that the reviewer's opinion is the final word.

 

[dwardzala]

Its a shame that generally all I have to go on is reviews in deciding which games to purchase (demo's aside because many are either decontented or time constrained such that I don't feel confident that I have the total picture of the game).

Game purchases for me are pretty much a crap shoot, where I find what I consider a gem about 1 time in 5.

 

[normal_man]

Why not read reviews from serveral sources? That's what I do most of the time.

 

[jayedub7423]

I think that reading more than one review is needed these days. I find that sometimes one review will touch on something that another review doesn't, and vice-versa. Really helps to give you a broader picture of the game.

 

[dwardzala]

I certainly agree that more information is better, but even when I've read multiple reviews on a game that leads me to believe its good, I am disappointed more often than not.

 

[jayedub7423]

I know that this is an old thread, but I had a thought just recently, and that is how long should one play a game before giving a review.

Recently I was listening to a podcast from the Australian Gamer, and one of the topics was the game Too Human. The two hosts stated that they have not played the game, but the guest host Yahtzee from Zero Punctuation did. In fact he did a video review of the game a few weeks ago, and he didn't care for the game very much.

But, he also admitted to only playing roughly two hours of the game. And my knee jerk reaction was that is just not enough time invested to give an accurate review. Then upon thinking about Too Human, i realized that there is probably very little difference in gameplay weather 2 hours in or 20 hours.

Too Human may be one of those games that doesn't require a lengthy gameplay to review. So with that said, does the type of game determine the length required to give an accurate review, or is it just up to the reviewer them self to determine that?

 

[Scott Tortorice]

Interesting question; one I suspect has more than one answer. For me, I can usually tell the quality of a game in under two hours. Now, mind you, I will continue to play the game for longer than that before I write a review, but I don't aim for a set amount of time.

In general, I believe a game should operate just like a book: the first few minutes need to hook the player. If a game fails to do that, it has not succeeded in my opinion. Mind you, that is not saying that the game will not later prove to be worthwhile, but rather that game X will probably not succeed in garnering a large audience because few casual gamers will stick with it long enough to find out how good it may actually be.

As for these reviewers who play a game for 90 hours just so they can discover that there is a bug in level 203...well, that is rarely necessary in my opinion. If you have to play a game for 90 hours to get a good feel for it, there is something seriously wrong. Of course, this excepts certain genres such as MMOs (as in Conan, sometimes devs are hiding a lack of content behind a firewall of early quality), but for most games, I would say ten hours should be more than enough to discover the depth and quality of a game.

 

[jayedub7423]

I actually like Too Human, so I suspect that is part of the reason why I had my initial reaction.

But you're correct about the hook aspect of a game. I just started playing through The Force Unleashed, and after less than a couple hours, I already had the feeling that I was not going to like the game much. But I have this strong desire to play through the game anyways for a few reasons, achievements and because it is Star Wars after all!!

 

[jwb3]

One complicating factor is that it's possible for an otherwise very good story-based game to be undermined by a weak ending. I remember people discussing various games (though not which ones) a decade or so ago and complaining that you finally won the game and practically just got dropped back to the DOS prompt.

Okay, so maybe it was two decades ago. :laugh: :laugh:


Seriously, though... I think in some cases it might have been so drastic that it would have rated a full point off, review-wise (assuming a 10-point scale).


Then again, maybe this is an old problem. Maybe these days, the eye candy that made for a stunning and dramatic "you won the game" sequence is part of the routine play of the game?


John

 

[Michael Dorosh]

One complicating factor is that it's possible for an otherwise very good story-based game to be undermined by a weak ending. I remember people discussing various games (though not which ones) a decade or so ago and complaining that you finally won the game and practically just got dropped back to the DOS prompt.

Okay, so maybe it was two decades ago. :laugh: :laugh:

Unlike back in the 50s when you won a game, and they gave you a ticker tape parade in your honour.

M-1 Tank Platoon's campaign was kind of lame in that regards - you got a victory or defeat screen showing either an M-1 rolling into Berlin (a static picture, not an animation), or an M-1 lying destroyed on the battlefield - and that was it. No viewable stats, no campaign summary, just a lousy screenshot basically saying "thanks for playing, you rock (you suck)".

 

[Navigator]

The solution to this problem, at least for me, is to:

1. find a reviewer or two whose gaming interests correlate with my own to some degree, in order to lend a bit more credibility to their opinions as it applies to my own personal preferences/interests...

2. climb off the hype train for games and wait a bit after release before buying, to let the dust settle and the truth to come to light...

3. find a site or two where actual gamers provide honest and unfiltered opinions on the games...

4. demo games whenever possible...

With all of the games I would like to purchase, I'm not wealthy enough to blow $40-60 on a game that I'm going to hate, because there are several others I'm passing on in the process. I need to hit the bulls-eye with my gaming dollars, so I'm a bit more careful and less trusting of gaming reviews, whether they're "lazy" at their job or not.