Separator

Exploring the Lifespan of a Review Score

37893_1338936035999_1309080061_30825631_6290042_n
Wednesday, December 08, 2010
EDITOR'S NOTEfrom James DeRosa

Chase raises an interesting question I hadn't really thought about before: What is the shelf life of a game review?

1UP’s Jeremy Parish recently reviewed Golden Sun: Dark Dawn for the Nintendo DS. He gave it a “B,” though you wouldn’t really know why by reading the text of his review.

Either way, the score seemed to perturb one commenter, who railed into Parish and 1UP for a lack of consistency. In his review, Parish stated that Dark Dawn improves the formula of the first two games. The user claimed that because 1UP graded Golden Sun: The Lost Age, the previous entry in the series, as an “A-," Dark Dawn deserves a score higher than it received. (Former sister publication EGM scored the original game, Golden Sun, quite high as well.)

This is foolish. It’s ridiculous to demand that two people of differing viewpoints stay consistent with each other’s opinions. That said, I think the angry commenter inadvertently stumbled upon an interesting topic: How should readers feel about past reviews years later?

 

Let’s take the Tony Hawk franchise as an example: It’s late 2001. Activision releases Tony Hawk's Pro Skater 3 and reviewers are lapping it up. GamePro awards it a much-coveted perfect score. Everyone’s happy.

Now jump forward to late 2010 and pop THPS3 into your favorite last-gen console. Do you think it’s still a 10?

More than likely, your answer is going to be no. Don't feel ashamed. Better Hawks and better games have come along. The genre has changed and evolved.

But review scores don’t evolve. They’re static numbers that stand stoically against the tidal wave of history. Sure, it doesn’t seem very fair. In a fantasy reality, reviewers would constantly update every score they've ever given as time moved forward, adjusting accordingly for modern developments. But that's just unrealistic and silly.

Also, all perfect review scores are not created equally. Metal Gear Solid 2: Sons of Liberty made its debut in 2001 and received the same grade from GamePro, but I’d hazard a guess that most reviewers would say it's withstood the test of time better than Tony Hawk's Pro Skater 3.

As an industry that works on shoestring budgets, readers can't expect game-reviewing outlets to constantly revise their scores.

I suppose the question is what "10 out of 10" really signifies. Does a perfect score for a PlayStation 2 title equate to a perfect score in the PlayStation 3 era? Or do all scores become null and void when a new console generation launches? How about when a sequel hits? Is a perfect score from 2006 more valid than one from 2005 simply because it's more recent?

It’s a tricky question and one I don’t believe has a definite answer. If anyone has any thoughts, I’d love to hear them.

 
Problem? Report this post
CHASE KOENEKE'S SPONSOR
Comments (13)
Default_picture
December 05, 2010

Reviews are a moment in time, as weird as it sounds. No one should update a score, and no one should have to. When reviewing a game, you're reviewing it based on either the standards of the time or by the merits of the game's strengths against the competition. When trying to "update" a score, you can't possibly put yourself in the same mindset about what that game did at its time. I can't review the original Half-Life now, since I can't place myself in a position to analyze how it moved the FPS forward. I can make a few great guesses, but that's all they are. And even if you had reviewed back then, going back and having a new "last word" on something is difficult.

This is why reviews are primarily a buyer's guide. Word of mouth will decide if, ten years from now, people should still go buy Call of Duty 4. Scores aren't really meant to comment on a game's quality as it fits (or will fit) into the continuity of gaming; it's trying to convince you on whether or not you should buy a game. As for when review scores are no longer valid in a purchasing decision, that depends on the game and the person buying the game.

Me_and_luke
December 05, 2010

Like we often say game X was "good for its time," I think a similar sentiment can be echoed for review scores.  Certainly, if Ocarina of TIme was released tomorrow, it would not have been given all the tens that it received 12 years ago.  But for its time, the game was so revolutionary and genre-defining that it was worthy of its tens.  

It sounds like the same can probably be said of Golden Sun (haven't played Dark Dawn).  The original Golden Sun's gameplay with the unique Djinn mechanic was well-received for its innovation.  Nearly a decade later, however, you're going to need to bring more the table - as well as improve on everything else - to garner the universal acclaim you once had.

As for debating the meaning of a "10", well, that's another discussion for another time. :)

Photo
December 05, 2010

Also there's the Left 4 Dead series. The second one added a ton more content, with only one year between it and the first, and yet, it garnered an average lower review score than the first. There's just a lot more to take into consideration than simple mechanics and camera angles when reviewing a video game. 

There184
December 06, 2010

I prefer reviews that describe how the reviewer experienced the game. Their evaluation should be biased by what they've been exposed to. So I'd expect an American football fan to enjoy Madden more than I do, just as I'd expect someone in the '70s to enjoy Pong more than I do. Their score should reflect their enjoyment of it when they were reviewing it. And that enjoyment can never change.

100_0503
December 06, 2010

It's not realistic to expect people to have exactly the same standards. Different people review games sometimes. It only makes sense that they'd have different opinions. Treating all the reviews at an outlet as one homogenous voice does a disservice to its individual writers.

Profile_pic4
December 06, 2010

Now, I am not an accountant, and in fact I failed an accounting class in college (there's a story there.. for another time).  I don't even play one on TV.

That said, as I read this story, all I could think of was "depreciation" in an accounting sense.

Dscn0568_-_copy
December 06, 2010

I think reviewing standards have probably changed as well. Readers seem a lot more critical about scores and whether the score is too high or not high enough. Then again, gaming sites probably didn't have the same amount of traffic and platforms for reader comment back when THPS3 and MGS2 were around.

37893_1338936035999_1309080061_30825631_6290042_n
December 07, 2010

I totally agree with everyone's comments. Suriel's right: updating scores is a terrible idea for many reasons including time, money, as well as mindset issues.

But some form of degradation is needed. As both you and Bryan pointed out, reviews are a moment in time. But saying that a game is "good for its time" is basically a form of degradation in and of itself. It's just not changing the number.

And Suriel, I totally agree on reviews as buyer's guides. That's what I try to do in my reviews as well. What I hinted at, but never said explicitly in my article is that the idea of review scores is a fundamentally flawed system. I'd prefer the world be rid of them, but with the infrastructure already in place, there's no way they're going away anytime soon.

Me_and_luke
December 07, 2010

[quote]But saying that a game is "good for its time" is basically a form of degradation in and of itself.[/quote]Not necessarily.  Take my OoT example; I meant to imply that the game wouldn't receive its tens if released tomorrow, because games like Twilight Princess and Okami would have now come before it.  It would no longer be a genre-definer.  It isn't necessarily a knock against the game's ability to hold up well, but rather that it simply wouldn't be as innovative if released in this day and age.

And, actually, now this approaches the territory of WHY you're scoring the game as high as you do.  What does that ten really represent?  I still don't believe review scores should ever change (I think we all feel they simply shouldn't exist), but it's sure something to think about.

37893_1338936035999_1309080061_30825631_6290042_n
December 07, 2010

@Bryan the degradation isn't meant to be a slight to the game, but rather to better showcase those titles that come after it. Review scores distort the gaming landscape. Twilight Princess, from a technical aspect, is in every way, better than Ocarina of Time, yet Ocarina has a better review score.

But instead of taking into account what that score means in today's gaming landscape (as you said, it wouldn't receive the 10s it got back in the day,) there are those (such as yourself,) who rightfully state the game was good for its time to perserve its place in gaming history.

There's nothing wrong with this. I personally think Babe Ruth is still the best baseball player to have ever played the game, despite most of his records being shattered. It's a sacred cow situation where tradition and protocol dictate our thoughts rather than logic.

Sorry, this sounds uppity and condescending and that's really not my intention.

Bottom line: assigning a number to a game becomes problematic as time goes on. If it weren't so damn convenient, it would have been done away with long ago. What really matters is what I'd paraphrase from Mr. Alex Martin: a review should reflect the enjoyment the reviewer had of a game at the time they were reviewing it. And that doesn't require a score to muddle things up.

37893_1338936035999_1309080061_30825631_6290042_n
December 07, 2010

I'd also like to qualify that this is not meant to be an analysis of the "perfect score." I believe games that received nines, fives, and even twos are subjectable to the same degradation.

230340423
December 07, 2010

I agree precisely with what Alex said at the top there. What I want to know when I read a review is if the reviewer enjoyed playing the game and why. Way too often, people get wrapped up in minutiae and forget to even mention if they had any damn fun at all.

December 08, 2010

I wrote about this around a month ago: http://www.bitmob.com/articles/rehash-or-definitive-version-future-proofing-review-archives , I kind of go off on a different tangent but I think my solution could work!

You must log in to post a comment. Please register if you do not have an account yet.