What Brink tells us about reviews and you

Me
Tuesday, May 10, 2011

Full disclosure: if I have a bias in regards to Brink, it’s in favor of. We’re in the multiplayer FPS doldrums, and Brink was the only hope of giving us an antidote prior to Battlefield 3 this November. The prospect of Brink failing to attract the attention of my gaming buddies, who are the only people I like to game online with, displeases me mightily. If Brink isn’t good I’ll accept that…but I’m not sure whose review to trust because I know how and when these Brink reviews were written.

The game journalists reviewing Brink either played the game at a review event, or received their review copies last week in the mail. The multiplayer matches they played were against each other, or against developers from Splash Damage. In either case they were not playing Brink in the same conditions in which you, the intended audience for these reviews, will be playing the game, which is really the only accurate way to report on the performance of a multiplayer shooter title.

 

Joystiq’s Brink review is extremely well written. It speaks to recent conversations about whether game critics need to discuss rules and mechanics more often. Griffin McElroy’s criticism is all about mechanics: not enough ability unlocks, problematic choke points, lack of matchmaking, weak AI, etc..

Some of his criticism sounds unassailable. When I interviewed Richard Ham for my Brink feature that ran on Gamasutra, I thought it odd that the game wasn't going to have party-based matchmaking, that one had to form Fire Teams after joining a match. Multiplayer-friendly shooters certainly benefit from lobby systems. Uncooperative AI in a team-based shooter is problematic.

But I also don't really care about any of this. I want to play Brink online, not with the AI. I'm willing to deal with the lack of a lobby system or party-based matchmaking. Getting parties together in the Bad Company series has never been smooth, and the game's inability to support groups larger than 4 has been a consistent irritation. I work around it.

It's McElroy's gameplay concerns that worry me, but I'm not sure if I can trust them. Maybe there actually ARE enough ability unlocks. I certainly tend to favor one class over another in class-based shooters, and don't care if I level all the classes up. Perhaps the parkour movement allows for flanking approaches to those choke points which McElroy hadn’t discovered. Perhaps leveled-up, experienced teams, with access to all their stat boosts and equipment, can coordinate to take down entrenched enemies in ways that McElroy didn’t have time to discover in his review session.

I have no doubt whatsoever that McElroy has accurately represented his experience playing Brink in good faith and with professional decorum…but the conditions in which he had to write his review were horrible. Playing with strangers. With no experience with the maps. With no real time to practice. On both latter points, a "12-hour review session" (which is not a solid 12 hours of gamplay) is nothing for serious multiplayer shooter gaming.

These are not the conditions in which to judge a team-based FPS title…but they’re the conditions McElroy was handed.

We could also discuss the 1UP review, written by Taylor Cocke, which I thought was even better. It sounds as if the author had appreciable FPS experience. Again, we have the bottleneck complaints, and this time the S.M.A.R.T. movement is more adequately addressed, but it sounds like network issues played a huge part in the parkour movement not working. I'm therefore hesitant to accept the statements about it not providing tactical advantages to deal with bottleneck issues.

At the end of the 1UP review, the following statement is given:

Editor's Note: This review was conducted before a day-one patch which is said to address, "sporadic visual glitches at distance, texture pop-up and some minor networking issues." We will play a few rounds with the patch applied and update the review text accordingly.


I reached out to my web of video game journalists to ask why their outlets insist on publishing release-day reviews of titles, when the quality of those reviews might benefit greatly from just waiting a little while. William O'Neal, the Editor-in-Chief of Best Buy's @Gamer magazine (for whom I've written several times) gave me an answer:

Simply put, editorial outlets are businesses and readers can be impatient and not-terribly loyal. And well...they have every right to be impatient as well as not-terribly loyal. Furthermore, games sell the vast majority of their copies within the first few weeks of their release. Add to that, there's also a diminishing return when it comes to reading about a newly released game. Game A comes out Tuesday and you're thinking about buying it. You read everything you can on Monday and Tuesday and probably make your purchasing decision. The number of people who are gonna read about it the following Monday or Tuesday has dropped off precipitously. So if you're an online editorial outlet and you do the "right thing" and post your review of a so-called "broken game" a week later (after it's been "fixed"), relatively speaking, no one is gonna read it.

Russ Pitts, the Editor-in-Chief of The Escapist, wrote this column on the state of the video game industry. And he basically echoes what the statement above suggests to me: this is the audience's fault.

I recently had a collection of game journos describe the mass market video game journalism audience as shallow, impatient, completely uninterested in critical inquiry, and almost illiterate, dangling on review scores to gleam meaning from our reviews. That is depressing as hell. I also can't refute it. These are experienced game journos telling me this, who understand their audience and their markets much better than I do at this point in my career.

A few editors I know lament all the talking that game journos engage in regarding how our work is done, versus just trying to do better work and addressing any shortcomings in our field thusly...but if this is the state of affairs, why bother? Why should we wait until Friday to get some Brink playing time in with veteran shooter gamers to see how the game play actually winds up? Or to discover how people take advantage of the parkour movement? Or to see if the network code gets patched such that the issue removes itself, and we can focus on other things?

In a world where pre-orders rule and the audience just wants to know whether they should pick up a game the day it comes out, would anyone have patience for that sort of thing?

The value of discussing how game journalists do their job, is to try and educate the audience. If the audience demands their reviews on day one, fine...but the audience has to understand the price to be paid accordingly. It may mean not getting accurate reviews because the audience forces game journos to operate in piss-poor review conditions in order to publish copy which is premature.

Game journos have to try and approximate how online play for shooters is going to work rather than actually knowing how it does work when all of you are playing the game. Game journos have to rush through titles, passing up on loads of tonal and textural content to get a baseline read of a game, and then issue an opinion that also has to sound experienced and authoritative.

I didn't get Brink today. I really wanted to, but $60 is a lot of money, and I'm plagued with doubt. I'll wait for a used copy instead, so that if the current reviews are accurate, I can return it to GameStop for a full refund, and no harm done. But I hate that my doubt has been seeded by reviews which had to be rushed in order to satisfy an audience which doesn't know any better than to give game journalists the time they need to produce the best, most accurate reviews they can. And I don't want to stop hoping that sometime soon, the audience will wise up.


Dennis Scimeca is a freelance writer from Boston, MA, who got his start writing about video games right here on Bitmob. He has recently contributed to Gamasutra, G4TV's The Feed, and Joystick Division, and has also contributed to GamePro, the Escapist, and @Gamer magazine. He blogs at punchingsnakes.com and randomly opines on Twitter: @DennisScimeca.

 
Problem? Report this post
BITMOB'S SPONSOR
Adsense-placeholder
Comments (21)
May 10, 2011

I just had a discussion about some friends on this topic. After I informed them that it was a more multiplayer heavy game, the first thing they asked was how I was going to forumlate a review with nearly no one online playing.

I delayed my review a bit until 12am eastern time so I could do this properly. It didn't change my opinion on the game too much, but I realized something about being a freelance journalist, and that being first is very important so you feel like you have to rush and put your review on Reddit, Digg, N4G, and other sites like that  in order to try and make a name for yourself and please the Press Representatives with a same day review.

 

May 10, 2011

Also, that was excellently written.

Me
May 11, 2011

Thanks much, Christian. :)

Lolface
May 11, 2011

I guess I'm part of the problem.

I don't really care about what conditions the reviewers are under while writing their reviews. All I really care about is whether the game is fun, or not. Being a gamer since the 16-bit era, in my experience, it only takes about 15 minutes to determine if a game is any good. I understand that this isn't the case for all games, however, if I start in mediocrity, then that shapes my entire view of the game. Can an accurate review be written given only 15 minutes of gameplay? For some games, yes, because they're terrible. However, most games do need more time. 12 hours? In my opinion, that's enough. Delving further and further into a bad game only reveals how bad a game is. If they didn't like in 2 hours, they aren't going to like it at 4, 6, or 12 hours of playtime, and it seems like both reviewers really didn't like the game.

Me
May 11, 2011

I can't tell you that you should care, but it's tempting. Review conditions have the potential to radically change the reviewer's perspective, and you're the person who winds up paying the price for that.

Take Dragon Age 2, for instance. The review copies for that game were very late in arriving to many members of the press, such that they had to blow through the game in 35-40 hours, sometimes over a single weekend.

The game has a good 65 hours' worth of content. My first playthrough clocked almost 69 hours.

So, the Dragon Age 2 reviewers skipped almost 1/3 of the game's content, and made their best assessment of what they saw. In this case, I think that worked out in EA's favor, as I've gone through the ENTIRE game, and my assessment of the game is much less kind that most of the critics'.

I suspect that if reviewers had been given more time with DA2, that review scores across the board would have been lower, as they'd have had more time to uncover outright bugs, wonky combat encounters, bad writing, etc. The sheer weight of experience that much more badness would have been reflected in the review scores.

How many people, who bought DA2 on the basis of a "generally favorable" Metascore for the game, found out they didn't like the game, based on the fact that they actually played through the whole thing and thus got to see what the critics didn't?

I don't put solid, critical stock in user scores on Metacritic any more than I put stock in Metascores, but look at the average user score for DA2 on MC versus the Metascore. That's a hell of a difference, isn't it?  Where there's smoke, there's usually fire.

I want to say you should care about review conditions, because you're potentially making decisions about spending money that you ostensibly work hard for, on video games for $60 a pop if you buy new, and close to release, based on the results of those reviews. But maybe you're the gambling type. ;)

Default_picture
May 11, 2011

I found Russ' column very disturbing, and not because I agree with him. I resent the elitism exhibited by Russ and other game journos who consider their primary audience morons and treat them accordingly. The great unwashed masses just can't comprehend the genius of Russ' calling them an idiot. Rather than delving into the root of gaming's "troubles" (which I think are vastly exaggerated--we're in a new golden age as far as I'm concerned), they blame gamers.

Here's a newsflash: most gamers don't care about the "games as art" debate, they don't read up on every facet of the industry like we do, and they don't take the industry as seriously as do game journos. They just want to play fun, compelling experiences. It's not because they're uncultured pond scum. It's because they have other more pressing concerns. Some people are sports fanatics and spend every waking moment educating themselves on the minutiae of stats and theory. We're into gaming.

If we listened to Russ (and those who think like him), we'd buy only the games officially anointed by critics and we'd rarely buy on launch day (to allow critics time to write more informed reviews).

Me
May 11, 2011

Russ's column was droll humor. I don't think it's meant to be taken literally. It sounds like that's what your doing...or maybe I'm not understanding your response.

Default_picture
May 11, 2011

I took Russ' column as a combination of satire and truth, as did most of the readers seemingly. Even if his angry tone and rhetoric was intentionally over-the-top, he clearly believes at least some of what he's saying, blaming gamers for the industry's perceived troubles.

Me
May 11, 2011

He's not wrong, though. In the end, the consumer ultimately calls all these shots. That's just a statement of fact, isnt it?

It's like democracy. We can bitch about the government all we like, but in the end, we vote these idiots into power. Anytime we like, we can just stop voting, and maybe things would change. Maybe people would get pissed off that the 10% of the people who keep voting are making decisions that affect the 90% who want to see changes made before we proceed...and them maybe that 90% would get angry enough to make the required noise to force those changes into being.

But as long as we keep voting, we have no right to complain. We're part of the problem.

If the audience does like the games being made, they need to stop buying them. If they don't like video game journalism, they need to stop reading it. If they keep buying, and keep reading...they really need to stop complaining. That's how I'd take Russ' point, and I think he's right.

Me
May 11, 2011

And how long until we can edit comments again? This stupid Mac keyboard I'm on right now is causing all kinds of typos I wouldn't commit on a PC. :(

Robsavillo
May 11, 2011

Dennis, I don't understand that comment.... The suggestion that voting with your wallet for a consumer product is an analogy to voting for political candidates is...confused at best.

If enough consumers don't buy games or magazines, sure enough, those developers and outlets go out of business. If enough people stop voting for their president, well, we only concentrate power into the hands of a minority of Americans who continue to vote. The governmen't isn't suddently overthrown because too few people went out to the polls.

Me
May 11, 2011

No, it's not a perfect metaphor. I don't really have the time right now to craft a perfect analogy. :P But my point is that if you participate in a system, you are partially responsible for what that system does.

Gamers complain about the quality of reviews...but apparently won't sit still for reviews that are written after a game comes out, or after a reviewers has had time to thoroughly play it. So they hold some culpability for the state of game reviews. Etc.

Robsavillo
May 11, 2011

Dennis, your suggestion, though, is that the best (only?) way to enact change is to stop participating in the system. And by your own definition, doing so would simultaneously absolve you of responsibility of the staus quo.

In other words, Pitts' article describes systemic issues that a pure market-based approach (abstaining from purchasing games/content) can't address alone.

Default_picture
May 11, 2011

I agree 100%, Dennis. Gaming is a perfect microcosm of the free markets in action. While I admit I may have misread his tone somewhat, it's Russ' implication that gamers are schizophrenic and don't know what's best for them that irks me.

Where's this massive gamers' uprising that has so wounded Russ? As far as I’m concerned, game journos are doing their job admirably—I only wish I’d listened to them regarding Homefront. If gamers wish to continue buying FPS's, as derivative as they've become, I see nothing wrong with that. If not, there's plenty of options, even among the Triple A crowd. And as even Russ admits, gamers want what they want. And if gamers don't want glossy, high-profile shooters like Modern Warfare, there's always quirky titles like Heavy Rain.

I just don't get this angst from Russ, and if it's to be believed, gamers. We're in a new golden age, and consumers' decisions aren't hamstringing (in any significant fashion) the health of the industry.

Default_picture
May 11, 2011

Dennis, maybe I'm just not understanding what Russ is suggesting. Should gamers cease purchasing games on launch day and allow critics time to fully digest the products?  Is it not critics' job to be speculative as to prospective gamers' experiences? That seems to me, the entire purpose of a review (as distinct from criticism, which exists for the sake of intellectual debate). I just can't feel sorry for Russ and others who feel gamers are being unfair by holding game journos to an exceedingly high standard. And yes, I realize the rest of his rant is intentional BS--i.e. defending journos who refuse to participate in the bribery process.

Btw, I share your frustrations, but I always put my comments through a spell checker.

Me
May 11, 2011

I think what Pitts is saying (I really shouldn't call him "Russ" as I don't know the man) is that consumers ask, and the industry produces. Therefore, if consumers don't like what's being produced, perhaps they need to ask themselves why they're requesting or buying it. That doesn't seem unreasonable...but if I go by the description of the mass market games media audience I've been given, then asking people to be reasonable might actually be unreasonable...and this is where I get off the train before my head explodes. ;)

Default_picture
May 11, 2011

Reminds me of the South Park episode "Smug Alert": http://www.southparkstudios.com/full-episodes/s10e02-smug-alert

Can't we enjoy critical inquiry and educate ourselves on gaming without looking down on those who don't?

Robsavillo
May 11, 2011

Funny, Ars Technica made a point to specifically explain why they're not going to review Brink yet: that Splash Damage released no PC code for review pre-launch, PSN is down, and the developer was late in sending out the only available version (Xbox 360) that also requires a day-one patch.

Me
May 11, 2011

I brought that point up to my journo web, Rob. Ars is not a small outlet. They've been invited to judge E3 this year - that speaks of huge numbers if nothing else. And Ars chose not to review Brink yet. Surely that suggests that not publishing a same-day review is a viable option without overwhelming, negative repercussions, doesn't it?

Robsavillo
May 11, 2011

Exactly.

Jayhenningsen
May 11, 2011

So, Dennis, now you're saying that maybe journalists _don't_ have to "try and approximate how online play for shooters is going to work rather than actually knowing how it does work when all of you are playing the game" or "rush through titles, passing up on loads of tonal and textural content to get a baseline read of a game, and then issue an opinion that also has to sound sound experienced and authoritative"?

You must log in to post a comment. Please register if you do not have an account yet.