Should You Trust Early Reviews Of 'Metal Gear Solid V: The Phantom Pain?'

The initial batch of reviews for Konami and Hideo Kojima’s ridiculously anticipated Metal Gear Solid V: The Phantom Pain hit the internet over the weekend, and those of you leaning to the skeptical end of the spectrum may have noticed something peculiar about them: The final scores are exceptionally high, yet many of them lack the detail you’d expect to accompany a sprawling open world game several years in development.

That’s because they originated from a controlled review event hosted by Konami. And while a review event isn’t inherently evil by definition, and shouldn’t be grounds to outright dismiss these early reviews, there are some important reasons you should digest them with a heavy helping of proverbial grains of salt.

What we know for certain is that the game’s publisher, Konami, hosted at least two review events for games press – one in the US, and one in the UK. For review events, the publisher typically flies journalists to a location for several days, and will cover all travel and accommodations. In my opinion there’s no ethical dilemma there. Most tech and games writers I know are completely capable of publishing honest criticisms with or without having their travel expenses taken care of. It’s a reality in this business, not a perk.

The genuine cause for concern lies in the way this review event was carried out. According to journalists from RockPaperShotgun, GamesRadar, and several posting at NeoGAF, reviewers were expected to binge-play Metal Gear Solid V in a span of 4 to 5 days. Here’s a revealing summary of the experience from GamesRadar’s Dan Dawkins:

“Based on the UK boot camp, I know of only one reviewer (who was able to play for six days) who has seen enough of the game to deliver a meaningful perspective… and I can't even explain why for fear of spoilers. In my boot camp, reviewers were charging through missions wearing the chicken hat (which makes you invisible) almost completely ignoring Mother Base and all the side-ops in a race for the 'end'. Will it score high? I mean, duh, but I don't feel the boot camp was sufficient basis to offer my views on Kojima's intentions and MGS5's abiding legacy. At times, the boot camp felt like being gifted a bottle of Macallan 1946 whiskey in a frat house and being told to chug, chug, chug.”

Why does that concern me? Because a majority of the reviews published thus far probably didn’t complete the game, much less give proper attention to Mother Base, a crucial (and admittedly fun sounding) base-building component that allows you to progress further into the storyline and do some cool stuff like R&D for new weapons, improve your team, and improve yourself. Some reviews indicate it’s the sleeper star of Metal Gear Solid V, while it’s likely that others didn’t even touch it.

The other elephant in the room is the suggestion that influential reviewers were charging through a stealth game effectively in God Mode, wearing an accessory that makes them invisible. This completely negates their ability to speak to the game’s difficulty, grind, and other related factors that could either deter or attract potential gamers.

RockPaperShotgun’s on-hand report makes matters worse, claiming that reviewers were under strict NDA to share only “information that was deemed necessary by Konami higher-ups.”

I completely understand why Konami would want to keep reviewers on a short leash. Having spoilers leaked online before millions of Metal Gear fans could experience the narrative for themselves would be less than ideal. Several of my colleagues in the industry have also told me this is why Konami is refusing to send review code to journos who weren’t at the event, until the day of release.

But yet again, that means the majority of reviews you’re going to read today were conducted in a very controlled environment, and on a pretty harsh deadline considering the game’s open-world nature and wealth of side quests, base management mechanics, and online functionality. Review events for major AAA releases like this aren’t uncommon. What troubles me personally is how several of these reviewers were given the option to go invisible in a stealth-driven game, and likely didn’t have time to touch several key ingredients that make up the overall Metal Gear Solid V experience.

Imagine reading a review of a Dark Souls game by a journalist who spent the entire game invincible.

For my part, I want to both read and write reviews that represent the average gamer, and I fear that these don’t. Does all this mean Metal Gear Solid V is a bad game and shouldn’t be earning these exceptionally high reviews? Not at all. Nor do I believe we should cast any nasty glances at the journalists who attended and devoted their entire weeks to eating, sleeping, and breathing this game. But I would caution everyone to give more weight to reviews that are published after the game is released, versus ones that were written based on Konami’s review events.

Pay more attention to “reviews-in-progress” from guys like Kotaku’s Jason Schreier.

And for the record, I attended one such event to preview Konami’s Metal Gear Rising: Revengeance. It was a professional affair and I came away from it holding Konami’s PR team in the highest regard. There’s no suggestion of foul play here, but I strongly disagree that this was the right environment for a massive, open world game to be reviewed.

What’s your take? 

Metal Gear Solid V: The Phantom Pain releases on all major platforms including PC on September 1, 2015.