DRM and other factors in Game Reviews
by Rohan

After my recent article on Silent Hunter 5 and Ubisoft’s fascist, broken DRM system, a large number of comments kept me unsure just what to do with regard to buying the game or not. Sure, I wanted to check it out – see how it panned out as a game, and see just how it compared to earlier Silent Hunter titles, but I also began to slide into the category that seemingly almost all the commenters is in. I just didn’t want to throw down cash for a title that I wouldn’t even properly be able to enjoy with Australia’s net connectivity being what it sometimes is.

In the end, the question became moot, as I was given a copy to review for GameArena. It went up the other day – you can give it a read here. If you don’t want to read the whole thing, here’s the gist – my last paragraph:

In all, the game seems to alternate between a wonderful, engrossing experience and a frustrating, incomplete one – and while we can hope that upcoming patches fix many of these (often niggling, sometimes major) problems, for the moment, Silent Hunter 5 remains a interesting but partially-broken game.

But reviewing Silent Hunter 5 presented me with a series of problems. Do I bring up the DRM? Is it reasonable to? What about the general state of the game? If a game is already being patched to remove some of the problems found in it at launch, should this factor in to a review?

In the end, the biggest problem was that Silent Hunter 5 is horribly broken. The most unique example is this: a fairly important feature of a submarine simulator, is the ability to use a bottom-facing active sonar tool to measure the depth under the keel of your boat. Now, this isn’t some fancy sim addon to make it easier for us poor guys at our keyboards. This is a very important feature that’s existed on real submarines since before World War 2.

See? There it is. ‘C’ on the picture. The navigator uses it. Now, if you’re diving a submarine, you want to know how much water you have under you, lest you slam into the bottom at speed. All sub sims have had this feature. It’s fairly bloody important. But Silent Hunter 5, at first glance, didn’t seem to. I could find literally no way to measure my depth under keel. I kept thinking I was missing something, until I began to read forum posts on subsim.com – and found that, indeed, the feature was missing.

Or, to be more precise, it was there – same as before, but had no keys bound for it. Let me clarify that even further again – it wasn’t just ‘unmapped’ and needed to be bound to something at an options screen. The only way to use the tool was to open up the config file manually and enter not just the keyname, but the actual hex value for the key itself!

Now, I was only too happy to do this to stop me accidentally slamming into the bottom of the ocean like a drunkard onto a pub floor. But obviously, this is not something most people would ever think to look up, and less again would actually go playing with config files.

This effectively amounts to reviewing a game with a mod, something that surely isn’t fair. I could have easily gone to subsim.com and download the already impressive array of mods and user-made patches to correct some of the huge array of flaws in the game as-released. But again – this is not a reasonable thing to do.

These kinds of things are obviously something that needs to factor into a review. But what if it was fixed in a patch? These days, games often need patching before they can even begin. Heavy Rain, when I first inserted it into my Playstation, required a 250MB+ patch before it would even load. (Anybody remember when console games couldn’t be patched? Nobody seemed to have a problem releasing games that worked right off the disc back then…)

So which of these things should be factored into reviews? Would it be fair of me to have, say, marked down Silent Hunter 5 for using a DRM system that I had an ethical issue with? Probably not, unless it directly affected my gaming experience (which it, fortunately, did not, thanks to my net connection being fairly stable).

But what about patches? The game already had a patch out on release, and more are obviously in the works. Hell, some games whose developers actually went under kept getting patched by first AND third parties for years after their release.

Should a game be reviewed based on its potential once patches become available? Or based on the state the game is in on the original media? It’s easy enough to say, “as released” for any number of reasons – not the least of which being that some people never patch games. They buy them from the store, take them home, and never once google for patches.

But these days, the “original media” might be Steam, or some other download service. If you get a brand new game on steam years after its release, you get the latest version right off the bat. Or in the case of Ubisoft’s new system, it enforces the application of the latest patches before it’ll even run your game, much like an MMO.

Or what about marking a game down based on ethics? If you have an arbitrary hate of either a publisher due to their business practices or simply a prior negative experience with them, should you factor that into your review?

Certainly not. You’re reviewing a game – not the method it was delivered to you, or the political views of its developers & publishers. And yet this is common now. I have already read quite a few reviews that rake Silent Hunter 5 or Assassin’s Creed 2 over the coals for their DRM, even if the actual scheme did not seem to affect the reviewer’s experience with the game in a negative way.

And so, in summary of these musings, I have decided to set down some laws which I intend to follow as closely as possible when writing reviews.

  1. A game should be reviewed without reference to its distribution method or other meta-game factors, as this may be different for individual users. The ‘gaming experience’ that is being reviewed begins once the game has loaded, not from the moment you insert the disc into your computer.
  2. A game should exist in a vacuum – no emotional or personal references to the publishers or developers should be made, as for most people just who the developer or lead designer is and what he did to a reviewer at a conference once in Toledo is not – and should not be – a factor in the game’s playability. No more than (in theory) an actor’s political activism or religious beliefs should not affect a film critic’s thoughts on the performance or film.
  3. Games no longer have a set and clearly defined and immutable “state at time of install”, but do have a “state at the time of reviewing”. As such, the latter state should be factored in carefully during a review – but this should not preclude a review being amended, or a game being “re-visited” at a later date in the event that this changes through patches or changes in hardware.

Not everybody is going to agree with me on these points, obviously – but these are the basic guidelines I intend to stick to. I may critique or rant about these other factors in separate articles, but I will not factor them into my review unless they directly affect the gaming experience.

Tags: , , , , , ,

10 Responses to “DRM and other factors in Game Reviews”

  1. Adam O'Neill says:

    I agree with points 2 and 3 but not 1. When I reviewed Gyromancer you can bet that I made mention of the fact that I needed to resize the window and restart my machine to even get the program to run, and was then subsequently forced to play the game in window mode when my preference is to play full screen.

    If a developer or their distributor release a game that has buggy code (in the instance of Gyromancer, which has still not seen a patch) or obtrusive DRM, then a fair review would include any negative experiences caused by these issues. In the case of your SH5 review then obviously the DRM caused you no problem, so it would be unfair to mention it, however having to bind a vital function of the game yourself is a negative experience, so it would be unfair not to factor either the inconvenience of having to patch the game yourself OR simply playing the game as is and if the lack of floor-depth sonar ruins your game experience then that’s how the game gets reviewed.

  2. Rohan says:

    I’d agree with that to some degree. But I still maintain that your review should begin once the game is loaded – as your issues may not be present for everybody otherwise. And if you literally can’t get the game running, then that’s certainly worth noting! But if the installer is a bit kludgy, for example, or it uses the annoying Starforce system and you find that frustrating, that shouldn’t be a factor – for the simple reason that it might be (and often is) later released on Steam or some other medium where the CD-protection system vanishes out the window, as does the installer.

  3. Jeremy says:

    I think it’s important to consider these factors as they’re part of the product for which gamers put down their hard-earned. They should know what they’re getting into. Is there a way of legally playing SH5 without dealing with Ubisoft’s crap? No? Well, then. That’s a factor in choosing something else that doesn’t make the experience frustrating.

    It’s like when a game is extra-expensive. The reviewer got his copy for free, but the purchaser is being asked to put up more money for this product than usual – shouldn’t the reviewer offer an opinion on whether it’s worth the extra, even though it didn’t actually affect him at all? As long as it’s made clear that the game would be worth buying at X price point, or once Y DRM was removed, or Z bugs were patched, then everyone’s nice and informed and the whole process was worthwhile.

  4. Rohan says:

    I think you missed my point. :-) The price, the distribution method, and the copy protection are variable factors. You cannot guarantee that they will remain the same. I could have, say, ragged on the DRM scheme used in Dangerous Waters or the fact that it came on CDs instead of one DVD, but I’d have been awfully embarrassed two weeks later… when they removed it entirely, and released a digital version on steam, nullifying both problems for many purchasers.

  5. Jeremy says:

    No, I got your point. But they affect purchasers NOW. They might change later – well, hell, update the review. You are reviewing the game at a particular time, and your review should reflect the reality at that particular time.

  6. Rohan says:

    I simply disagree, there. I feel a game review should stand the test of time, and still be relevant down the line. I was considering this while re-reading reviews from a 1998 issue of PC Powerplay the other day.

  7. Adam O'Neill says:

    I totally agree with your point on installers, none of them are perfect and even Steam games need the occasional verify and restart after the installation is complete. What I’m talking about are lasting issues that do affect game play after the setup is complete, eg with Gyromancer I only mention the method of getting the game to run because it meant that I was forced to play in window mode, which I can’t stand.

  8. Kinsley says:

    I think you should talk about it all. These side issues are, after all, information that is relevant to me as a gamer. I would certainly like to know about anything that might effect my enjoyment of a game that I have paid for.

    If you have the advantage of publishing criticism online instead of in dead trees, you can even write about these issues in separate articles and cross-link them to the reviews, if you want to keep the reviews focused and pristine. You could even update reviews when patches come along that significantly change the game.

    For my part, I put a much greater value on a source of reviews and gaming news that I can trust to talk openly and honestly, even if I disagree with some of the opinions shared, or some of the scoring criteria.

  9. Rohan says:

    Oh, I’m all for talking about it. Even, if it’s really important, IN the review (and, like you said, linking elsewhere).

    But I don’t think it should affect the review score of the game.

  10. Brian McMahon says:

    It is an interesting debate. The gaming magazine publisher, both online and traditional paper are very wishy-washy about DRM. Occasionally a reviewer will be obviously incensed by some type of DRM and blast it in a review. Or there maight be a feature about DRM issues. But you never see the DRM consistantly dealt with in review. I would say that the DRM issues, if any, should put in the review as a sidebar or an appendix. I believe the review should stand the test of time for the content of the game, but anything that interferes with the player ability to play the game might render the review a moot point, at least for that player.

    It doesn’t matter how good the game is if you can not stand a particular DRM.

Leave a Reply