Flare to Scott Tortorice re: Your CMSF v1.04 Review

KG_Jag

KG Vice Kommandir
Joined
Aug 5, 2002
Messages
1,782
Reaction score
180
Location
New Braunfels, TX/Reno, NV
Country
llUnited States
Hi Scott--this thread was invited (at least in part) by Don in this thread:

http://forums.gamesquad.com/showthread.php?p=946885#post946885

Please be sure to read my post #21 and Don's post #36. With that background, I have some questions about your CMSF review and related matters. They are:

1. For many of us your narrative in your review was difficult to reconcile with the final rating you gave the game--7.0/"Good". It is also evident from reading your "Concluding Comments" in the review that you struggled with this issue. Specifically how (e.g. what key aspects the game in v1.04) did you arrive at the number score and rating you gave the game?

2. I noticed that your review did not address human to human play. As a veteran CM x 1 player, you know this is an important feature of not only CM (all versions), but most other war games as well. Why did you not address this in your review? If you had considered it, what impact, if any, would it have had on the point score and rating you gave to CMSF in your review?

3. For purposes of reaching the point score and rating in GS reviews, is there any mandatory process or structure that must be followed, or are these things left completely (at least in most cases) to the discretion of the individual reviewer?

4. The GS scoring system provides for 10 possible ratings of a game by a reviewer, depending on what point score is given by the reviewer. See it here:

http://www.gamesquad.com/index.php?option=com_rating&id=20

After 40 reviews over 14 + months, a full 50% of these ratings categories have remained unused. The point scores issued for these 40 reviews cover a range of just 35% of the available point range. Will you and Don revisit the current scoring/rating system and see if can be more informative/meaningful to the gamer?

Thanks in advance for your time and effort.
 

Dr Zaius

Chief Defender of the Faith
Joined
May 1, 2001
Messages
8,903
Reaction score
518
Location
The Forbidden Zone
First name
Don
Country
llUnited States
I can provide some insight on question #3.

3. For purposes of reaching the point score and rating in GS reviews, is there any mandatory process or structure that must be followed, or are these things left completely (at least in most cases) to the discretion of the individual reviewer?
We used to have a system like you describe, which is very common on many game sites. If memory serves, reviewers rated each game in 6 categories and these categories were then combined to reach the final score.

This type of system has advantages, in that it helps give reviewers a structure to follow and automatically generates the final score for them. However, the system also has weaknesses. With a system like that, it means that games with less than stellar graphics and sound (wargames?) will have a difficult time ever reaching the higher ratings. Obviously, that means that the majority of wargames would fall into the mediocre range even if they have outstanding gameplay and overall design.

Now some might argue this is a good thing and the way it should be. But the reviewers helped convince me that they should be the ones deciding what the final score should be, not an artificial formula. The argument centered around the idea that no formula, no matter how well designed, is appropriate for all types of games in all situations. I eventually agreed, and we decided that the reviewer should make this determination.

Obviously, this approach gives reviewers more control and more leeway to go in either direction. But since the review itself is already nothing more than the reviewer's opinion after playing the game, we felt the score should also be at the reviewer's discretion.

The editors do, of course, give the reviewers some guidance to help them adhere to our standards, but the final decision is up to them. So long as the review is fair and informative and the final score is supported by the text, the score is up to the individual reviewer.

Every game site uses a different methodology to achieve it's scores, and all of them have advantages and disadvantages. In the end, it is always the text of the article that matters most.

P.S. I don't know if Scott will choose to enter this discussion or not. Generally, the reviewers let their articles speak for themselves as there will always be some who disagree with a particular review. I will notify him that this thread is here.
 

ER_Chaser

Member
Joined
Aug 5, 2002
Messages
2,962
Reaction score
1
Location
NYC
Country
llChina
Don, I remember there were 6 individual scores displayed for the readers as well? I was out of the loop for a long time, rule changed since?
 

Dr Zaius

Chief Defender of the Faith
Joined
May 1, 2001
Messages
8,903
Reaction score
518
Location
The Forbidden Zone
First name
Don
Country
llUnited States
Don, I remember there were 6 individual scores displayed for the readers as well? I was out of the loop for a long time, rule changed since?
Yes, we did away with that formula and the reviewer simply assigns an overall score. That said, reviewers are encouraged to cover all these important areas within the body of the review.
 

Scott Tortorice

Senior Member
Joined
Nov 18, 2003
Messages
7,663
Reaction score
115
Location
The shadows
Country
llUnited States
Hey KC,

Flare spotted! :smoke:

As Don pointed out, I don’t usually respond to threads that debate the merit of a particular review for two reasons:

a) Attacking/defending a review is part of the fun of reading them and I do not want to inhibit or enflame the debate with my presence.

b) The review is what it is. All that I had to say about a particular game has already been stated in the review, leaving me with little to add.​

Having said that, I will be happy to respond to this thread because everyone has been very respectful and have raised some good points as well. So here goes:

“1. For many of us your narrative in your review was difficult to reconcile with the final rating you gave the game--7.0/"Good". It is also evident from reading your "Concluding Comments" in the review that you struggled with this issue. Specifically how (e.g. what key aspects the game in v1.04) did you arrive at the number score and rating you gave the game?”

Giving a numerical score to a review is always problematic for me because reviewing a game is a very subjective process; there are no true quantitative values that can be added up at the end of a review to determine a verifiable score (i.e., how do you calculate the “fun factor?”). As such, I ultimately determine a score based upon an aggregate impression of everything that was good or bad about a particular game. As I indicated in my review, CMSF was a very difficult game to evaluate because there was quite a mix of qualities that ranged from needing improvement to nicely done. It might interest you to learn that initially I was going to assign it a score of around 6.5, but then I played the game some more and assigned it a score of 6.7. By the time I had reached the end of my review I ultimately decided that it really deserved to be elevated from “fair” to “good” with a 7.0 as that was the general feeling (or 'impression') I was getting the more I delved into the game (and just as importantly, started to appreciate CMSF for what it is and not what I wanted CMx2 to be). Again, this is a completely subjective determination and I realize there is plenty of room for debate and argument, but I thought 7.0 was about right (and truth be told, I was kicking around a 7.4-.7.5 near the end of the process but decided against this score because it just didn’t seem like a good fit).

As you can see from the above, scoring a game, at least for me, has a great deal to do with intangibles and a simple emotional reaction to the title. It is sort of like Supreme Court Justice Stewart’s remark that "I can't define pornography, but I know it when I see it." Same idea with me. I cannot define a 7.0 game (or a 9.0 or a 6.0), but I know one when I play it. :)


2. I noticed that your review did not address human to human play. As a veteran CM x 1 player, you know this is an important feature of not only CM (all versions), but most other war games as well. Why did you not address this in your review? If you had considered it, what impact, if any, would it have had on the point score and rating you gave to CMSF in your review?

I am a predominately solo gamer and my reviews reflect that preference. Unless a game is billed as a multiplayer-centric title, such as World in Conflict, my reviews don’t generally delve into PBEM or head-to-head play. Furthermore, I had played CMx1 both as a solo and PBEM experience and never saw any significant distinction between the two modes vis-à-vis the overall quality of the game. With that in mind, I saw no reason to expect a significantly different experience by playing CMSF as a PBEM game. As such, that aspect of the game was not covered.

3. For purposes of reaching the point score and rating in GS reviews, is there any mandatory process or structure that must be followed, or are these things left completely (at least in most cases) to the discretion of the individual reviewer?

Don handled this one.

4. The GS scoring system provides for 10 possible ratings of a game by a reviewer, depending on what point score is given by the reviewer. See it here:

http://www.gamesquad.com/index.php?o...m_rating&id=20

After 40 reviews over 14 + months, a full 50% of these ratings categories have remained unused. The point scores issued for these 40 reviews cover a range of just 35% of the available point range. Will you and Don revisit the current scoring/rating system and see if can be more informative/meaningful to the gamer?


The only thing I would contribute here is that I look forward to the day when numerical scores are abandoned by the industry and reviews are allowed to stand on their own merits. What’s the point of going to all this trouble writing a review if people are only going to pay attention to the final score? I find these numerical values contribute nothing to the reviewing process other than to act as a sort of 'cheat sheet' for the industry so that it can quickly market a game as "good" or label a game as "bad" based upon largely arbitrary scoring.


Thanks in advance for your time and effort.

My pleasure!
 
Last edited:

KG_Jag

KG Vice Kommandir
Joined
Aug 5, 2002
Messages
1,782
Reaction score
180
Location
New Braunfels, TX/Reno, NV
Country
llUnited States
Thanks Scott.

Maybe one day reviewers and gamers will be able to enter into more substantive and mature--but still sometime passionate--discussions of reviews and the games that spawned them. It's my belief that the reviewers, gamers and the sites hosting the reviews would all benefit.

A major reason I had asked about human vs. human play is that this has been a major issue and question discussed on this (SZO at the time) forum since the first word of CMSF and the CM x 2 engine. Many of the folks here had indicated that a working PBEM was a make or break feature with regard to their interest in CMSF and the CM x 2 engine. Based on the virtual absence of CMSF ladders anywhere on the net, I assume that it was and remains a problem.
 

Scott Tortorice

Senior Member
Joined
Nov 18, 2003
Messages
7,663
Reaction score
115
Location
The shadows
Country
llUnited States
Thanks Scott.

Maybe one day reviewers and gamers will be able to enter into more substantive and mature--but still sometime passionate--discussions of reviews and the games that spawned them. It's my belief that the reviewers, gamers and the sites hosting the reviews would all benefit.
That's a point worthy of consideration. For me, I believe that the people who read the reviews should not feel like 'big brother' is watching and prepared to jump in at a moments notice to defend his review. As someone who reads a lot of reviews, I understand a great deal of enjoyment is had by debating them with others and I don't want to chill that process (for better or worse :)).

A major reason I had asked about human vs. human play is that this has been a major issue and question discussed on this (SZO at the time) forum since the first word of CMSF and the CM x 2 engine. Many of the folks here had indicated that a working PBEM was a make or break feature with regard to their interest in CMSF and the CM x 2 engine. Based on the virtual absence of CMSF ladders anywhere on the net, I assume that it was and remains a problem.
That is a very good point and in the light of it I sort of now wish that I did take a closer look at the PBEM portion so I could definitively answer your question (after all, this is my job as reviewer). I apologize for that oversight. However, I suspect there are no nasty surprises lurking with CMSF as a PBEM game as I have come across some interesting PBEM AARs over at Battlefront.com.

Thanks for the feedback!
 
Joined
Aug 26, 2003
Messages
1,884
Reaction score
4
Location
England
Country
llUnited Kingdom
No disrespect, but who cares what reviewers say about games anyway? I've never taken reviewers seriously and I never will because I learnt long ago they usually can't be trusted and are often slipped a few dollars by the game producers to only say nice things..;)
That's why my offer to more than one game company to playtest their stuff was turned down because they know I can't be bought, and that if their game stinks i'll say so straight out.
PS -CMSF has great potential but until its fully developed it'll remain just an average game, we don't need reviewers to tell us that..:)
 
Last edited:

Bertram

Member
Joined
Nov 23, 2003
Messages
699
Reaction score
0
Location
Nevada
Country
llUnited States
No disrespect, but who cares what reviewers say about games anyway? I've never taken reviewers seriously and I never will because I learnt long ago they usually can't be trusted and are often slipped a few dollars by the game producers to only say nice things..;)
That's why my offer to more than one game company to playtest their stuff was turned down because they know I can't be bought, and that if their game stinks i'll say so straight out.
PS -CMSF has great potential but until its fully developed it'll remain just an average game, we don't need reviewers to tell us that..:)
I have to disagree here Mick. Unless you own every game from every area in your gaming interest you must be making decisions based upon 'reviewers' comments. Now these reviewers might not be paid and your comments are likely directed to paid reviewers. But even with paid reviewers, their comments can be beneficial to most of us. Certainly if most/many paid reviewers dislike a game or at least rate it in the lower 50% of 'their' typical rankings look out. Individual features are often well described and you can learn of certain game traits. I'm not so cynical to think all their comments are paid off. I'm more worried that as gamers, they want the next new one to be great, and might be looking through some tinted glasses.

You say, "CMSF has great potential but until its fully developed it'll remain just an average game, we don't need reviewers to tell us that..:)" So who tells you that? Do you just buy it and find out for yourself?

Usually, I'll wait, read the paid reviews and the comments/reviews from players, then venture out and purchase what looks interesting. I didn't do that with CMSF and this time got scalped.
 

Okimaw

Member
Joined
Apr 10, 2006
Messages
535
Reaction score
0
Location
Cree Territory
Country
llCanada
"Usually, I'll wait, read the paid reviews and the comments/reviews from players, then venture out and purchase what looks interesting. I didn't do that with CMSF and this time got scalped."

You didn't get scalped Bertram, you just "don't get" all the nuances of asymetrical warfare or was that you don't know how to use hot keys, or was it you only play as Germans on flat maps? Did you try red vs red or blue vs blue or special red vs plain blue? :clown: Jk Battlefront dude is delusionsal or just a jerk, I got suckered inot buying a piece of crap too.
 
Last edited:

Dr Zaius

Chief Defender of the Faith
Joined
May 1, 2001
Messages
8,903
Reaction score
518
Location
The Forbidden Zone
First name
Don
Country
llUnited States
No disrespect, but who cares what reviewers say about games anyway? I've never taken reviewers seriously and I never will because I learnt long ago they usually can't be trusted and are often slipped a few dollars by the game producers to only say nice things..
No disrespect? Interesting thing to say right before you call someone a liar and say they can't be trusted. Whatever.

For the record, this site has as of this writing never done any advertising with Battlefront Games.
 

Geordie

CM Moderator
Joined
Jan 27, 2005
Messages
2,111
Reaction score
13
Location
Scotland
Country
llUnited Kingdom
No disrespect, but who cares what reviewers say about games anyway? I've never taken reviewers seriously and I never will because I learnt long ago they usually can't be trusted and are often slipped a few dollars by the game producers to only say nice things..;)
That's why my offer to more than one game company to playtest their stuff was turned down because they know I can't be bought, and that if their game stinks i'll say so straight out.
PS -CMSF has great potential but until its fully developed it'll remain just an average game, we don't need reviewers to tell us that..:)
The above is nothing more than your usual making a statement to get a certain reaction. Your basically accusing all reviewers of being dishonest without 1 shred of proof.

Obviously based upon the world according to POS.....
 
Top