Third Party Publisher Rate of Publication Question

Gunner Scott

Forum Guru
Joined
Jan 27, 2003
Messages
13,737
Reaction score
2,669
Location
Chicago, IL
Country
llUnited States
This I think is a huge problem with many of the newer scenarios, balanced but dreadfully boring or are Alamo scenarios. Waiting 5 or 4 turns for your forces to be engaged is not much fun nor are scenarios that rely on players to take a building or one small area on the map. A few examples of really boring or Alamo scenarios come to mind:
FW Siberian Diversion
SP Armor at Kumch'on (this was more of an alamo scenario)
AP Misty Morning (something)


scott


I would hope a good scenario would get that benefit. If a scenario is balanced but boring, it's nearly as unplayable as an unwinnable-for-(side) one.
 

von Marwitz

Forum Guru
Joined
Nov 25, 2010
Messages
14,358
Reaction score
10,207
Location
Kraut Corner
Country
llUkraine
How many products does Desperation Morale playtest before publishing their reviews? Do they even have face to face playtests of new products?
Desperation Morale does not claim to play the products before its reviews. It does not need to playtest, because this needs to be done before publication of the product itself.

While I agree that actually playing products before reviewing them would produce even deeper feedback, some experience that Pitman has will allow him to identify some obvious glitches that any experienced player would take note of. The 'specialty' of Mark in this regard is that he might be the only one systematically buying all that stuff. Furthermore his reviews don't focus upon the scenarios themselves but rather on elements that might give clues on their quality: Errors in printing, credits to playtesters (or the lack thereof), changes (or the lack thereof) to subsequent editions of a product, the quality of the previous version of which might have been made apparent by people who actually did play it, etc.

Basically, this is the most what one could expect if one man alone is doing the job. It is valuable to get information on these tidbits that Pitman provides on DM. And that is the very reason why his site is viewed as a valuable resource by many.

von Marwitz
 
Last edited:

Michael Dorosh

der Spieß des Forums
Joined
Feb 6, 2004
Messages
15,733
Reaction score
2,765
Location
Calgary, AB
First name
Michael
Country
llCanada
Desperation Morale does not claim to play the products before its reviews. It does not need to playtest, because this needs to be done before publication of the product itself.

While I agree that actually playing products before reviewing them would produce even deeper feedback, some experience that Pitman has will allow him to identify some obvious glitches that any experienced player would take note of. The 'specialty' of Mark in this regard is that he might be the only one systematically buying all that stuff. Furthermore his reviews don't focus upon the scenarios themselves but rather on elements that might give clues on their quality: Errors in printing, credits to playtesters (or the lack thereof), changes (or the lack thereof) to subsequent editions of a product the quality of the previous version of which might have been made apparent by people who actually did play it, etc.

Basically, this is the most what one could expect if one man alone is doing the job. It is valuable to get information on these tidbits that Pitman provides on DM. And that is the very reason why his site is viewed as a valuable resource by many.

von Marwitz
No arguments with any of that, save my original point in the breakaway thread. If one of the criticisms a reviewer is making is that a product wasn't playtested - my contention is that there would be no way to know that unless you actually played it yourself. (This is barring an admission by the publisher that they didn't playtest it, which has happened from time to time.)

To reiterate, I think you are correct in what you are saying re: the DM site, and I too find it for the most part valuable if for nothing else than as a catalogue.

But Mark is the one asking about how to identify if something has been playtested or not. I don't think you can do that with a simple chronology or comparing to a list of release dates, for the reasons that Jim, Chris and others have elucidated very well earlier in the thread.

Put another way - if Mark wants to criticize the playtesting of a product, that's fair game, but he'll need to put the work in to do it fairly. I don't doubt he can pick up "obvious glitches" (as you put it) from a general scan of the scenario card, and probably better than most of us. I do think, however, you can't speak to thorough testing of something unless you submit it to some actual scrutiny yourself.

It might beg the question of just how you would determine if something has been playtested. What does that look like? Spelling errors or glitches on the scenario card? Well, that would be more a lack of editing and proofing, though it could indicate it passed through very few hands in the playtest as well.

Wouldn't "lack of playtesting" only reveal itself in things like not having enough turns for an attacker to complete the VC? Or having lopsided win-loss results after multiple playings? I mean, isn't that what playtesting is supposed to prevent? And how would you be able to identify that kind of evidence with a simple scan of the scenario card? If it was that easy, I have to believe there would be no reason to ever complain of poor playtesting.
 

von Marwitz

Forum Guru
Joined
Nov 25, 2010
Messages
14,358
Reaction score
10,207
Location
Kraut Corner
Country
llUkraine
my contention is that there would be no way to know that unless you actually played it yourself. (This is barring an admission by the publisher that they didn't playtest it, which has happened from time to time.)
Well, at least one indicator (but not proof, of course) for lack of playtesting would be the absence of giving credit to playtesters. There is no reason not to do it if you have playtesters:

They put their time and effort in it, so it is the decent thing to do. Furthermore it will also be beneficial from a commercial point of view. If you can give credit to a number of playtesters whose names are well known in the community, this will surely not be detrimental to the sales that product might generate.

A second indicator for the absence of playtesting would be the following: You have an umpteenth edition of some scenario which has an uneven ROAR record based on a somewhat substantial number of playings. If the umpteenth edition of that scenario is still the same as the first edition except for artwork, title, etc. but the units, VC and SSR remain (almost) identical, then it is very likely that no playtesting went into the improvement of that scenario.

von Marwitz
 

Brad M-V

Senior Member
Joined
Jul 20, 2012
Messages
646
Reaction score
330
Location
British Columbia
Country
llCanada
Absence of evidence is not evidence of absence.

JR
It is in ASL, otherwise the 2nd Edition's introduction wouldn't tell us to: "Play by what the rules specifically allow or disallow, not by what they don't specifically prohibit." :)
 

xenovin

Elder Member
Joined
Aug 7, 2010
Messages
1,983
Reaction score
1,165
Location
Skynet
First name
Vincent
Country
llUnited States
I playtest with a guy who has designed scenarios since the 1980s and he spots things right away before we even play the thing! Usually it’s conflicts between SSRs, VC, setup areas, or how you can break a scenario by using (or not using!) some given equipment. So a very experienced player can pick up on some of these issues that should not be there in a play tested published scenario.

I would just add in other threads Mark has stated a scenario needs to be playtested at least 30 times to work out all the bugs. That seems pretty high to me unless you keep having major revisions. So by Mark’s math, you can see how much time it takes just to put six scenarios together and thus his inference on a group putting out more than one pack a year.

It might beg the question of just how you would determine if something has been playtested. What does that look like? Spelling errors or glitches on the scenario card? Well, that would be more a lack of editing and proofing, though it could indicate it passed through very few hands in the playtest as well.
 

Sparafucil3

Forum Guru
Joined
Oct 7, 2004
Messages
11,335
Reaction score
5,071
Location
USA
First name
Jim
Country
llUnited States
I would just add in other threads Mark has stated a scenario needs to be playtested at least 30 times to work out all the bugs.
I know of great scenarios, which have be well received by the ASL community, that were playtested no more than twice. Over test them and you begin to get confusing and conflicting feedback. And lets not even begin to talk about the Singling debate on who to balance it for (top players or lesser mortals). -- jim
 

jrv

Forum Guru
Joined
May 25, 2005
Messages
21,998
Reaction score
6,206
Location
Teutoburger Wald
Country
llIceland
That is correct. And for that reason I wrote:

"Well, at least one indicator (but not proof, of course)..."
Right, and your comment was just a dog whistle, saying that although the dullards don't take it as proof, everyone should.

JR
 

Pitman

Forum Guru
Joined
Jan 27, 2003
Messages
14,104
Reaction score
2,371
Location
Columbus, OH
Country
llUnited States
I would just add in other threads Mark has stated a scenario needs to be playtested at least 30 times to work out all the bugs. That seems pretty high to me unless you keep having major revisions. So by Mark’s math, you can see how much time it takes just to put six scenarios together and thus his inference on a group putting out more than one pack a year.
Where did I say that?
 

Pitman

Forum Guru
Joined
Jan 27, 2003
Messages
14,104
Reaction score
2,371
Location
Columbus, OH
Country
llUnited States
Absence of evidence is not evidence of absence.

JR
A great statement to make in a vaccuum. Not very applicable here, where the custom is to acknowledge and credit playtesters. There was a reason Critical Hit stopped listing playtesters on its products, for example. That's because its playtesting crew left.
 

Pitman

Forum Guru
Joined
Jan 27, 2003
Messages
14,104
Reaction score
2,371
Location
Columbus, OH
Country
llUnited States
I know of great scenarios, which have be well received by the ASL community, that were playtested no more than twice. Over test them and you begin to get confusing and conflicting feedback. And lets not even begin to talk about the Singling debate on who to balance it for (top players or lesser mortals). -- jim
Those alleged scenarios got lucky. Anybody who releases a scenario that has been playtested no more than twice is simply throwing the dice.
 

boylermaker

Senior Member
Joined
Jan 22, 2012
Messages
581
Reaction score
526
Location
Virginia
Country
llUnited States
Absence of evidence is not evidence of absence.
Finally, the fodder for the Bayesian-vs-Frequentist flame war that this forum deserves. I'm with Mark on this one--absence of evidence is often weak evidence of absence, but the stronger the custom of crediting playtester becomes, the stronger evidence not-crediting-playtesters becomes.
 

Sparafucil3

Forum Guru
Joined
Oct 7, 2004
Messages
11,335
Reaction score
5,071
Location
USA
First name
Jim
Country
llUnited States
Those alleged scenarios got lucky. Anybody who releases a scenario that has been playtested no more than twice is simply throwing the dice.
Well, they have done it multiple times with the same success. If the winners of ASLOk aren't reliant on luck, they aren't either. JMO, YMMV. -- jim
 

von Marwitz

Forum Guru
Joined
Nov 25, 2010
Messages
14,358
Reaction score
10,207
Location
Kraut Corner
Country
llUkraine
Right, and your comment was just a dog whistle, saying that although the dullards don't take it as proof, everyone should.

JR
Sorry, pal. But that is plain b**t.

I said what I said and not what you try to put in my mouth.

von Marwitz
 

von Marwitz

Forum Guru
Joined
Nov 25, 2010
Messages
14,358
Reaction score
10,207
Location
Kraut Corner
Country
llUkraine
It's still a game, right?
So goes the rumor...

But there are also mutterings that some of us have been found lying on their backs beneath a sheet of thick white and blue cardboard having lost their marbles until they thought they'd seen a pair of dice clattering by showing a 1,2 upon which the cardboard vanished all of a sudden and they stood upright scratching themselves what the heck had just hit them.

von Marwitz
 
Top