Third Party Publisher Rate of Publication Question

jrv

Forum Guru
Joined
May 25, 2005
Messages
21,998
Reaction score
6,206
Location
Teutoburger Wald
Country
llIceland
There was a reason Critical Hit stopped listing playtesters on its products, for example. That's because its playtesting crew left.
You have evidence or knowledge of that? Because it's a rather serious accusation to make, I would think.

JR
 

Pitman

Forum Guru
Joined
Jan 27, 2003
Messages
14,104
Reaction score
2,371
Location
Columbus, OH
Country
llUnited States
You have evidence or knowledge of that? Because it's a rather serious accusation to make, I would think.

JR
You know their playtesting crew was Dave Lamb, PJ Norton and their crew, right? This is not exactly top secret information.
 

jrv

Forum Guru
Joined
May 25, 2005
Messages
21,998
Reaction score
6,206
Location
Teutoburger Wald
Country
llIceland
You know their playtesting crew was Dave Lamb, PJ Norton and their crew, right? This is not exactly top secret information.
No, I do not know that. I do not track the players who playtest for various groups. And you are sure that's all there were, and they all left?

JR
 

hongkongwargamer

Forum Guru
Joined
Apr 4, 2013
Messages
7,180
Reaction score
5,569
Location
Lantern Waste
Country
llUnited Kingdom
No, I do not know that. I do not track the players who playtest for various groups. And you are sure that's all there were, and they all left?

JR
Don't worry. Likely that all the CH playtesters will see this and announce themselves in support. You'd expect a few of them to be on GS right?
 

Gunner Scott

Forum Guru
Joined
Jan 27, 2003
Messages
13,737
Reaction score
2,669
Location
Chicago, IL
Country
llUnited States
RT did away with playtesters, too much lead time to put out a product. If it does get any playtesting, it is usually from designers submitting their stuff to RT. But I dont think anybody has submitted anything to RT in years. Supposedly he has two designers that send him their stuff, Larry winslow and Mark "tank" Porterfield, Dont really think these guys exist but RT say's he gets designs from them. I am kinda looking forward to CH Leningrad modules, those might not be playtested but I am more into the military history part of ASL and having fun.

If you were a CH playtester, would you admit it? ;) -- jim
 

Tooz

Elder Member
Joined
Sep 17, 2004
Messages
1,338
Reaction score
1,061
Location
York, PA
Country
llUnited States
So I am curious. Imagine an "average" Third Party Publisher in terms of size and, perhaps more importantly, number of associates of that TPP who are willing to playtest for it.

What rate of publication for that publisher would make you start to wonder if its goals were beginning to outstrip its realistic ability to fully playtest and develop its products?

Would more than one product per year start having you a bit concerned? (that is when I personally tend to start having doubts)

More than two products a year?

Do you think a TPP can actually release three fully developed and playtested products per year?

Do you trust that a TPP that released any number of products per year was fully playtesting and developing them beforehand?

Have you ever thought of any past TPP, "Man, they are releasing too many products too fast for them to really have fully playtested them"?

For clarity and for purposes of this discussion, let's describe a "product" as basically equivalent a scenario pack or scenario map/pack with at least 6 scenarios, a HASL-like product, or a number of newsletters during the course of a year that have a total of at least 8 scenarios in them. Let's not worry about very small products, like equivalents to MMP's "bonus packs."
BFP and LFT are the two TPP that I prefer, with a nod going to LC. The first two rarely in memory have released more than two products in a given year. LC is always working on something but their releases do take time.
I trust their playtesting because I usually have done so for all three.
 

hayman

Senior Member
Joined
Jul 7, 2008
Messages
677
Reaction score
266
Location
Sydney
Country
llAustralia
some of us have been found lying on their backs beneath a sheet of thick white and blue cardboard having lost their marbles until they thought they'd seen a pair of dice clattering by showing a 1,2 upon which the cardboard vanished all of a sudden and they stood upright scratching themselves what the heck had just hit them.
I'll have what he's on.;)
 

JoeArthur

Elder Member
Joined
Jan 12, 2016
Messages
1,108
Reaction score
1,067
Location
Broadstairs
Country
llUnited Kingdom
Surely the question is what produces better scenarios? If the answer is playtesting then you are going to want to see some proof of rigorous playtesting.

Is it luck that 3rd RTR In The Rain is on a knife edge or was it created through rigorous playtesting or was it the skill of the designer?
 

xenovin

Elder Member
Joined
Aug 7, 2010
Messages
1,983
Reaction score
1,165
Location
Skynet
First name
Vincent
Country
llUnited States
Mark - my apology as I was not able to find any such quote by you. I did find in your DM blog your discussion on playtesting in "Making Sausages" (2010) where you says there is no magic number but you reference 10-12 playtestings as typical (maybe less and maybe some more).

Where did I say that?
 

Michael Dorosh

der Spieß des Forums
Joined
Feb 6, 2004
Messages
15,733
Reaction score
2,765
Location
Calgary, AB
First name
Michael
Country
llCanada
I would just add in other threads Mark has stated a scenario needs to be playtested at least 30 times to work out all the bugs. That seems pretty high to me unless you keep having major revisions. So by Mark’s math, you can see how much time it takes just to put six scenarios together and thus his inference on a group putting out more than one pack a year.
Mark - my apology as I was not able to find any such quote by you. I did find in your DM blog your discussion on playtesting in "Making Sausages" (2010) where you says there is no magic number but you reference 10-12 playtestings as typical (maybe less and maybe some more).
I think you were closer here than you realize, Xenovin. Mark's actual words from his book are:

So playtest you must. How much? That's difficult to answer...Scenarios should be playtested until they seem ready. But keep in mind that there is a law of diminishing returns. One can never know everything there is to know about a scenario. Playtesting a scenario eight times may reveal 95% of the problems in the scenario. Playtesting it eight more times may reveal 4% more. It may take twenty more playtests to get that last 1% and most designers don't have that luxury.
So in effect, he's saying it might take 36 playtests to work out 100% of the bugs. This is from page 50 of Scenario Designers Guide. So, yes, he did say what you suggested he said - but it was in his book, not his blog.

Put another way, he's also suggesting you can get to a 95% solution much more easily.
 
Last edited:

Pitman

Forum Guru
Joined
Jan 27, 2003
Messages
14,104
Reaction score
2,371
Location
Columbus, OH
Country
llUnited States
Mark - my apology as I was not able to find any such quote by you. I did find in your DM blog your discussion on playtesting in "Making Sausages" (2010) where you says there is no magic number but you reference 10-12 playtestings as typical (maybe less and maybe some more).
Thanks for the apology. I knew I had never said any such thing, because I don't believe that. I've never had one of my own scenarios playtested 30 times, much less had that expectation for others.

I talked a little bit about playtesting in my Scenario Designer's Guide, pointing out that there was no magic number (in part, because every scenario is different--some initial designs are better/more solid than others and may require less playtesting, while some scenario concepts are inherently more complex and may require relatively more playtesting), and pointing out that, beyond a certain point, additional playtests don't even necessarily add much additional value commensurate to their cost in time and effort:

"Scenarios should be playtested until they seem ready. But keep in mind that there is a law of diminishing returns. One can never know everything there is to know about a scenario. Playtesting a scenario eight times may reveal 95% of the problems in the scenario. Playtesting it eight more times may reveal 4% more. It may take twenty more playtests to get that last 1% and most designers don't have that luxury."
 

sswann

Elder Member
Joined
Jan 31, 2003
Messages
2,911
Reaction score
1,379
Location
Middle of Kansas
First name
Steven
Country
llUnited States
As to Balance...
Just my observations...

When I first started designing, I used to worry about balance.
But as I matured and gained experience, my criteria changed.
Now days I have only two questions for playtesters.
The first is... "Is the scenario FUN?"
the second is ... "Did it go to the last turn?"
If I get positive results for those two questions, then the scenario is finished as far as I am concern.
Perfect Balance is nice, but can take the Fun out of a scenario.
History is also great, but players should remember that the designer has only the history available to him at the "time of designing".
Some of us cannot afford large libraries (as I live on SSI mostly and the US Gov does not pay retirees well).
 

Jazz

Inactive
Joined
Feb 3, 2003
Messages
12,188
Reaction score
2,739
Location
The Empty Quarter
Country
llLithuania
The first is... "Is the scenario FUN?"
the second is ... "Did it go to the last turn?"
Going to the last turn is effectively as good a balance as a scenario can achieve.

I would also add that there are scenarios that have different balance based on the experience and skill of the opponents.

Relatively inexperienced players require longer and larger scenarios for there to be a balance between attacker/defender wins. As the skill of the players increases, the amount of time and forces required to strike that balance decreases.
 

Mister T

Elder Member
Joined
Aug 12, 2006
Messages
4,204
Reaction score
1,680
Location
Bruxelles
Country
llFrance
Going to the last turn is effectively as good a balance as a scenario can achieve.
This is not enough. Going to the last CC or winning/losing by 1 VP/EVP would be optimal. Of course it's hard to achieve from a design point of view.
 

sswann

Elder Member
Joined
Jan 31, 2003
Messages
2,911
Reaction score
1,379
Location
Middle of Kansas
First name
Steven
Country
llUnited States
It would be great if I could design a scenario that went to the last DR/dr...
BUT... I ain't that good.
 

jrv

Forum Guru
Joined
May 25, 2005
Messages
21,998
Reaction score
6,206
Location
Teutoburger Wald
Country
llIceland
It should go down to the last DR/dr and then continue as a brawl in the hallway/aisles/street. It's only good enough if the po-po have to be called. It should not result in a slap fight or hair-pulling.

JR
 
Last edited:

bprobst

Elder Member
Joined
Oct 31, 2003
Messages
2,532
Reaction score
1,437
Location
Melbourne, Australia
First name
Bruce
Country
llAustralia
Going to the last turn is effectively as good a balance as a scenario can achieve.
I don't agree at all. Balance is "does each side have a reasonable chance of winning assuming all other things are equal". I know that can involve a lot of judgement calls -- and a bunch of experienced players might all make different calls. 99% of the time it boils down to an informed guess on the part of the designer. More importantly perhaps, "balance" in no way implies "fun".

A scenario that "always goes to the last turn" has a good chance of being exciting and fun -- and no-one should complain about that. All it really means, though, is that the designer has done a good job of estimating the amount of time required to bring the scenario to a conclusion. It doesn't really say anything about "balance". (It's also not even a good guarantee of "fun" value. A scenario might have one side who is tremendously more powerful than the other, but literally requires a bare minimum number of turns in order to achieve the VC. Every single turn of that game might involve the more powerful side utterly dominating the battlefield as he moves inexorably towards his objective. That is probably not a "fun" scenario to play, even though it always "comes down to the last turn".)
 
Top