Third Party Publisher Rate of Publication Question

Pitman

Elder Member
Joined
Jan 27, 2003
Messages
14,016
Reaction score
1,937
Location
Columbus, OH
Country
llUnited States
So I am curious. Imagine an "average" Third Party Publisher in terms of size and, perhaps more importantly, number of associates of that TPP who are willing to playtest for it.

What rate of publication for that publisher would make you start to wonder if its goals were beginning to outstrip its realistic ability to fully playtest and develop its products?

Would more than one product per year start having you a bit concerned? (that is when I personally tend to start having doubts)

More than two products a year?

Do you think a TPP can actually release three fully developed and playtested products per year?

Do you trust that a TPP that released any number of products per year was fully playtesting and developing them beforehand?

Have you ever thought of any past TPP, "Man, they are releasing too many products too fast for them to really have fully playtested them"?

For clarity and for purposes of this discussion, let's describe a "product" as basically equivalent a scenario pack or scenario map/pack with at least 6 scenarios, a HASL-like product, or a number of newsletters during the course of a year that have a total of at least 8 scenarios in them. Let's not worry about very small products, like equivalents to MMP's "bonus packs."
 

Michael Dorosh

der Spieß des Forums
Gold Supporting Member
Joined
Feb 6, 2004
Messages
15,224
Reaction score
2,016
Location
Calgary, AB
First name
Michael
Country
llCanada
So I am curious. Imagine an "average" Third Party Publisher in terms of size and, perhaps more importantly, number of associates of that TPP who are willing to playtest for it.

What rate of publication for that publisher would make you start to wonder if its goals were beginning to outstrip its realistic ability to fully playtest and develop its products?
I think it is unhelpful to make sweeping generalizations, and far more helpful to rate products on their actual merits, not what someone "thinks" the publisher "may" have done.

Generally I look at production quality (including stuff like typos and proofreading) but more importantly word of mouth from those whose opinion I trust, and occasionally a look at win-loss records on ROAR or the Scenario Archive to see if it is getting widely played, and if the scenarios seem balanced.

Honestly, I couldn't care less how something gets to market. I've been involved in playtests for board and computer wargames, including as a scenario designer, and can tell you that playtesting is never as extensive as one would hope and I think there is an unrealistic expectation in many quarters about what should be done in a playtest. (Seeing this way more in computer games now as well, and I think it will continue to be the norm given the number of content providers, and their ability to reissue stuff using consumers as unpaid beta testers).

Besides, some designers can get it right off the hop. I've tested for Lone Canuck, and we were amazed at how few adjustments George's scenarios needed based on our results (for example we played every scenario in one of the Blitzkrieg packs twice, changing sides and had very few recommendations to make). I firmly believe no scenario can ever be "perfect" anyhow based on differing play styles and understanding of the game.

If I am having fun, and others are reporting they are having fun, I see no reason to question the playtesting of this or that at all. In particular, trying to devise a formula where products are assigned a value dependent on number of releases per year can serve no useful purpose.
 
Last edited:

Tuomo

Keeper of the Funk
Joined
Feb 10, 2003
Messages
3,357
Reaction score
2,426
Location
Rock Bottom
Country
llUnited States
What rate of publication for that publisher would make you start to wonder if its goals were beginning to outstrip its realistic ability to fully playtest and develop its products?
If your goal is not necessarily to put out quality products, then rate of production isn't as important.

If I am having fun, and others are reporting they are having fun, I see no reason to question the playtesting of this or that at all.
I see your point and yet I also disagree. At some level, if people are happy, then who's to criticize. And yet, people seem to value Desperation Morale largely because its product reviews serve as a filter to help them avoid low-quality products. So the drive to value quality is laudable, I think, on a practical and pragmatic level at the very least. For most ASL producers, I think their primary reason for valueing quality is because they want to do their work well.

All that said, my rough feeling is that pretty much all of the TPMs (and MMP as well) could double their rate of production and I wouldn't particularly worry. I tend to think ASL publishers actually over-polish their products, either out of an abundance of caution about protecting their reputations or perhaps because they don't want to give the perception that they're not putting enough effort into polishing their products. Or, heck, maybe they're resource-limited in terms of people to do the work and/or money to pay for it.
 

Michael Dorosh

der Spieß des Forums
Gold Supporting Member
Joined
Feb 6, 2004
Messages
15,224
Reaction score
2,016
Location
Calgary, AB
First name
Michael
Country
llCanada
I see your point and yet I also disagree. At some level, if people are happy, then who's to criticize. And yet, people seem to value Desperation Morale largely because its product reviews serve as a filter to help them avoid low-quality products.
How many products does Desperation Morale playtest before publishing their reviews? Do they even have face to face playtests of new products?

Seems disingenuous to say you have to have a rigorous standard of playtesting (what that standard is hasn't been discussed yet in this thread) in order to produce a quality product, but you don't have to play it at all to either recommend it as one or declare that it isn't.

But, I don't mean to derail this thread, so if you wish that can be discussed here instead: http://www.gamesquad.com/forums/index.php?threads/third-party-reviewer-rate-of-publication-question.154529/
 
Last edited:

Michael Dorosh

der Spieß des Forums
Gold Supporting Member
Joined
Feb 6, 2004
Messages
15,224
Reaction score
2,016
Location
Calgary, AB
First name
Michael
Country
llCanada
All that said, my rough feeling is that pretty much all of the TPMs (and MMP as well) could double their rate of production and I wouldn't particularly worry. I tend to think ASL publishers actually over-polish their products, either out of an abundance of caution about protecting their reputations or perhaps because they don't want to give the perception that they're not putting enough effort into polishing their products. Or, heck, maybe they're resource-limited in terms of people to do the work and/or money to pay for it.
I think most of your comments are spot on, as usual. I wonder in return - how many people buy products based on their perception of playtesting? I honestly don't know. There seems to be a sizeable proportion of the community that buy up copies of stuff they'll never play anyway (sometimes multiple copies). Just to have, or just because MMP produced it, or their favourite TPP that has established - to them - a reputation for quality.

I think resource limitations would be a common theme among any publisher today - I'm not even sure it is possible to have "sufficient" access to editorial and proofing assistance.
 

Vinnie

See Dummies in the index
Joined
Feb 9, 2005
Messages
16,404
Reaction score
2,177
Location
Aberdeen , Scotland
Country
llUnited Kingdom
I think a lot depends on how deep the pool of talent is. So he TOP have a large number of players testing and suggestingbwith some products taking years to come to fruition. Others, not so much.
Additionally, you can get a false impression if a company gets several products ready at the same time. Some of those may gave been tested for years previously but onky now come to market.
 

Double Deuce

Forum Conscript
Joined
Dec 15, 2002
Messages
4,704
Reaction score
64
Location
Crossville, TN
Country
llUnited States
A factor that probably can't be evaluated properly in development time is some people just have a knack for getting the initial concept pretty well balanced. That undoubtedly speed up the time getting the overall product tested and out the door.
 

hongkongwargamer

Forum Guru
Joined
Apr 4, 2013
Messages
6,285
Reaction score
3,666
Location
.. beneath the Pale Moonlight
Country
llZimbabwe
I am not sure if we purchase new products as a direct result of the amount of playtesting done (which is not uniformly reported on products pre-purchase anyway).

We are more likely to purchase new products based on the amount of buzz (or lack there of) from playtesters. We are very likely to purchase new products based on the accumulated "total" (with more recent experience weighted heavier) of personal experience we had with the same publisher.

That accumulated experience can be from playing - so in this case "balance" will be a criteria. The amount of playtesting done for those previously published products will come through indirectly. Or if the designer indeed have a talent for balancing scenarios, that will come through too. The thing here is : it's the accumulated perception of balance with previous products that pulls your new purchasing decision trigger, how that publisher achieve that balance is immaterial. We take that accumulated perception and project it on to new purchasing decisions.

That accumulated experience can be based on how much fun they are, even though imbalanced. Those folks then base their new purchase decisions on (the accumulated total with a decay) how much fun they had with that TPP's previous products. That's fine too.

That accumulated experience can be based on how pretty the maps are, or how many new counters they get. Or if a new product is about an interesting bit of WW2 combat history that wasn't wholly covered by the market, and that TPP did okay with other parameters we care about, we might take a chance and jump in. Some folks just want to have everything - okay. We all make our purchasing decisions based on our personal wants and needs.
 

Mister T

Elder Member
Joined
Aug 12, 2006
Messages
4,010
Reaction score
1,420
Location
Bruxelles
Country
llFrance
Would more than one product per year start having you a bit concerned? (that is when I personally tend to start having doubts)
A single annual release would generally be the right pace. Any faster pace implies cutting corners.

Besides, while pace is a necessary factor for quality, it is not a sufficient one. Incompetent editors / poor designers / weak playtesters are more likely to weigh on quality than just pace.
 

zgrose

Elder Member
Joined
Jun 13, 2004
Messages
3,759
Reaction score
624
Location
Kingwood, TX
First name
Zoltan
Country
llUnited States
Do you think a TPP can actually release three fully developed and playtested products per year?
I believe a TPP can release 3 products that are fun to play but may not produce 50-50 results on ROAR.
I also see TPP and MMP release a few products that can be rather lame regardless of balance.

Let the products stand on their merits.
 

Pitman

Elder Member
Joined
Jan 27, 2003
Messages
14,016
Reaction score
1,937
Location
Columbus, OH
Country
llUnited States
I am not sure if we purchase new products as a direct result of the amount of playtesting done (which is not uniformly reported on products pre-purchase anyway).

We are more likely to purchase new products based on the amount of buzz (or lack there of) from playtesters. We are very likely to purchase new products based on the accumulated "total" (with more recent experience weighted heavier) of personal experience we had with the same publisher.

That accumulated experience can be from playing - so in this case "balance" will be a criteria. The amount of playtesting done for those previously published products will come through indirectly. Or if the designer indeed have a talent for balancing scenarios, that will come through too. The thing here is : it's the accumulated perception of balance with previous products that pulls your new purchasing decision trigger, how that publisher achieve that balance is immaterial. We take that accumulated perception and project it on to new purchasing decisions.

That accumulated experience can be based on how much fun they are, even though imbalanced. Those folks then base their new purchase decisions on (the accumulated total with a decay) how much fun they had with that TPP's previous products. That's fine too.

That accumulated experience can be based on how pretty the maps are, or how many new counters they get. Or if a new product is about an interesting bit of WW2 combat history that wasn't wholly covered by the market, and that TPP did okay with other parameters we care about, we might take a chance and jump in. Some folks just want to have everything - okay. We all make our purchasing decisions based on our personal wants and needs.
Just fyi, I wasn't asking about purchasing products but about third party publishers being spread too thin, to the point that their products may suffer.
 

Pitman

Elder Member
Joined
Jan 27, 2003
Messages
14,016
Reaction score
1,937
Location
Columbus, OH
Country
llUnited States
A single annual release would generally be the right pace. Any faster pace implies cutting corners.

Besides, while pace is a necessary factor for quality, it is not a sufficient one. Incompetent editors / poor designers / weak playtesters are more likely to weigh on quality than just pace.
Glad to see your post. Until you posted, it seemed like I was the only soul out there who realized that TPP who churn products out may not be giving them the tender loving care they need in terms of playtesting and development.
 

hongkongwargamer

Forum Guru
Joined
Apr 4, 2013
Messages
6,285
Reaction score
3,666
Location
.. beneath the Pale Moonlight
Country
llZimbabwe
Just fyi, I wasn't asking about purchasing products but about third party publishers being spread too thin, to the point that their products may suffer.
Yeah sorry I got distracted.

There are two parts to how a product gets to market I suppose. There are “designer teams” that put something together at no time cost to the Publisher and there is the core developer team that PT (perhaps), proofread (perhaps) and publish the product.

I suppose an upper limit can be found by looking at how many products MMP or BFP puts out a year (2?). Design, edit, proofreading aside, attracting enough people to PT and getting enough PT in is perhaps the biggest bottleneck. Whether your TPP has good enough reputation for designers to want to work with you is another (affecting # of good designs available to you for publishing). So I imagine the established reputation of the TPP and the reputation/ social pull of the Designer/ Designer team is a big factor when it comes to # of products a year.

So - I would say around 2/3. Too far beyond that I’d wonder not only about playtesting but about quality of design as well. Prodigies not withstanding.
 

zgrose

Elder Member
Joined
Jun 13, 2004
Messages
3,759
Reaction score
624
Location
Kingwood, TX
First name
Zoltan
Country
llUnited States
Glad to see your post. Until you posted, it seemed like I was the only soul out there who realized that TPP who churn products out may not be giving them the tender loving care they need in terms of playtesting and development.
You may also want to consider that the demographics of what players want out of TPP has changed. YMMV
 

Evan Sherry

Elder Member
Joined
Apr 19, 2004
Messages
1,253
Reaction score
506
Location
Tampa, FL
Country
llUnited States
So I am curious. Imagine an "average" Third Party Publisher in terms of size and, perhaps more importantly, number of associates of that TPP who are willing to playtest for it.

What rate of publication for that publisher would make you start to wonder if its goals were beginning to outstrip its realistic ability to fully playtest and develop its products?

Would more than one product per year start having you a bit concerned? (that is when I personally tend to start having doubts)

More than two products a year?

Do you think a TPP can actually release three fully developed and playtested products per year?

Do you trust that a TPP that released any number of products per year was fully playtesting and developing them beforehand?

Have you ever thought of any past TPP, "Man, they are releasing too many products too fast for them to really have fully playtested them"?

For clarity and for purposes of this discussion, let's describe a "product" as basically equivalent a scenario pack or scenario map/pack with at least 6 scenarios, a HASL-like product, or a number of newsletters during the course of a year that have a total of at least 8 scenarios in them. Let's not worry about very small products, like equivalents to MMP's "bonus packs."
I've considered this a lot over the last 23 years. The size of the product and more specifically, the number of scenarios in a product is what gets my attention. I know what it takes to playtest 22 scenarios and get them ready for print.

A backlog of scenarios is very helpful in getting a product ready. That is to say having a number of scenarios that have been designed and partially play-tested for a couple of years or more, can make production smoother and more efficient. I may release up to two products in a year, but the components of those products often have been in the pipe much longer than a year.

Also, it is essential that the scenario designer plays his own raw, first draft scenarios before letting play-testers have them. That way the designer works the bugs out first and does not waste valuable play-tester assets. If I do not see each scenario designer listed as part of the playtest team, I also begin to view this as a sort of "fire and forget" method of scenario design and question the validity of the product.

The size of the playtest group is important, but more important is the experience level of the members of the playtest group. Increased size of the group allows more scenarios to be played in a shorter time. But sheer volume of playtesting is not nearly as important as quality and timely testing of a given version of a scenario. The core of my playtest group are scenario designers as well as players and have been playing together for over 20 years. Randy Thompson is a very good player who has been playing my SL/ASL scenario designs with me since 1980, so he knows how to balance a scenario. We just lost Dave Brown who died of a heart attack in August. He had been playing with us since Schwerpunkt Volume #2. He was one of the guys who attended almost every gaming session that I hosted. He played a lot of scenarios. It will take at least two very dedicated players to make up the volume of games that Dave would have play-tested.

Having outside play-testers or having remote test groups is great, but I found that outside play-testers do not have the motivation/dedication to playtest scenarios that may be updated in a rapid manner. My playtesting is on a tight schedule and receiving a playtest result on version #2 of a scenario weeks or months after sending updates and when the in-house group is on version #4 or higher, may have limited value by time it is received.

So the answer to your question is difficult and complex. My answer is that releasing two well play-tested products in a year is very difficult and demanding. Few can do it. That is the reason that the last Schwerpunkt was released in December 2017 and that none are scheduled for the near future. I had been on a grueling playtest schedule since 1993 when we started designing original scenarios for the Florida ASL Tournament. After hurricane Irma hit us in 2017, it all caught up with me and I decided that I was not going to even attempt to release two products in time for ASLOK any longer. It is simply too much work to playtest that many scenarios. So, when I see a group release one product, I say it is definitely doable. When I see a group release two products, I say they busted their ass doing it if it turns out to be a well-playtested and balanced set of scenarios. When I see more than that or a giant product, I do start to have my doubts unless the products have been in long-term production and just happened to be released in the same year.
 
Last edited:

Sparafucil3

Forum Guru
Joined
Oct 7, 2004
Messages
9,903
Reaction score
2,911
Location
USA
First name
Jim
Country
llUnited States
To me, it comes down to who is designing and who is play testing. Good players and a designer who won't listen will tend to produce bad scenarios. A good designer with bad players will tend to produce interesting scenarios that tend to have balance issues. A good designer who is himself a good player, AND has a good grasp on the rules, is the best scenario designer you can hope for. They can over come input from players based on their experience and attract really good players into their test group. In the end, as a consumer, it comes down to word of mouth and history. I honestly think we wouldn't even be having this conversation if there weren't so much emphasis on tournament play. JMO, YMMV. -- jim
 

Tuomo

Keeper of the Funk
Joined
Feb 10, 2003
Messages
3,357
Reaction score
2,426
Location
Rock Bottom
Country
llUnited States
Glad to see your post. Until you posted, it seemed like I was the only soul out there who realized that TPP who churn products out may not be giving them the tender loving care they need in terms of playtesting and development.
Starting to regret the time I took to try to understand the original post and to formulate a thoughtful reply.

I thought surely it couldn't be simply bashing the old "Critical Hit puts out too much stuff too fast" idea. That's too tired a horse to beat. Beyond them, who else could we possibly be referring to? I don't recall seeing a rash of products from any of the other TPMs recently.

So it's no surprise to me that people generally answered the first question (of the six that were posed). In fact, I'm a tad surprised/heartened that this didn't degenerate into a CH Bashing Thread. Maybe we've decided to move on to other things.
 
Top