Playtesting in the age of play the scenario once and move on.

[Actionjick]

A clunky title but there you are. The modern taste for not playing a scenario multiple times would seem to me to make finding playtesters who are willing to replay the Designer's latest creation a significant amount of times difficult.

How many times does a Designer feel a playtester has to play a scenario to get a good feel for it's merits and flaws?

Given that your scenario, if played at all by the larger ASL community, is likely to only be played once by them is playtesting as important as it was wbitd?

Any thoughts on this would be greatly appreciated.

We would playtest a scenario at least four times.

Thanks!

 

[Robin Reeve]

I would consider that, if a scenario were to be played once only, playtesting is very important.
If seriously unbalanced and/or with errors on the VC/SSR, the risk is that players will shun the designer's or producer's other scenarios.
The first impression will be the only one in such a case, so it is decisive.

I don't think that it is a recent "taste" to play a scenario only once: TPP have been dumping hundreds of scenarios on the market for years and, even if they play them once, most people cannot keep pace.

Many speak of the present situation as the "golden age" of ASL.
I rather see it as a cholesterol stuffed "obese age". And the overabundance of products is paired with excellent designs - quite like cholesterol filled food is delicious.
My real life coronaries have to keep away from that type of stuff.
My ASLverse gamer heart also has to be selective - and I confess that I have a "too much to eat" feeling - especially as I play quite a lot of other games (not all wargames, btw).

TL;DR : good playtesting is mostly important when people only play a scenario once.

 

[Carln0130]

A clunky title but there you are. The modern taste for not playing a scenario multiple times would seem to me to make finding playtesters who are willing to replay the Designer's latest creation a significant amount of times difficult.

How many times does a Designer feel a playtester has to play a scenario to get a good feel for it's merits and flaws?

Given that your scenario, if played at all by the larger ASL community, is likely to only be played once by them is playtesting as important as it was wbitd?

Any thoughts on this would be greatly appreciated.

We would playtest a scenario at least four times.

Thanks!

It varies a lot with the complexity of the scenario. One tactic I use to combat this is to try and recruit as many playtesters as possible and also to playtest each scenario myself with various playtesters. The value of a playtest from anyone is important, but I try to give each scenario at least one playtest between two top tournament players, so that it can get a good, high level test. Between playing them yourself multiple times against various opponents and matching people carefully to give them at least one try, you never let one go until you are satisfied with the results of several playings.

For instance, one project I am involved in, the SCW playtests, we had roughly 8 of the scenarios definitely "put to bed" and a further 4 " very close" with another 4 still developing. Rather than let the "put to bed crowd" go, I asked some top tier players in Jersey to please give them another playing. A couple of the scenarios came back "don't change a thing" but others did require additional tweaks. Therefore, yes, you still play the crap out of these scenarios because who wants to turn out crap? Your name is one of the ones that goes on the product, so you take pride in knowing when it is fully baked.

For BFP's Sangshak, for example, the 10 scenarios I feel are currently in the bag I still farm out to playtesters, some of whom play them multiple times. Others do not. Also, Martin has been kind enough to look at including some of them in his tournament over in England this coming year, to get additional testing there.

So, to sum up my response to the original post, hell yes you still test the crap out of them. Due to our little obsession being so nuanced, even then something may slip through, but certainly anything I am currently involved in sees some playtesters willing to play a scenario multiple times and others who are not, but YOU should play your scenario multiple times before it goes out the door. You should invite players of all skill levels to play them, so that you get a feeling of where the balance is on a given scenario at different levels. While I never discourage playtesters who are new, I do understand their limitations. To expect a new player to find wrinkles in a scenario a veteran or elite player will find is simply unrealistic. Can the new player find things? Absolutely, but it is a different type of feedback. Often based on a fresh faced view of the hobby and the scenarios in general. One group helps you keep things fresh the other precise and balanced. You come to learn who you can trust to play scenarios multiple times and who you can't rely on to play them even once and adjust accordingly. Taking an attitude of "well, they will only play it once before moving on to a hotter chick" shows a lack of pride in your work. I have not encountered anyone in the last two decades who has taken that approach. I would not feel comfortable working with someone who did. People purchase modules and scenario packs. Not playtest kits. ;-)

Just one man's opinion.

 

[Actionjick]

I would consider that, if a scenario were to be played once only, playtesting is very important.
If seriously unbalanced and/or with errors on the VC/SSR, the risk is that players will shun the designer's or producer's other scenarios.
The first impression will be the only one in such a case, so it is decisive.

I don't think that it is a recent "taste" to play a scenario only once: TPP have been dumping hundreds of scenarios on the market for years and even playing them once, most people canno keep pace.

Many speak of the present situation as the "golden age" of ASL.
I rather see it as a cholesterol stuffed "obese age". And the overabundance of products is paired with excellent designs - quite like cholesterol filled food is delicious.
My real life coronaries have to keep away from that type of stuff.
My ASLverse gamer heart also has to be selective - and I confess that I have a "too much to eat" feeling - especially as I play quite a lot of other games (not all wargames, btw).

TL;DR : good playtesting is mostly important when people only play a scenario once.

ROFLMFAO!! The cholesterol filled obese age!!!! These Moderators have a funny side that isn't what you would expect.

Btw I think that assessment is right on!

 

[Actionjick]

It varies a lot with the complexity of the scenario. One tactic I use to combat this is to try and recruit as many playtesters as possible and also to playtest each scenario myself with various playtesters. The value of a playtest from anyone is important, but I try to give each scenario at least one playtest between two top tournament players, so that it can get a good, high level test. Between playing them yourself multiple times against various opponents and matching people carefully to give them at least one try, you never let one go until you are satisfied with the results of several playings.

For instance, one project I am involved in, the SCW playtests, we had roughly 8 of the scenarios definitely "put to bed" and a further 4 " very close" with another 4 still developing. Rather than let the "put to bed crowd" go, I asked some top tier players in Jersey to please give them another playing. A couple of the scenarios came back "don't change a thing" but others did require additional tweaks. Therefore, yes, you still play the crap out of these scenarios because who wants to turn out crap? Your name is one of the ones that goes on the product, so you take pride in knowing when it is fully baked.

For BFP's Sangshak, for example, the 10 scenarios I feel are currently in the bag I still farm out to playtesters, some of whom play them multiple times. Others do not. Also, Martin has been kind enough to look at including some of them in his tournament over in England this coming year, to get additional testing there.

So, to sum up my response to the original post, hell yes you still test the crap out of them. Due to our little obsession being so nuanced, even then something may slip through, but certainly anything I am currently involved in sees some playtesters willing to play a scenario multiple times and others who are not, but YOU should play your scenario multiple times before it goes out the door. You should invite players of all skill levels to play them, so that you get a feeling of where the balance is on a given scenario at different levels. While I never discourage playtesters who are new, I do understand their limitations. To expect a new player to find wrinkles in a scenario a veteran or elite player will find is simply unrealistic. Can the new player find things? Absolutely, but it is a different type of feedback. Often based on a fresh faced view of the hobby and the scenarios in general. One group helps you keep things fresh the other precise and balanced. You come to learn who you can trust to play scenarios multiple times and who you can't rely on to play them even once and adjust accordingly. Taking an attitude of "well, they will only play it once before moving on to a hotter chick" shows a lack of pride in your work. I have not encountered anyone in the last two decades who has taken that approach. I would not feel comfortable working with someone who did. People purchase modules and scenario packs. Not playtest kits. ;-)

Just one man's opinion.

An excellent response! Thanks for the insight. I think a mix of grognards and newer players is precisely the way to go about it. Thanks again for taking the time to respond with such an informative answer.

 

[Actionjick]

ROFLMFAO!! The cholesterol filled obese age!!!! These Moderators have a funny side that isn't what you would expect.

Btw I think that assessment is right on!

TCFOA!! That gets funnier and more poignant every time I read it!!

Randy's post about the preview of CH's latest product was good for some rug burns too!

Well done Gentlemen and to Michael for his work on CASLO.

 

[Robin Reeve]

TCFOA!!

Tightly Coupled Fractal Octagonal Rrings Array ?

 

[Actionjick]

Tightly Coupled Fractal Octagonal Rrings Array ?

Now stop it! I'm supposed to be rebuilding her garden raised beds not nursing rug burns and trying to think of witty responses.

Keep it up though! Hilarious stuff.

 

[Steve E7]

My thoughts. Very good points here in that with so many scenarios to play these days, playing a scenario only once can be the norm. I often do that.
However some scenarios are so compelling and have so much to offer that they bring you back to them, especially to play it against a different opponent or side.
Rigorous playtesting shows in those scenarios. Take a new scenario like AP169 The Beasts Have Arrived. Looked like a must play as one flipped through the Action Pack, we played it right away. 50 playings on ROAR now, nearly even. A nice job playtesting it, and its paying off. I appreciate the effort in playtesting. If a scenario is going to rise to the top of playlists in this era of "once and move on" as the original poster correctly observes, a thorough playtest is worth the time.

 

[von Marwitz]

I am not a scenario designer, so my experience with playtesting has its limits.
In general, the more playtesting, the better. No doubt about that.

I believe that the way how to best playtest a scenario should depend on how and by whom it will mostly be played after final publication.

I agree, that nowadays most scenarios published are played once by the 'end-users'. The number of available stuff and new stuff being published each year currently has made playing the same scenario multiple times less attractive. Some of the scenarios will develop a 'good reputation', so you might see them again in tournament selections or they become attractive for repeated play simply because they are fun and tense. But as a baseline, we can assume today that the greatest number of scenarios released gets played once.

The second question is: Who will play the scenario? To whom is it addressed? It has often been stated that a scenario might be balanced between top players but unbalanced between average players and vice versa. It is an art - with some luck involved I suppose - to come up with a design that is balanced and interesting for all levels of skill. Most players are not top players. So the overall player-base benefits most by scenarios that work best for the average player. Statistics...

So I believe as a rule of thumb, playtesting should assure that the scenario 'works' fine for the first playing by average players. That said, probably (solid) average players are most important for playtesting with a good dose of cracks thrown in who will have better skills of finding possible less obvious flaws or methods to 'break' a given design. This combination will probably yield the best results for most people.

If a scenario design turns out to work as well for multiple playings by the same players and for players of different skill levels (or a combination therof), then we have probably ventured onto the terrain of 'art' with a dose of 'luck'.


Of course, this approach will not work for everyone. It might work best for most.
I remember having played numerous scenarios that I have only 'grokked' (at least that is what I fancy...) after the first playing. Cracks would likely have done so for their first playing. In such a case, I'd only be ready to play the scenario again, if it was otherwise very exciting and interesting. With the continuous loads of new stuff out there, in most cases I would not bother to give a (from my personal perspective) a 'meh' scenario a second try merely for now having a better understanding of it. After all, that would only be playing a 'meh' scenario with better understanding but not with better fun. But I am sure, that doing exactly this - replaying something after having 'grokked' it in a first game, may be the attraction.

What I stated so far is most fitting for scenarios that have one general approach to win. Some fine designs can be played and won following completely different approaches (i.e. high replay value). Usually, the existence of completely different approaches is somewhat apparent from the start. In such cases, multiple playtests of the same version by the same pair of playtesters is beneficial, as it will feel like different 'first playings' despite being the same scenario.

von Marwitz

 

[Actionjick]

My thoughts. Very good points here in that with so many scenarios to play these days, playing a scenario only once can be the norm. I often do that.
However some scenarios are so compelling and have so much to offer that they bring you back to them, especially to play it against a different opponent or side.
Rigorous playtesting shows in those scenarios. Take a new scenario like AP169 The Beasts Have Arrived. Looked like a must play as one flipped through the Action Pack, we played it right away. 50 playings on ROAR now, nearly even. A nice job playtesting it, and its paying off. I appreciate the effort in playtesting. If a scenario is going to rise to the top of playlists in this era of "once and move on" as the original poster correctly observes, a thorough playtest is worth the time.

Very nicely put. Thanks!

 

[Actionjick]

I am not a scenario designer, so my experience with playtesting has its limits.
In general, the more playtesting, the better. No doubt about that.

I believe that the way how to best playtest a scenario should depend on how and by whom it will mostly be played after final publication.

I agree, that nowadays most scenarios published are played once by the 'end-users'. The number of available stuff and new stuff being published each year currently has made playing the same scenario multiple times less attractive. Some of the scenarios will develop a 'good reputation', so you might see them again in tournament selections or they become attractive for repeated play simply because they are fun and tense. But as a baseline, we can assume today that the greatest number of scenarios released gets played once.

The second question is: Who will play the scenario? To whom is it addressed? It has often been stated that a scenario might be balanced between top players but unbalanced between average players and vice versa. It is an art - with some luck involved I suppose - to come up with a design that is balanced and interesting for all levels of skill. Most players are not top players. So the overall player-base benefits most by scenarios that work best for the average player. Statistics...

So I believe as a rule of thumb, playtesting should assure that the scenario 'works' fine for the first playing by average players. That said, probably (solid) average players are most important for playtesting with a good dose of cracks thrown in who will have better skills of finding possible less obvious flaws or methods to 'break' a given design. This combination will probably yield the best results for most people.

If a scenario design turns out to work as well for multiple playings by the same players and for players of different skill levels (or a combination therof), then we have probably ventured onto the terrain of 'art' with a dose of 'luck'.


Of course, this approach will not work for everyone. It might work best for most.
I remember having played numerous scenarios that I have only 'grokked' (at least that is what I fancy...) after the first playing. Cracks would likely have done so for their first playing. In such a case, I'd only be ready to play the scenario again, if it was otherwise very exciting and interesting. With the continuous loads of new stuff out there, in most cases I would not bother to give a (from my personal perspective) a 'meh' scenario a second try merely for now having a better understanding of it. After all, that would only be playing a 'meh' scenario with better understanding but not with better fun. But I am sure, that doing exactly this - replaying something after having 'grokked' it in a first game, may be the attraction.

What I stated so far is most fitting for scenarios that have one general approach to win. Some fine designs can be played and won following completely different approaches (i.e. high replay value). Usually, the existence of completely different approaches is somewhat apparent from the start. In such cases, multiple playtests of the same version by the same pair of playtesters is beneficial, as it will feel like different 'first playings' despite being the same scenario.

von Marwitz

As usual an excellent response. Thanks!

 

[Actionjick]

I would consider that, if a scenario were to be played once only, playtesting is very important.
If seriously unbalanced and/or with errors on the VC/SSR, the risk is that players will shun the designer's or producer's other scenarios.
The first impression will be the only one in such a case, so it is decisive.

I don't think that it is a recent "taste" to play a scenario only once: TPP have been dumping hundreds of scenarios on the market for years and, even if they play them once, most people cannot keep pace.

Many speak of the present situation as the "golden age" of ASL.
I rather see it as a cholesterol stuffed "obese age". And the overabundance of products is paired with excellent designs - quite like cholesterol filled food is delicious.
My real life coronaries have to keep away from that type of stuff.
My ASLverse gamer heart also has to be selective - and I confess that I have a "too much to eat" feeling - especially as I play quite a lot of other games (not all wargames, btw).

TL;DR : good playtesting is mostly important when people only play a scenario once.

I agree that the taste for new scenarios goes back a long ways. Whatever keeps the system going is ok by me.

 

[Old Noob]

As long as there are people wanting to create scenarios for areas that have not had an ASL scenario depicting combat (e.g., Iwo Jima in the 2022 WO pack),
we will get new ASL scenarios. [Hint - anybody seen a scenario for Estonia {either 1941 or 1944}?]

 

[Eagle4ty]

As long as there are people wanting to create scenarios for areas that have not had an ASL scenario depicting combat (e.g., Iwo Jima in the 2022 WO pack),
we will get new ASL scenarios. [Hint - anybody seen a scenario for Estonia {either 1941 or 1944}?]

Agree whole heartedly as there are thousands of engagements that have not been touched on for a tactical presentation and many that have only a scenario or two represented for a major engagement [EX: Operation Bluecoat and Goodwood, battles around Arracourt, Operation Blau 1 North or even the thrust south into the Caucasus. Heck even Normandy hasn't been overdone (Cherbourg?), let alone New Guinea, Leyte and many other Pacific operations.

 

[wrongway149]

Agree whole heartedly as there are thousands of engagements that have not been touched on for a tactical presentation and many that have only a scenario or two represented for a major engagement [EC: Operation Bluecoat and Goodwood, battles around Arracourt, Operation Blau 1 North or even the thrust south into the Caucasus. Heck even Normandy hasn't been overdone (Cherbourg?), let alone New Guinea, Leyte and many other Pacific operations.

Very few 'battle of the Bulge scenarios set after Jan 1, 1945, even though there was still quite a lot of important activity in several areas.

(Extreme winter/deep snow rules might have to make an appearance)

Vosges campaign?

My favorite east front that has had very little ASL done -- Dukla Pass

 

[wrongway149]

Battle of Dukla pass

 

[Psycho]

Battle of Dukla pass

hey Pete here's some Dukla Pass movies you can put on in the background while you design or play:
Tankova Brigada (1955)

I thot I had another one but I guess youtube blocked it. I gotta find it again somewhere

 

[Carln0130]

As long as there are people wanting to create scenarios for areas that have not had an ASL scenario depicting combat (e.g., Iwo Jima in the 2022 WO pack),
we will get new ASL scenarios. [Hint - anybody seen a scenario for Estonia {either 1941 or 1944}?]

Actually, Iwo has seen several other scenarios. One in the YASL 2 pack from LFT. (Shameless plug mode switched off now). There were a few in the old Dagger scenarios and Darrell Anderson did a nice job on the topic for CH.

Certainly not saying there isn't room for more though

 

[ecz]

Nowadays the production of new scenarios is much higher than 10 or 20 years ago. Unless the number of players is also increased ( and I do not think so) it's hard to think that today we have more playtesters than 15 years ago. More scenarios and the same number or less of active players means less playtesters per scenario as average cutting the number of playings for any playtest. The good new is that this does not imply that old scenarios, tested more longer, are necessarily better than today.



Most Designers and publishers(and playtesters too) are into this business by decades. They have learned how to create a good scenario and to recognize the bad ones, thus is not necessary to test to the death more and more times.



but let me say two words about playtesting

I think that playtesting works in two directions. The first is to check if the scenario works, if there are mistakes in the OBs, in the SSRs or in the VCs. The scenario "works" if everything put into it has a reason to exist and is useful to add options or at least is a necessary help to the attacker or to the defender. It works if everything allowed, prohibited by a SSR or requested by the VCs is crystal clear. More eyes read the scenario sheet, and more likely it will be formally perfect. Nothing else to add.



Secondary the playtest helps to find and acceptable balance.

This is a critical point.

Balance is subjective in any scenario, because it depends on who actually plays it. I would be suspicious if the reports from the playtest section say for 50/50 when the playtesters are all different in skill and experience. If an evenly balance appears before or after publication, it happens by chance.

So I would use as playtesters for balance only top players . If not available I would not waste further time to search an instable balance possible just because incidentally the playtesters played at a similar ( not high) level.



This brings to the final point. At which level of play a scenario should be balanced? Some say that a scenario ideally should be balanced at any level of play. I disagree. If a scenario is balanced at any level it happens usually by chance or because is dull and flat and everybody notice at first sight the few obvius and necessary things to do.

Personally I do not have an answer. I think it's impossible to have scenario fun, with high replayability and balanced for newbies and for top players at the same time. Not at least in the conventional form we are used.

Opinions?