Playtesting in the age of play the scenario once and move on.

Actionjick

Forum Guru
Joined
Apr 23, 2020
Messages
7,468
Reaction score
4,992
Location
Kent, Ohio
First name
Darryl
Country
llUnited States
Nowadays the production of new scenarios is much higher than 10 or 20 years ago. Unless the number of players is also increased ( and I do not think so) it's hard to think that today we have more playtesters than 15 years ago. More scenarios and the same number or less of active players means less playtesters per scenario as average cutting the number of playings for any playtest. The good new is that this does not imply that old scenarios, tested more longer, are necessarily better than today.



Most Designers and publishers(and playtesters too) are into this business by decades. They have learned how to create a good scenario and to recognize the bad ones, thus is not necessary to test to the death more and more times.



but let me say two words about playtesting

I think that playtesting works in two directions. The first is to check if the scenario works, if there are mistakes in the OBs, in the SSRs or in the VCs. The scenario "works" if everything put into it has a reason to exist and is useful to add options or at least is a necessary help to the attacker or to the defender. It works if everything allowed, prohibited by a SSR or requested by the VCs is crystal clear. More eyes read the scenario sheet, and more likely it will be formally perfect. Nothing else to add.



Secondary the playtest helps to find and acceptable balance.

This is a critical point.

Balance is subjective in any scenario, because it depends on who actually plays it. I would be suspicious if the reports from the playtest section say for 50/50 when the playtesters are all different in skill and experience. If an evenly balance appears before or after publication, it happens by chance.

So I would use as playtesters for balance only top players . If not available I would not waste further time to search an instable balance possible just because incidentally the playtesters played at a similar ( not high) level.



This brings to the final point. At which level of play a scenario should be balanced? Some say that a scenario ideally should be balanced at any level of play. I disagree. If a scenario is balanced at any level it happens usually by chance or because is dull and flat and everybody notice at first sight the few obvius and necessary things to do.

Personally I do not have an answer. I think it's impossible to have scenario fun, with high replayability and balanced for newbies and for top players at the same time. Not at least in the conventional form we are used.

Opinions?
Very, very nice post. Well thought out and expressed well also.

A lot to think about.

When we playtested our focus was more on the does it work level than balance. We considered balance, as competent playtesters should, but balance was a secondary consideration. Playability and fun were much more important.

I like your opinion of " balance " and will discuss it shortly with Brigadier Bacardi.

Very nice post. Well done you and thanks for the input!!🤗
 

Actionjick

Forum Guru
Joined
Apr 23, 2020
Messages
7,468
Reaction score
4,992
Location
Kent, Ohio
First name
Darryl
Country
llUnited States
I would consider that, if a scenario were to be played once only, playtesting is very important.
If seriously unbalanced and/or with errors on the VC/SSR, the risk is that players will shun the designer's or producer's other scenarios.
The first impression will be the only one in such a case, so it is decisive.

I don't think that it is a recent "taste" to play a scenario only once: TPP have been dumping hundreds of scenarios on the market for years and, even if they play them once, most people cannot keep pace.

Many speak of the present situation as the "golden age" of ASL.
I rather see it as a cholesterol stuffed "obese age". And the overabundance of products is paired with excellent designs - quite like cholesterol filled food is delicious.
My real life coronaries have to keep away from that type of stuff.
My ASLverse gamer heart also has to be selective - and I confess that I have a "too much to eat" feeling - especially as I play quite a lot of other games (not all wargames, btw).

TL;DR : good playtesting is mostly important when people only play a scenario once.
I just love " the cholesterol stuffed obese age "!!!!! 🤣🤣🤣🥰🥰🥰🥰🥰
 

Actionjick

Forum Guru
Joined
Apr 23, 2020
Messages
7,468
Reaction score
4,992
Location
Kent, Ohio
First name
Darryl
Country
llUnited States
Nowadays the production of new scenarios is much higher than 10 or 20 years ago. Unless the number of players is also increased ( and I do not think so) it's hard to think that today we have more playtesters than 15 years ago. More scenarios and the same number or less of active players means less playtesters per scenario as average cutting the number of playings for any playtest. The good new is that this does not imply that old scenarios, tested more longer, are necessarily better than today.



Most Designers and publishers(and playtesters too) are into this business by decades. They have learned how to create a good scenario and to recognize the bad ones, thus is not necessary to test to the death more and more times.



but let me say two words about playtesting

I think that playtesting works in two directions. The first is to check if the scenario works, if there are mistakes in the OBs, in the SSRs or in the VCs. The scenario "works" if everything put into it has a reason to exist and is useful to add options or at least is a necessary help to the attacker or to the defender. It works if everything allowed, prohibited by a SSR or requested by the VCs is crystal clear. More eyes read the scenario sheet, and more likely it will be formally perfect. Nothing else to add.



Secondary the playtest helps to find and acceptable balance.

This is a critical point.

Balance is subjective in any scenario, because it depends on who actually plays it. I would be suspicious if the reports from the playtest section say for 50/50 when the playtesters are all different in skill and experience. If an evenly balance appears before or after publication, it happens by chance.

So I would use as playtesters for balance only top players . If not available I would not waste further time to search an instable balance possible just because incidentally the playtesters played at a similar ( not high) level.



This brings to the final point. At which level of play a scenario should be balanced? Some say that a scenario ideally should be balanced at any level of play. I disagree. If a scenario is balanced at any level it happens usually by chance or because is dull and flat and everybody notice at first sight the few obvius and necessary things to do.

Personally I do not have an answer. I think it's impossible to have scenario fun, with high replayability and balanced for newbies and for top players at the same time. Not at least in the conventional form we are used.

Opinions?
Brigadier Bacardi thinks that is a damn good post and we couldn't agree more with it. Captain Bacchus, Colonel Cuervo and the whole gang of my drinking buddies think the same. This will most likely lead to more pontificating about " balance " on my part.

Lol, I really do have strong opinions about balance.
 

fenyan

Elder Member
Joined
Nov 19, 2004
Messages
703
Reaction score
1,194
Location
California
Country
llUnited States
Agreed, have your best players playtest the scenarios.

Though the main goal is to have fun, everyone wants a decent shot (say around 40% or more) to win a scenario.
 

Actionjick

Forum Guru
Joined
Apr 23, 2020
Messages
7,468
Reaction score
4,992
Location
Kent, Ohio
First name
Darryl
Country
llUnited States
The current taste for once and move on would make fun much more important than balance. If you are never going to play it again how can you know if it is balanced or not?
 

Actionjick

Forum Guru
Joined
Apr 23, 2020
Messages
7,468
Reaction score
4,992
Location
Kent, Ohio
First name
Darryl
Country
llUnited States
Agreed, have your best players playtest the scenarios.

Though the main goal is to have fun, everyone wants a decent shot (say around 40% or more) to win a scenario.
That seems about right for a ballpark estimate.
 

Actionjick

Forum Guru
Joined
Apr 23, 2020
Messages
7,468
Reaction score
4,992
Location
Kent, Ohio
First name
Darryl
Country
llUnited States
Brigadier Bacardi thinks that is a damn good post and we couldn't agree more with it. Captain Bacchus, Colonel Cuervo and the whole gang of my drinking buddies think the same. This will most likely lead to more pontificating about " balance " on my part.

Lol, I really do have strong opinions about balance.
I played 90% plus of my scenarios against Fish. He was undoubtedly the best SL/ASL player of his time. YMMV but YWBW.

Q. How do you judge if a scenario is balanced when you are lucky to win three out of ten scenarios against guy you are playtesting with?

A. Well if you beat him three straight times then there might be am issue. 🤣

Seriously it is more a matter of " feel ". See the Blink thread.

Some scenarios just didn't work.
 

von Marwitz

Forum Guru
Joined
Nov 25, 2010
Messages
14,358
Reaction score
10,207
Location
Kraut Corner
Country
llUkraine
So I would use as playtesters for balance only top players . If not available I would not waste further time to search an instable balance possible just because incidentally the playtesters played at a similar ( not high) level.

This brings to the final point. At which level of play a scenario should be balanced? Some say that a scenario ideally should be balanced at any level of play. I disagree. If a scenario is balanced at any level it happens usually by chance or because is dull and flat and everybody notice at first sight the few obvius and necessary things to do.

Personally I do not have an answer. I think it's impossible to have scenario fun, with high replayability and balanced for newbies and for top players at the same time. Not at least in the conventional form we are used.

Opinions?
See my post #10 above.

To partly quote myself on your point:

"Who will play the scenario? To whom is it addressed? It has often been stated that a scenario might be balanced between top players but unbalanced between average players and vice versa. It is an art - with some luck involved I suppose - to come up with a design that is balanced and interesting for all levels of skill. Most players are not top players. So the overall player-base benefits most by scenarios that work best for the average player. Statistics...

So I believe as a rule of thumb, playtesting should assure that the scenario 'works' fine for the first playing by average players. That said, probably (solid) average players are most important for playtesting with a good dose of cracks thrown in who will have better skills of finding possible less obvious flaws or methods to 'break' a given design. This combination will probably yield the best results for most people.

If a scenario design turns out to work as well for multiple playings by the same players and for players of different skill levels (or a combination therof), then we have probably ventured onto the terrain of 'art' with a dose of 'luck'.

Of course, this approach will not work for everyone. It might work best for most."


von Marwitz
 

ecz

Partisan Captain
Joined
Aug 31, 2003
Messages
4,430
Reaction score
599
Location
Italy
Country
llItaly
the question behid my thoughts partially overlapping on yours is: should the Scenario Designer (or the producer for what it matters) let a scenario "unbalanced" by purpose at high level of play if it appears be evenly balanced at average level?

in other words, if the TD can make a change that makes balanced the scenario at the best level of play based upon a hint given by a top player but that change seems unnecessary or, even worse, unbalances the scenario at a lower level of play, what should decide the Designer?
 

ecz

Partisan Captain
Joined
Aug 31, 2003
Messages
4,430
Reaction score
599
Location
Italy
Country
llItaly
an example will clarify which is my doubt.

The Victory Conditions of the Scenario " Conquer that Village" say in the first version for playtest: The attacker must control 10 buildings. For most playtesters that number is perfect, it's a fair number of buildings and the attacker wins about 50% of the times.

BUT the two best playtesters of the group later show that a very competent attacker can conquer even 12 or 13 buildings. The Scenario Designer then check personally and sees that 10 is too low after a pair of playings vs these top players: A perfect attack can conquer all 13 buildings while the best defense stops the attacker just the 50% of the times while it has no chance if the attacker must conquer only 10.

And now? the Producer/Designer will hold low the VC because most audience is confortable with an "easy" for the attacker 10 buildings, or will consider the advaned plans and tactis that just the 10% could reasonably adopt allowing a more in depth attack capable to reach 13 buildings? And what about ROAR? it's better a scenario widely played and apparently balanced with the low VCs suggested by all the playtesters but two, or a scenario that looks impossible for the attacker at first sight but where there is the evidence that is challenging if played at high level?

I Think that any scenario should be balanced at the higher level possible, even at the cost to appear "easy" or "impossible" at the average attacker or defender. The top players must be favored always. But I'm not so sure that everyone agrees
 

von Marwitz

Forum Guru
Joined
Nov 25, 2010
Messages
14,358
Reaction score
10,207
Location
Kraut Corner
Country
llUkraine
The Victory Conditions of the Scenario " Conquer that Village" say in the first version for playtest: The attacker must control 10 buildings. For most playtesters that number is perfect, it's a fair number of buildings and the attacker wins about 50% of the times.

BUT the two best playtesters of the group later show that a very competent attacker can conquer even 12 or 13 buildings. The Scenario Designer then check personally and sees that 10 is too low after a pair of playings vs these top players: A perfect attack can conquer all 13 buildings while the best defense stops the attacker just the 50% of the times while it has no chance if the attacker must conquer only 10.

And now?
I think something like the ABS (Abilene Bidding System?) or some bidding system used in some tournaments could address these issues: You think you can do 12 buildings instead of 10? Go ahead and bid that and play the side you want if your bid exceeds that of your opponent.

Normally such bidding is done only for purposes of getting the side you want. In theory it could also be extended on the skill-level as it is often informally done: The better player will voluntarily give the less experienced one "the balance" on the scenario card to make the playing more interesting.

In this, cracks may well be able to judge what they think they can "relinquish" to make the game close vs. an average opponent. If it's crack vs. crack, then the bidding will be more about which side to play rather than shifting the balance.

von Marwitz
 
Reactions: ecz

von Marwitz

Forum Guru
Joined
Nov 25, 2010
Messages
14,358
Reaction score
10,207
Location
Kraut Corner
Country
llUkraine
The top players must be favored always. But I'm not so sure that everyone agrees
Indeed, I do not agree. :)

By definition, the top players can only be a small fraction of the player base.

I believe that a game should be 'optimized' for a large fraction of its player base rather then merely a small portion of it.

Thus my focus on the 'solid average player' as the basis of contemplation.

von Marwitz
 

ecz

Partisan Captain
Joined
Aug 31, 2003
Messages
4,430
Reaction score
599
Location
Italy
Country
llItaly
Indeed, I do not agree. :)

By definition, the top players can only be a small fraction of the player base.

I believe that a game should be 'optimized' for a large fraction of its player base rather then merely a small portion of it.

Thus my focus on the 'solid average player' as the basis of contemplation.

von Marwitz
So, as ipothetycal scenario Designer, you would leave the easy VC just because most playtesters says "ten" fits well with their playing style disregarding what you noticed after two seasoned top players pointed out?
 

ecz

Partisan Captain
Joined
Aug 31, 2003
Messages
4,430
Reaction score
599
Location
Italy
Country
llItaly
I think something like the ABS (Abilene Bidding System?) or some bidding system used in some tournaments could address these issues: You think you can do 12 buildings instead of 10? Go ahead and bid that and play the side you want if your bid exceeds that of your opponent.

Normally such bidding is done only for purposes of getting the side you want. In theory it could also be extended on the skill-level as it is often informally done: The better player will voluntarily give the less experienced one "the balance" on the scenario card to make the playing more interesting.

In this, cracks may well be able to judge what they think they can "relinquish" to make the game close vs. an average opponent. If it's crack vs. crack, then the bidding will be more about which side to play rather than shifting the balance.

von Marwitz
I had in mind, back in the days, a different way to address the iussue, assuming the difference in two close sets of VCs is a iussue after all.

A change in perspective during the scenario design process capable to make obsolete once for ever any argument about balance because the players decides by theirselves the level of commitment of the attacker IF the play the attacker role.
A kind of ABS, but more simple and automatically integrated in a SSR assigning sides and setting the level of the VC at the same time.

Unfortunately I was unable to hit the heart of the Designers and none of them, unless I missed something, never replied the VC system used in the scenario The Bet, runner up in a Monkey with Typewriters contest of half a century ago... That scenario was later published by LFT (LFT 181).

The original idea, in spite of praise and compliments, was never take seriously.
 

von Marwitz

Forum Guru
Joined
Nov 25, 2010
Messages
14,358
Reaction score
10,207
Location
Kraut Corner
Country
llUkraine
So, as ipothetycal scenario Designer, you would leave the easy VC just because most playtesters says "ten" fits well with their playing style disregarding what you noticed after two seasoned top players pointed out?
Yes, I believe so.

However, you should note that for the 'solid average' ASL player, 'ten' does not necessarily mean that the VC are easy. From their (play-test) perspective, 'ten' is just what they can manage to conquer (or to defend) with their skill level.

Of course, this has to be taken with a grain of salt:
For some scenarios, it takes some time until the 'community' has grokked them. When they are newly published, results and opinons may be mixed. But after a while, a particular 'tactic' emerges and word spreads about it.
In such a case, the 'grokking' is not done by a crack all by himself, who as a result would say "12", but it is rather a "hivemind-process" that average players take advantage of via word of mouth/forum/VASL observation/whatever.
It depends on the individual scenario, if such a "hivemind-solution" can be anticipated. If so, then the original balance might indeed better be "11" or "12" instead of "10" to provide the scenario with a longer "service life".

von Marwitz
 
Reactions: ecz

Hutch

Curator of the ASL Armory
Joined
Feb 16, 2004
Messages
2,457
Reaction score
1,788
Location
FL
First name
Hutch
Country
llUnited States
I think another factor is the company that is producing the scenarios. How many times have we seen "submitted to..." and not seen the final product for a few years. I think Mark P. was making statements about companies that were producing a high volume of scenarios in a short period of time. He called out a few as I remember, not just one certain company.
 

ecz

Partisan Captain
Joined
Aug 31, 2003
Messages
4,430
Reaction score
599
Location
Italy
Country
llItaly
(...)
For some scenarios, it takes some time until the 'community' has grokked them. When they are newly published, results and opinons may be mixed. But after a while, a particular 'tactic' emerges and word spreads about it.
In such a case, the 'grokking' is not done by a crack all by himself, who as a result would say "12", but it is rather a "hivemind-process" that average players take advantage of via word of mouth/forum/VASL observation/whatever.
It depends on the individual scenario, if such a "hivemind-solution" can be anticipated. If so, then the original balance might indeed better be "11" or "12" instead of "10" to provide the scenario with a longer "service life".

von Marwitz
the part about the "grokking" of the scenario implies that a scenario that appears balanced initially, after years of playing may reveal a weak spot and be considered unbalanced or viceversa. More than 30 years of ASL confirms this. It's an event out of the control of any Designer or playtest group.

Well, incidentally one of the many advantages of any scenario using the bid system (like the Bet) is that they are eternal because if and when a new tactic or a new playing style or a rule changes the way to play affecting the balance, the only thing that a player needs is to adjust accordingely his bid to play the attacking side.
Thus the hard part when someone design such a kind of scenarios is just make them interesting for both players, orginal and replayable. The balance is no more a problem. This brings a series of positive effects linked in a chain. No need to mention them here.

BTW is exactly what happened to the famous award-winning wargame "Paths of Glory" (GMT, 1999) featuring a bid system to assign side and VC at the same time. Intially the suggested bid ( in VP to assign to the other side at game end) to play the Central Powers was "zero" because the game appeared evenly balanced. Then the years of praticse revelaled that the Allied powers had an edge. Hence the suggested bid become 1 and finally, nowadays, 2. Today at high level who plays the Central Powers usually obtains a two-points bonus offered by the side willing to play the Allied Powers that has slghtly better chances. This suggested 2- points bonus assures today to start a balanced game.
20 years ago the perception of balance was different.
 

von Marwitz

Forum Guru
Joined
Nov 25, 2010
Messages
14,358
Reaction score
10,207
Location
Kraut Corner
Country
llUkraine
Hence the suggested bid become 1 and finally, nowadays, 2. Today at high level who plays the Central Powers usually obtains a two-points bonus offered by the side willing to play the Allied Powers that has slghtly better chances. This suggested 2- points bonus assures today to start a balanced game.
20 years ago the perception of balance was different.
This is a good point.

In ASL, there are those old 10 Turn scenarios. With the contemporary "Move it!" style of play as opposed to the older style which had more Prep Fire in it, some of these "oldies" seem unbalanced with 10 Turns today as using the "Move it!"-style they can be dealt with in 8 or 9 Turns.

von Marwitz
 

ecz

Partisan Captain
Joined
Aug 31, 2003
Messages
4,430
Reaction score
599
Location
Italy
Country
llItaly
or think about all the old scenarios that now are ruined by the VBM freeze tactic, non common in the early ASL days.
 

Actionjick

Forum Guru
Joined
Apr 23, 2020
Messages
7,468
Reaction score
4,992
Location
Kent, Ohio
First name
Darryl
Country
llUnited States
This is a good point.

In ASL, there are those old 10 Turn scenarios. With the contemporary "Move it!" style of play as opposed to the older style which had more Prep Fire in it, some of these "oldies" seem unbalanced with 10 Turns today as using the "Move it!"-style they can be dealt with in 8 or 9 Turns.

von Marwitz
My recollection is that the older scenarios were very movement oriented. Its been a long time so my recollection may be faulty.
 
Top