Ray Tapio is da Man!

Stardragon99

Member
Joined
May 13, 2005
Messages
472
Reaction score
36
Location
Pyrenees
Country
llCanada
Of course the one underlying assumption for all playtesting and all ROAR reports was that the two players were of perfectly equal ability and were equally capable of capitalizing on the rulebook. Im a newer (weak) player and i usually lose a lot. You can assume that every reporting done my partner on our games skews the balance - not sure that is a fair representation to the designer or the playtest process. Maybe all the other games out there take place between opponents of very equal ability - though i doubt it.
 

ON TOP ASL

Play J94, J98, J110, J111
Joined
Oct 27, 2003
Messages
791
Reaction score
17
Location
Rättvik, Sweden
Country
llSweden
Good point.
I tend to use unbalanced scenarios in teaching new players of ASL (taking the 'hard' side of course :cool:). That make's the game more interesting for me, and will occasionally push the scenario in other direction (towards more balanced) than otherwize would be.
 

Will Fleming

Senior Member
Joined
Apr 22, 2003
Messages
4,410
Reaction score
424
Location
Adrift on the Pequod
Country
llUnited States
I would take one playtest from top notch players over 20 playings of mere mortals. I would take the opinion of a top notch player after a casual glance of the scenario over 20+ normal game reports.

My definition of 'balanced' would be one that the two top players would be happy to play either side in a game against each other in the final round of a big tournament.

Of course for mere mortals like myself, the W/L ratio can be totally different since we (I) miss so many things during the course of the scenario.

Pitman and Glennbo both have their own style/systems. Let them do what they want. Just keep the stuff coming :)
 
Last edited:

Bret Hildebran

Elder Member
Joined
Jan 31, 2003
Messages
4,884
Reaction score
1,279
Location
NE OH
Country
llUnited States
Of course the one underlying assumption for all playtesting and all ROAR reports was that the two players were of perfectly equal ability and were equally capable of capitalizing on the rulebook. Im a newer (weak) player and i usually lose a lot. You can assume that every reporting done my partner on our games skews the balance - not sure that is a fair representation to the designer or the playtest process. Maybe all the other games out there take place between opponents of very equal ability - though i doubt it.
Your playtesting can still work, although the playtest coordinator needs to be aware that they can't just lump your games into the pool & say "ah, perfect balance as it's 5-5". If you have a good feel for your skills relative to your opponent you can judge balance based on how close you got to a W:
  • Lost on last DR - unbalanced dog pro your side
  • Lost on next to last turn - pro your side
  • Lost on turn 4 of 7 - balanced
  • Lost on turn 2 of 7 - pro other side...
You get the idea. If you're self aware enough to evaluate how the scenario went relative to how you typically play your opponents, you can provide valuable playtest input. If you blindly say "went to the last DR, must be balanced" and ignore the fact that you're 2-27 vs. your opponent, your playtest input isn't that valuable, unless you provide enough context for the playtest coordinator to evaluate the situation.

But keep playtesting, provide lots of input on your game & context on your skills relative to your opponent and your game will improve. Playtesting will force you to reflect on the scenario just played and evaluate what went wrong/right which is a great way to learn and should preclude the "my dice were against me" syndrome that lots of folks fall into...
 

Bret Hildebran

Elder Member
Joined
Jan 31, 2003
Messages
4,884
Reaction score
1,279
Location
NE OH
Country
llUnited States
"Few Returned" was playtested more than any other recent ASL production. At least it was touted as such.
Was it really? I could certainly buy that it was playtested by MORE different people than any other pack, but I doubt it was actually playtested more than a typical Journal/AP etc. in total playings. Perhaps I'm mistaken there, but that's my feel. The playtesting was somewhat more in public because Mark was recruiting other playtesters & building buzz about the product which is not the case for a typical Journal. And FWIW, I think the way Mark playtested was a very good choice for that pack overall.

It's the dirty little secret of ASL that tons of playtesting won't improve the quality of scenarios or make them more balanced. Those of us in the know are aware of this, but the ASL public at large still swallows this lie, and are encouraged to keep doing so by those with an agenda. It makes me sick to see the best designers in the world hiding or exaggerating their numbers of playtests because ASL players have been brainwashed.
I think there's a happy medium somewhere. You want the scenario playtested by a broad enough group that the SSRs/VC are buttoned up with tight verbiage and a variety of tactics have been explored to ensure the scenario isn't broken. Too little playtesting runs the risk of missing something obvious there and a broken scenario. If you have a McGrath-Chaney dream matchup playtesting for you, you could likely get away with 1 or 2 playings perhaps, but they're far from the average ASLers.

Too much playtesting can likely also have issues - mostly determining what the signal is from amongst the noise. I've seen some scenarios (Danger Forward anyone?) broken by too much playtesting as significant changes made late based on the final playings can have the effect of oversteer on a scenario & sometimes push it off the road of balance into the ditch of dogginess. Who to listen to in playtesting reports and how not to overreact to the last playing, or even harder your own personal experiences in the scenario is an artform in itself for the playtest coordinator.

As Will pointed out quite well, personally I'd lean towards quality over quantity in playtesting. I'd listen to feedback on a Bendis-Pleva matchup over 10 playings from average ASLers that are prone to "came down to the last DR, it's balanced" syndrome, but may ignore the 5 consecutive snake eyes one side needed to get to that point or the 8 boneheaded move the favored side made to actually allow the scenario to be close instead of a blowout...
 

'Ol Fezziwig

Repressed Dissident
Joined
Nov 18, 2004
Messages
6,640
Reaction score
725
Location
hazy fold of reality
Country
llUnited States
the "my dice were against me" syndrome that lots of folks fall into...
The biggest error in playtesting, I think; some guys forget they're not supposed to be playing it competitively as much as critically.
 

daveramsey

Elder Member
Joined
Jun 10, 2006
Messages
1,805
Reaction score
1,037
Location
Hertfordshire
First name
Dave
Country
llUnited Kingdom
So if the top players decide a scenario is balanced, what about the rest of us down in the depths of ability?

Is "Angels at the Airfield" balanced between top players, but only played by normal players and as such, the essence of the attack missed by the majority of the US players? Scouts out, between two beginners will probably be a walk over for the defender everytime - how many times do you really think beginners will play it after their first time?

Balanced should mean balanced at all levels of play - there's no expectation that to play a balanced game both players must be experts. This is the problem - between novices, one scenario may look perfectly balanced, and yet as their experience grows the balance may swing in an apparent arms race.

Hence I fall back to it should be fun, and interesting (have options) for both players. To hell with the playtesting guys, just make us fun scenarios!

Would you more likely buy a pack called "Balanced-o-Rama! The perfectly balanced 10 scenarios - guaranteed to be even, over 100 playings or your money back!" or "Fun-o-Rama! 10 scenarios that will have you reminiscing for years to come."?

By the way, whilst you're saving all that time playtesting, you can also spend less time researching the historical nature of the scenario, too. Providing it "might" have happened, I'm happy with that. Man, I'm easy to please... :)
 

Stardragon99

Member
Joined
May 13, 2005
Messages
472
Reaction score
36
Location
Pyrenees
Country
llCanada
In essence i was saying that it is possible that ROAR ratings in no way are capable of 'statistically' representing balance and ultimately may be a poor reflection of true balance.
 

Pitman

Forum Guru
Joined
Jan 27, 2003
Messages
14,104
Reaction score
2,371
Location
Columbus, OH
Country
llUnited States
He claims that more playtesting = more balanced scenarios.
Well, what I actually said was that a scenario that has been playtested more times has a better chance of being balanced. There is never a guarantee when it comes to design and development. It's hard to see how someone could really argue with this point.

Yet his own pack belies this. If I may tease him a bit, one can only guess as to how awful it would have been if it hadn't been playtested a billion times. Perhaps Mark isn't a good enough designer to produce quality scenarios unless they've been playtested 60 times each. Work on it Mark. I've seen David Lamb come up with a balanced scenario after only two or three playtests.
Well, I guess what you mean to say is that it turns out that a scenario that had only been playtested 2-3 times actually was balanced. IMO, it is not possible to determine with any certainty that a scenario that has only been playtested 2 times is balanced. It's a crap shoot.


"Few Returned" was playtested more than any other recent ASL production. At least it was touted as such.
Not to my knowledge. I have no idea if it was playtested more than any other production. It did have more playtesters than any other, but that does not mean it had a higher total number of playtests.

The reason that it's huge ammount of playtesting was advertised was to take advantage of the average ASL player's gullible acceptance of the fallacy that massive playtesting makes scenarios more balanced. It doesn't. Otherwise "Few Returned" wouldn't be posting numbers virtually identical to scenario packs that have minimal playtesting.
I don't know to what extent MMP mentioned playtesting at all in its marketing of Few Returned. As for myself, most of the times I ever mentioned the playtesters it was simply to express how grateful I was to them.


Once again: it is the ability of the designer and his playtesters to know when a scenario is finished, and putting an appropriate stop to playtesting that determine it's quality.
Two times is not enough. It is not enough for people to try different strategies, to determine if a scenario can be broken, etc.


It's the dirty little secret of ASL that tons of playtesting won't improve the quality of scenarios or make them more balanced. Those of us in the know are aware of this, but the ASL public at large still swallows this lie, and are encouraged to keep doing so by those with an agenda. It makes me sick to see the best designers in the world hiding or exaggerating their numbers of playtests because ASL players have been brainwashed.
That's a fundamentally flawed assertion. It is certainly true that no amount of playtesting can guarantee a scenario will be balanced or fun, and it is certainly true that beyond a certain point you hit the law of diminishing returns. But I think very few people besides yourself would accept the notion that a scenario that has been playtested 2-3 times is more or equally likely to be balanced than a scenario that has been playtested 8-10 times.
 

Pitman

Forum Guru
Joined
Jan 27, 2003
Messages
14,104
Reaction score
2,371
Location
Columbus, OH
Country
llUnited States
Clearly you find it incomprehensible that a scenario can ever be recognized as finished so quickly. Since you lack the abiltiy to tell when a scenario is finished, you keep on playtesting it to death. Just because you're not skilled enough to pull it off, don't drag the rest of us down to your level.

A near-perfect scenario was designed. One of the best playtesters in the world put his stamp on it, and it succeeded because of their combined talents, and CONFIDENCE...not "luck". You act like we were lazy and just threw it out there hoping for the best and got lucky. That's not only untrue, it's insulting.
I haven't acted as if you, or anybody else, were lazy or genius or what have you. Unlike you, I have not been making this personal. I have not talked about you, or your skills or lack thereof, or any of your scenarios or packs, I have only been responding generically to statements such as "playtested 2-3 times."

But I will not back down from my belief that playtesting a scenario only 2-3 times is inadequate. It is not enough, really, even to test out different strategies, and if one doesn't at least playtest the main possible strategies, how can one truly have a grasp on whether or not it is balanced? One can't, in such a situation; one just has to hope for the best.
 

Pitman

Forum Guru
Joined
Jan 27, 2003
Messages
14,104
Reaction score
2,371
Location
Columbus, OH
Country
llUnited States
But given how far they both fall short of that unlikely goal, both the "Glennbo" method of relying on sheer talent and gut feeling, and the "Pitman" method of relying on dogged persistence, seem to be about equal as ways to achieve the goal... within our limited ability to statistically measure the outcomes.
I don't have any method and I am not touting myself as a better designer or developer than anyone else on the planet. I don't know why this is becoming a Mark vs Glenn thing, except that Glenn seems to want to make it so.

I simply don't believe that it is possible to say that a scenario has been fully playtested after only 2-3 playtests. I think that does allow adequate examination of the major aspects of the scenario, attack and defense, nor does it necessarily allow sufficient testing of changes made after the first or second playtest.

I think this applies whether someone is a genius designer like Glenn or a mediocre designer like (fill in the blank).
 

Pitman

Forum Guru
Joined
Jan 27, 2003
Messages
14,104
Reaction score
2,371
Location
Columbus, OH
Country
llUnited States
A thread that's been hijacked by a good asl discussion!

The thing that worries me about this whole topic is that the focus seems to be about balance and no one's mentioning the "fun" factor of the scenario.

Balanced scenarios can be un-fun, and fun scenarios can be unbalanced.
I think the focus has not been on that, Dave, because unlike balance, it is easier to determine if a scenario is actually fun to play with just a few playtests.
 

Pitman

Forum Guru
Joined
Jan 27, 2003
Messages
14,104
Reaction score
2,371
Location
Columbus, OH
Country
llUnited States
Of course the one underlying assumption for all playtesting and all ROAR reports was that the two players were of perfectly equal ability and were equally capable of capitalizing on the rulebook. Im a newer (weak) player and i usually lose a lot. You can assume that every reporting done my partner on our games skews the balance - not sure that is a fair representation to the designer or the playtest process. Maybe all the other games out there take place between opponents of very equal ability - though i doubt it.
That is not an assumption. Most designers try to take skill levels into account when evaluating playtests, I think.
 

Bret Hildebran

Elder Member
Joined
Jan 31, 2003
Messages
4,884
Reaction score
1,279
Location
NE OH
Country
llUnited States
So if the top players decide a scenario is balanced, what about the rest of us down in the depths of ability?
You have something to strive for?

Is "Angels at the Airfield" balanced between top players, but only played by normal players and as such, the essence of the attack missed by the majority of the US players?
Personally I don't think that one is balanced at any level, but I've only played the initial release - don't know if the MM '99 version is better (no record on ROAR for the reprint).

Scouts out, between two beginners will probably be a walk over for the defender everytime - how many times do you really think beginners will play it after their first time?
And the solution is to print something very pro-attacker so that a newbie who doesn't realize what smoke assets his vehicles have and how to cross a bridge under fire can still win? That's not really practical is it? Shouldn't we strive to teach the beginner what he could better do to win? After the beginner gets his butt kicked as the Ami & wonders how this barker was ever published, he could read the Scenario Replay in the Journal (or was it an annual?) and see how the incomparable J.R. Tracy solved the same problems and he learns a lot. Suddenly he's not a beginner anymore, but making his first steps towards Grognard status. Heck maybe he even plays it again as the Ami or next time he's on the attack with the Americans thinks to check out his smoke assets etc. & utilizes some of those fancy tactics and his game leaps a couple levels...

Balanced should mean balanced at all levels of play - there's no expectation that to play a balanced game both players must be experts. This is the problem - between novices, one scenario may look perfectly balanced, and yet as their experience grows the balance may swing in an apparent arms race.
Well that's a nice ideal, but relatively impractical in all but the simplest of scenarios IMO. Maybe it's true for something like Guards Counterattack with a relatively fixed setup and simple OB, but add toys, vehicles etc. and the complexities just get way too high to anticipate what a bad player won't know to do. Will he forget his smoke assets? Will he always prep? Will he leave his tanks exposed w/o infantry support? Who can tell? If we balance for one level of incompetence, is it unbalanced for a different type of bad play that we didn't anticipate? That would be tricky.

Obviously in my view, we should try to balance for top levels of play - the ideal play basically. If inexperienced players question the balance, there should be lessons they can learn to improve their play and improve the perceived balance of the scenario.

Saying a scenario is unbalanced by bad play by one side is essentially a given. And there will definitely be scenarios where between two newbies it's much easier to play one side or the other. Many scenarios have a learning curve that a certain strategy/tactic won't be obvious to an inexperienced player, but is necessary for that side to be competitive. Scenarios that have a steep curve for one side, but not the other will be perceived as unbalanced to newbies. Guess you're arguing that the learning curves for both sides should be comparable to keep the balance in relative parity, which isn't a bad idea, but is certainly easier said than done and would preclude a lot of scenarios from being published.

Hence I fall back to it should be fun, and interesting (have options) for both players. To hell with the playtesting guys, just make us fun scenarios!
Well this is without a doubt true. If it's not fun, don't bother. Calling a coin flip is perfectly balanced, but that doesn't make it fun.

The irony is that often its the toys in a scenario that are fun and the more toys the more likely it is to be misplayed by inexperienced players. i.e. my guess is the simple goal of fun likely inherently defeats your goal of balanced to all levels of play.

Would you more likely buy a pack called "Balanced-o-Rama! The perfectly balanced 10 scenarios - guaranteed to be even, over 100 playings or your money back!" or "Fun-o-Rama! 10 scenarios that will have you reminiscing for years to come."?
Hmmmm...Given the lessons of Greenland and Iceland, I'd be quite suspicious of "Fun-o-Rama!" as false advertising, but your point stands that if it's not fun it doesn't matter that much if it's balanced. By the same token though, I'm unlikely to play a fun dog multiple times either. Now fun and relatively balanced I may play quite a bit over the course of several years on the tourney scene...

Bottom line - I'm strongly in the camp of balancing for the top players and encouraging the rest of the ASL world to improve their play towards that ideal. Not to mention I don't really think balancing for bad play & ideal play is possible and balancing solely for bad play is just a terrible idea, plus is it really possible to predict all the forms of bad play and balance for that? But really, who do you want telling you your scenario is a 3-Legged-Unbalanced-Barker - Steve Pleva or some newbie playing his 3rd game?
 

Pitman

Forum Guru
Joined
Jan 27, 2003
Messages
14,104
Reaction score
2,371
Location
Columbus, OH
Country
llUnited States
Was it really? I could certainly buy that it was playtested by MORE different people than any other pack, but I doubt it was actually playtested more than a typical Journal/AP etc. in total playings. Perhaps I'm mistaken there, but that's my feel. The playtesting was somewhat more in public because Mark was recruiting other playtesters & building buzz about the product which is not the case for a typical Journal. And FWIW, I think the way Mark playtested was a very good choice for that pack overall.
I would say you are correct.
 

Tater

Elder Member
Joined
Oct 25, 2003
Messages
9,827
Reaction score
542
Location
Ardmore, TN
Country
llUnited States
But I will not back down from my belief that playtesting a scenario only 2-3 times is inadequate. It is not enough, really, even to test out different strategies, and if one doesn't at least playtest the main possible strategies, how can one truly have a grasp on whether or not it is balanced? One can't, in such a situation; one just has to hope for the best.
:laugh::laugh::laugh:...

WOW...I would sure hate to see what Mark might say if he WAS trying to make it personal! :nuts:

I think we need Mark and Glennbo in some "FRIDAY NIGHT SMACK-DOWN"!!!
 

Pitman

Forum Guru
Joined
Jan 27, 2003
Messages
14,104
Reaction score
2,371
Location
Columbus, OH
Country
llUnited States
Balanced should mean balanced at all levels of play - there's no expectation that to play a balanced game both players must be experts. This is the problem - between novices, one scenario may look perfectly balanced, and yet as their experience grows the balance may swing in an apparent arms race.
No, I have to reject this. You have to assume balanced given competent play, otherwise it is useless. Because novices are so incompetent at using armor on the attack, for example, a scenario that will be balanced for two novices will be very tough on the defender when played by two competent players.
 

Bret Hildebran

Elder Member
Joined
Jan 31, 2003
Messages
4,884
Reaction score
1,279
Location
NE OH
Country
llUnited States
In essence i was saying that it is possible that ROAR ratings in no way are capable of 'statistically' representing balance and ultimately may be a poor reflection of true balance.
This is certainly true - we have no way of knowing the matchups to determine if there was a skewing in sides. Maybe the experienced Groggie always takes the Germans so ROAR shows it as unbalanced, but it's really a side selection issue? (Note: in my experience this is somewhat unlikely as I'd guess 80%, or more, of scenarios sides are diced for, but that doesn't mean sides can't be skewed randomly particularly in scenarios with few playings)

Once we get to large enough sample sizes you hope that effect goes away, but it's merely a hope and not a certainty. Cool would be being able to pull out only those matchups between 2 top players to check on balance. I doubt we have enough samples to make that practical, if it were possible to mine the data that tightly anyway - WeASL likely could make it happen eventually I'd guess given enough data points...
 
Top