APCR (ASL Player Comparative Ratings)

Aaron Cleavin

Elder Member
Joined
Sep 27, 2004
Messages
3,093
Reaction score
570
Location
Sydney
Country
llAustralia
All

I am in the process (well advanced) of building a very generic ASL player ratings engine based on the GLICKO Methodology
and covering both friendly and tournament play.

The GLICKO methodology is an improvement over the original ELO (Chess Ratings system)


It is designed to take account not only of the ratings difference of players when computing Rating changes but also the deviation (Volatility) of a players rating.
It is designed for computation of rating over a period rather than game by game (I have chosen 2 month periods as per the papers recommendations).

The focus initially is on building the engine and getting a sizable of data ingested and validated to fairly rigorous standards.
Every game recorded must have
Playing Date
Attacking player
Defending Player
Scenario Code
Result (Attacker wins of loses (W/L))
GameType (Tournament/Friendly)
TournamentCode (If a Tournament Game)

Methods of submitting tournament games going forward are under consideration, whether only TD's or players as well can submitt tourney results is very much TBD.
No three player games or draws are supported

Each Player must have a Nationality
Each Scenario must have 1-2 Attacking side, 1-2 Defending Sides, 1-2 Designers, A publication, a code , a name

The core db objects are
Publisher, Publication, Nationality, Person, Scenario, Game, Gametype, Tournament, Rating

Once this initial seed is done, work will commence on the Web interface to the system. ultimately it will likely be hosted on AWS (Amazon Web Services)

Security privacy and respect for others ways of looking at things, catering fro as many people as possible with divergent view on the functionality and usefulness of such and engine
will all be core values.

I have received an large number of offers of assistance on this undertaking and once the seed is complete (April 30th 2019) I will be really actively engaging on these offers.

Status and detail reports to follow

What I have covered here is very much just the top layer I can assure though that the db and engine design and consistency checking is very much at a professional level, I have put about 100 hours into this so far and it is much further along than I had expected.

For those interested the Core stack is
PostgreSQL : db , SQL, PL/pgSQL
Web interface: Python/Django
A whole lot of Excel for Data Cleansing Seeding and validation.
 

Aaron Cleavin

Elder Member
Joined
Sep 27, 2004
Messages
3,093
Reaction score
570
Location
Sydney
Country
llAustralia
Status Update: 30 march 2019

Core Engine and db object complete
Seed Data added and validated

Core table row counts
Publisher: 37
Publication: 936
Scenario: 1297
Person: 1984
Game: 2159
Tournament: 10

Ratings: 66298 (Computed on 2 month basis for each player since a players first game for Friendly and Tournie)
Full historical Computation takes <10 seconds

Next Steps:
1500 Scenarios, 400 tournies & 10000 + games to be Data cleansed and added
(Have data already cleansing expected to take 2-3 days)

If anyone wants a ratings history, happy to provide a report for now though the games data is dominantly from 6 seed players.
If a file is provided with the format as per previous post I can process within a day or so and report on the results
 

Matt Book

Senior Member
Joined
Nov 9, 2004
Messages
1,777
Reaction score
233
Country
llUnited States
Will the scenario format entered include balance and ABS used?
 

Pacman Ghost

Member
Joined
Feb 25, 2017
Messages
228
Reaction score
110
Location
A maze of twisty little passages, all alike
Country
llAustralia
This looks pretty interesting.

Are you going to open-source it? I'm pretty flat out right now, but I do a lot of data work and might be able to contribute at some point.

Is there going to be an API? IIRC, there was discussion in another thread about how all this should work, but if the data is freely available, people can just do it however they want :) More seriously, I could e.g. integrate it into vasl-templates.
 

Aaron Cleavin

Elder Member
Joined
Sep 27, 2004
Messages
3,093
Reaction score
570
Location
Sydney
Country
llAustralia
Will the scenario format entered include balance and ABS used?
No as the information is seldom captured and hard to track meaning of, I think scenarios do come a little more balanced over time as people use Balance/ABS, and
this is another reason that some scenarios on ROAR appear a little more balanced than they make actually be without any balance provisions. It is a good candidate for future development though, it wouldn't be too hard to add.
 

Honosbinda

Member
Joined
Mar 15, 2014
Messages
500
Reaction score
81
Location
Eastbourne Sussex UK
Country
ll
It's true that AREA rating here http://www.asl-area.org/Area.html hasn't been updated since 2017. Is this data included in this new model? I didn't see that it had, but hopefully it is and it should be.

People have years of investment in building their AREA ratings and as far as I know a couple of years of tournament results are still pending to be added and ratings recalculated. If my understanding is correct, all tourneys in England submit to AREA and the last time I went to Bitter Ender, they were still seeding tournaments based on AREA.

In the case of AREA, only tournament play is rated and only TDs can submit.

I'm all in favor of continuing some kind of player rating system in ASL; it's unfortunate AREA has come to the situation it has since 2017. If it's not possible to add the AREA results and AREA is defunct, so be it. But if not, then we'll have two competing systems out there for this. That would be troubling.
 

hongkongwargamer

Forum Guru
Joined
Apr 4, 2013
Messages
5,767
Reaction score
2,870
Location
17 hexes
Country
llPortugal
It's true that AREA rating here http://www.asl-area.org/Area.html hasn't been updated since 2017. Is this data included in this new model? I didn't see that it had, but hopefully it is and it should be.

People have years of investment in building their AREA ratings and as far as I know a couple of years of tournament results are still pending to be added and ratings recalculated. If my understanding is correct, all tourneys in England submit to AREA and the last time I went to Bitter Ender, they were still seeding tournaments based on AREA.

In the case of AREA, only tournament play is rated and only TDs can submit.

I'm all in favor of continuing some kind of player rating system in ASL; it's unfortunate AREA has come to the situation it has since 2017. If it's not possible to add the AREA results and AREA is defunct, so be it. But if not, then we'll have two competing systems out there for this. That would be troubling.
You need to be live to compete. I don't think AREA is live, hence the start of this conversation/thread: http://www.gamesquad.com/forums/index.php?threads/area-a-small-info.143586/#post-1910330 .
 

Honosbinda

Member
Joined
Mar 15, 2014
Messages
500
Reaction score
81
Location
Eastbourne Sussex UK
Country
ll
You need to be live to compete. I don't think AREA is live, hence the start of this conversation/thread: http://www.gamesquad.com/forums/index.php?threads/area-a-small-info.143586/#post-1910330 .
Thanks for that link. I vaguely remember looking at that over a year ago but must have lost my tracking on it somehow to have seen the more recent entries

In that discussion, there is no resolution that AREA is dead, merely dormant.

I'm with ECZ in that it is dormant, waiting for someone to take over from Bruno. I had had the impression he was going to revive himself and continue the mission, but based on new info in that conversation, it seems not.

If someone needs to step up to the plate and get the data entered, then that's what can be done. There are over a decade of results for tournaments from all over the world in that data.

I'll do it. Happy to have someone check my work, of course.

Alternatively, the data from AREA can be recalculated and added to the database for this new method, if possible. There is no good reason to forsake this data and start a new basis without it.

AREA has functioned just fine in seeding tournaments and measuring the relative skills of players. Hence, we don't need a new system if it's not going to account for the years of data we already have.

Marc
 

dlazov

Elder Member
Joined
Mar 22, 2004
Messages
7,545
Reaction score
871
Location
Toledo, Ohio
First name
Don
Country
llUnited States
By trade I am a software engineer specializing in both java and JavaScript with Angular and Material Design front end experience.

Not sure what front end you’ll use but if it’s Angular and you need some help let me know.
 

Pacman Ghost

Member
Joined
Feb 25, 2017
Messages
228
Reaction score
110
Location
A maze of twisty little passages, all alike
Country
llAustralia
Just thinking out loud about some of things we could do with this data (i.e. take it FWIW :-/)...
  • use it as a definitive record of scenarios and their ID's (and an API would allow other programs access to it)
  • do a more sophisticated analysis of balance i.e. not just win/loss record, but taking into account the relative strength of each player for each win/loss
  • track the increase/decrease of a scenario's popularity over time, perhaps grouped by publisher
  • track which scenarios have been used for tournaments
  • track a person's win/loss record against another person (perhaps taking into scenario balance as described above) e.g. might be nice to know, if you play with the same group of people a lot e.g. at a club
  • track win/loss record between countries/states e.g. Australia vs. New Zealand, Texas vs. Montana :)
I guess my point is that this information can be used for a lot more than just calculating rankings.

An API would let me integrate this information into vasl-templates. People might not want it in the VASL scenario itself, but it'd be neat to have it in an info window while you're setting the scenario... 🆒 Or if the API was aware that it could be called by a label within VASSAL, return some archaic HTML that rendered the latest information into your scenario.
 

ecz

Partisan Captain
Joined
Aug 31, 2003
Messages
4,303
Reaction score
395
Location
Italy
Country
llItaly
(...)

The focus initially is on building the engine and getting a sizable of data ingested and validated to fairly rigorous standards.
Every game recorded must have
Playing Date
Attacking player
Defending Player
Scenario Code
Result (Attacker wins of loses (W/L))
GameType (Tournament/Friendly)
TournamentCode (If a Tournament Game)

(...)
No three player games or draws are supported
(...)
well done
two things:
1) draws happen in tournament games. They are not encouraged by nobody, but often they are the best and not painful (for TDs and players) solution.
As draws happen also in Chess, I suppose is not that hard implement this function for ASL.

2) I suggest that the first player named is the Axis/North Korean/"bad" side by default, while the second is always the one playing the Allied/US/Partisan/"Good" side.

This makes more simple the registration process:

If you read Catanzaro - Cleavin 1-0 ASL 1 VSL19 it means that Catanzaro playing as Finnish beated Cleavin (Russian) during VASLeague 2019 in scenario 1 Fighting Withdrawal ( yes I hope someone will play vs me in tournament scenario 1 as Russian :) ) .
I believe it's annoying (and a little redundant) write who's the attacker or who's the defender for all scenarios because requires the TDs actually check or know the scenario. Or you must ask the players to specify this. But I know by experience that, sometime, is hard to get detailed infos by all players. I think it's better try to make the registration process as quick and easy as possible even at the cost to get less data.

Finally a new rewrite of tournament names associated to the results, possibly in a summary version of the names in code, it's really useful.
For example all games played for VASLeague could be simply coded with VSL+"year" with no further distinctions between TOP, east or west leagues, for example. Any tournament could have an individual code name.

And last but not least as said above I think we really need to save and import in the new APCR all data at the moment stored in the old dormient AREA.
 

Honosbinda

Member
Joined
Mar 15, 2014
Messages
500
Reaction score
81
Location
Eastbourne Sussex UK
Country
ll
well done
two things:
1) draws happen in tournament games. They are not encouraged by nobody, but often they are the best and not painful (for TDs and players) solution.
As draws happen also in Chess, I suppose is not that hard implement this function for ASL.

2) I suggest that the first player named is the Axis/North Korean/"bad" side by default, while the second is always the one playing the Allied/US/Partisan/"Good" side.

This makes more simple the registration process:

If you read Catanzaro - Cleavin 1-0 ASL 1 VSL19 it means that Catanzaro playing as Finnish beated Cleavin (Russian) during VASLeague 2019 in scenario 1 Fighting Withdrawal ( yes I hope someone will play vs me in tournament scenario 1 as Russian :) ) .
I believe it's annoying (and a little redundant) write who's the attacker or who's the defender for all scenarios because requires the TDs actually check or know the scenario. Or you must ask the players to specify this. But I know by experience that, sometime, is hard to get detailed infos by all players. I think it's better try to make the registration process as quick and easy as possible even at the cost to get less data.

Finally a new rewrite of tournament names associated to the results, possibly in a summary version of the names in code, it's really useful.
For example all games played for VASLeague could be simply coded with VSL+"year" with no further distinctions between TOP, east or west leagues, for example. Any tournament could have an individual code name.

And last but not least as said above I think we really need to save and import in the new APCR all data at the moment stored in the old dormient AREA.
I don't see any reason to change from AREA on the basis of remarks so far.
AREA can be revitalized and maintained.
At the very least, no matter what, AREA ratings must be updated to include the last two years of results, so we have a benchmark to compare those results with this new method based on chess, which, by the way, has no random element, so I'm not convinced that it is better by any means.

I am sure everyone would like to see what the new ratings would be after being transmogrified by this new method. Wouldn't it be interesting if Steve Pleva was all of a sudden ranked lower from #1? :) :) :)

No, we absolutely need AREA as a benchmark.
 

von Marwitz

Forum Guru
Joined
Nov 25, 2010
Messages
9,854
Reaction score
3,076
Location
Kraut Corner
Country
llGibraltar
1) draws happen in tournament games. They are not encouraged by nobody, but often they are the best and not painful (for TDs and players) solution.
As draws happen also in Chess, I suppose is not that hard implement this function for ASL.

2) I suggest that the first player named is the Axis/North Korean/"bad" side by default, while the second is always the one playing the Allied/US/Partisan/"Good" side.
1) Good point!

2) More fool proof would be Attacker first, Defender second IMHO. How was it handled by Bruno's AREA?

von Marwitz
 

Aaron Cleavin

Elder Member
Joined
Sep 27, 2004
Messages
3,093
Reaction score
570
Location
Sydney
Country
llAustralia
By trade I am a software engineer specializing in both java and JavaScript with Angular and Material Design front end experience.

Not sure what front end you’ll use but if it’s Angular and you need some help let me know.
That would be my expectation too.
It is readily done yes, one issue is the data is incomplete in terms of scenario codes and attacking defending sides, probably 85-90% is usable
 

hongkongwargamer

Forum Guru
Joined
Apr 4, 2013
Messages
5,767
Reaction score
2,870
Location
17 hexes
Country
llPortugal
2) I suggest that the first player named is the Axis/North Korean/"bad" side by default, while the second is always the one playing the Allied/US/Partisan/"Good" side.
I disagree, I think player one should be whatever player one is on the scenario card.

I agree with Zov .. eg with Chinese Civil War scenarios ie Red Chinese vs GMT - you might start a new war trying to label whichever side as "bad". I am sure there are other "situations" ..
 

Aaron Cleavin

Elder Member
Joined
Sep 27, 2004
Messages
3,093
Reaction score
570
Location
Sydney
Country
llAustralia
well done
two things:
1) draws happen in tournament games. They are not encouraged by nobody, but often they are the best and not painful (for TDs and players) solution.
As draws happen also in Chess, I suppose is not that hard implement this function for ASL.

2) I suggest that the first player named is the Axis/North Korean/"bad" side by default, while the second is always the one playing the Allied/US/Partisan/"Good" side.

This makes more simple the registration process:

If you read Catanzaro - Cleavin 1-0 ASL 1 VSL19 it means that Catanzaro playing as Finnish beated Cleavin (Russian) during VASLeague 2019 in scenario 1 Fighting Withdrawal ( yes I hope someone will play vs me in tournament scenario 1 as Russian :) ) .
I believe it's annoying (and a little redundant) write who's the attacker or who's the defender for all scenarios because requires the TDs actually check or know the scenario. Or you must ask the players to specify this. But I know by experience that, sometime, is hard to get detailed infos by all players. I think it's better try to make the registration process as quick and easy as possible even at the cost to get less data.

Finally a new rewrite of tournament names associated to the results, possibly in a summary version of the names in code, it's really useful.
For example all games played for VASLeague could be simply coded with VSL+"year" with no further distinctions between TOP, east or west leagues, for example. Any tournament could have an individual code name.

And last but not least as said above I think we really need to save and import in the new APCR all data at the moment stored in the old dormient AREA.
A draw is technically an incomplete game, as <<1% of ASL scenarios actually allow a draw and should not be used in tournament play, incomplete games should not be recorded, coding in draws is not very hard but I am reluctant to record incomplete games. One way would be to record draws but NOT included them in any rating computations.

The Finnish and others swapped Sides (Axis/Allied) so Attacker and defender is much better, it is just as easy to record attacker and defender. TD's should know the scenario played at least in terms of Validating the Scenario.

APCR has full scenario Data (1500 now and 2700) within a few days so a Play Record (Based on Handle (First Name + Last Name as the are many Duplicate last names) would look like

01-Apr-2019, Enrico Catanarzo, Aaron Cleavin, W, VSL19 fairly simple really, if an error is made on whether the player was attacking or defending it will affect scenario statistics a little but not ratings
as a s long as the player put first was the one winning or losing (or possibly drawing) then ratings will be correctly computed.

Bringing in AREA data in is pretty easy but only 85-90% of AREA Data is usable owing to incomplete / wrong or missing scenario codes or attacking/defending sides.
 

Aaron Cleavin

Elder Member
Joined
Sep 27, 2004
Messages
3,093
Reaction score
570
Location
Sydney
Country
llAustralia
To clarify around Draws, In Chess a stronger player will lose some ranking when drawing with a weaker player, as he might have some expectation of winning.
If a draw in fact represents and incomplete game then no exchange of ratings should occur.
This is a key design decision which if why I have been reluctant to include draws for now.

I can see Enrico's predicament though, soo -- If a game was started (A scenario and sides were chosen) then we could include it as an "I" incomplete game, recorded in system, but not used in any rating calculation.
 
Last edited:
Top