ChatGTP

Tin_Man

Recruit
Joined
May 14, 2023
Messages
13
Reaction score
25
Country
llUnited States
This may be a bit of a taboo subject but for those who are trying to learn ASL without a mentor, I have found that ChatGTP is pretty good at answering questions related to interpretation of basic rules. I haven't spent much time testing it with more difficult conditions but it does a pretty decent job with simple queries.
 

Jazz

Inactive
Joined
Feb 3, 2003
Messages
12,302
Reaction score
2,892
Location
The Empty Quarter
Country
llLithuania
How did you know if it answered your rulz question correctly? I mean, if you are asking it I assume that means you do not know the answer yourself?

Given enough questions/corrections, it may indeed someday in the very distant future be correct answering ASL rulz questions, but how will we know it has reached that point? I know of at least one instance where it answered a question incorrectly. I would have to dig for the exact reference, someplace here or on Discord IIRC.
 

DVexile

Elder Member
Joined
Mar 27, 2022
Messages
676
Reaction score
1,122
Location
Baltimore, MD
First name
Ken
Country
llUnited States
How did you know if it answered your rulz question correctly? I mean, if you are asking it I assume that means you do not know the answer yourself?
This is exactly the issue with asking ChatGPT pretty much any question about anything. Worse, because of how ChatGPT is designed it will always answer with essentially absolute certainty even when the answer is entirely wrong (and cite references that may not even support the wrong answer it has confidently given).

Remember - as a language model ChatGPT fundamentally is optimized at creating text that sounds like a correct answer. There is nothing about ChatGPT that ensures it creates text that is actually the correct answer. Because of the way these models work there is usually a good probability that the "sounds correct" response is the "actually correct" response but it is also not too difficult to get wrong responses.
 

Tuomo

Keeper of the Funk
Joined
Feb 10, 2003
Messages
4,772
Reaction score
5,888
Location
Rock Bottom
Country
llUnited States
Don't care. If ChatGPT supports the rules interpretation that helps me kill that Panther in the heat of the moment, you bet I'm quoting it.
 

Tin_Man

Recruit
Joined
May 14, 2023
Messages
13
Reaction score
25
Country
llUnited States
But....
Dont we run into the exact same problem when we get rules interpretations from people?
 

Robin Reeve

The Swiss Moron
Staff member
Moderator
Joined
Jul 26, 2003
Messages
20,004
Reaction score
6,137
Location
St-Légier
First name
Robin
Country
llSwitzerland
But....
Dont we run into the exact same problem when we get rules interpretations from people?
People who know the rules, within a community of players which self regulates itself are much more reliable than an AI which processes are unknown - and which, when tested, showed that it could give misleading answers.

Please provide ChatGPT answers to your rules questions, so that experienced players can evaluate them.
 

DVexile

Elder Member
Joined
Mar 27, 2022
Messages
676
Reaction score
1,122
Location
Baltimore, MD
First name
Ken
Country
llUnited States
But....
Dont we run into the exact same problem when we get rules interpretations from people?
The difference is that people have the goal of giving you the correct answer, though of course they may make mistakes or misinterpretations. ChatGPT itself quite honestly does not have the goal of giving you a correct answer at all. The designers of ChatGPT have the goal of creating something that probably gives you a correct answer, but the architecture of ChatGPT only does that indirectly. This probably seems like a subtle distinction, but it is pretty fundamental to the limits of things like ChatGPT.

When people make mistakes or misinterpretations they can engage in a rational discussion to support their interpretations and correct them. Other people chime in with correcting information or interpretations. Within the context of ASL rules this means the discussion follows a logical path to a correct conclusion or an agreement that the interpretation is ambiguous and needs errata or a Q&A.

When ChapGPT makes a mistake and you tell it that its first answer was wrong its response is typically a completely new answer seemingly from left field that may be correct or may just be differently incorrect. ChatGPT does not understand logic directly at all whereas people do. It is a language model and so it only understands logic as an inference from how articles on the web it was trained on happened to use logic in their content (correctly or incorrectly).

This is the key point of @Jazz's reply. If you take the time to interact with ChatGPT on a topic you are in fact an expert on already you will pretty soon find it really gives some odd but "sounds correct" answers. Because you are already an expert in the field you'll know this. If you aren't you'll be led further astray.

ChatGPT is different from doing say a Google search and parsing the responses yourself. The top Google results are typically written by experts and if you take just a little time you can understand the context of the referenced page and confirm that the answer is actually contextually relevant to your question. Sure, there is always a small chance the referenced page is just plain wrong but if this was the case ChatGPT would do no better (same training data).

When you ask ChatGPT there is an additional (pretty frequent) potential for error in that ChatGPT will base its answer on entirely the wrong context. So even if ChatGPT based its answer on text from an "correct" webpage it is very possible the webpage didn't actually match the context of your question at all.

Again, the very best distillation of the immense complexity behind ChatGPT that I've read from AI experts is that what things like ChatGPT do are produce output that "sounds like the correct answer". That is fundamentally what they do, they have no idea or mechanism to evaluate whether the answer is correct. Indirectly this often results in a correct answer, but with distressingly high frequency it does not. And as has been pointed out since having references is part of "sounding correct" ChatGPT will happily create non-existent references or point you at actual references that in fact do not support the answer ChatGPT gave you.
 

Tin_Man

Recruit
Joined
May 14, 2023
Messages
13
Reaction score
25
Country
llUnited States
I do agree with your point to a certain extent, Im just saying that even on this forum I have read varying opinions from experienced players who have different interpretations of various rules. With that being said, I also acknowledge that agreement on a particular rule interpretation amongst a community of playes is more vital to the game than any one view or interpratation.
 

Jazz

Inactive
Joined
Feb 3, 2003
Messages
12,302
Reaction score
2,892
Location
The Empty Quarter
Country
llLithuania
But....
Dont we run into the exact same problem when we get rules interpretations from people?
Good point.

With some people, possibly....but there are a number of people both here, on Discord, and/or the various FB ASL pages who have been playing and reading rules for multiple decades and are accepted experts. You need to go by what someone you trust to read the rulz correctly, which is the same thing as ChatGPT, except these various experts actually have a pedigree as to their expertise. Something that is very much lacking with ChatGPT/
 

Tin_Man

Recruit
Joined
May 14, 2023
Messages
13
Reaction score
25
Country
llUnited States
Opinions are something that seldom change...After reading all of your inciteful responses and giving ample thought to all of your points, I have to say that I find myself agreeing with all of your statements.
 

Tuomo

Keeper of the Funk
Joined
Feb 10, 2003
Messages
4,772
Reaction score
5,888
Location
Rock Bottom
Country
llUnited States
What do we win for being the earliest to predict that Tin_Man is a ChatGPT bot?
 

Tin_Man

Recruit
Joined
May 14, 2023
Messages
13
Reaction score
25
Country
llUnited States
If you can prove it, Ill send you my collection of Journals
 

Robin Reeve

The Swiss Moron
Staff member
Moderator
Joined
Jul 26, 2003
Messages
20,004
Reaction score
6,137
Location
St-Légier
First name
Robin
Country
llSwitzerland
I have read varying opinions from experienced players who have different interpretations of various rules.
There are debates, of course.
But 99% of the time, they are resolved by a common agreement.
Otherwise, the question is sent to MMP and Perry gives a quite definitive answer - or further on an erratum is published in a Journal, because there was something wrong or missing in the rules.
I don't recall different schools of rules interpretations, where unsolved questions are hanging in the air for a long time.
There has been no dispersion between players because of different and permanent disagreements about rules interpretations.
Solving rules questions has much more to do with logics than opinions. Opinions occur when people try to appeal to "reality" rather than apply the rules as written - which is a fallacious path to follow when trying to apply rules.
 

Tin_Man

Recruit
Joined
May 14, 2023
Messages
13
Reaction score
25
Country
llUnited States
well in that case, no sense in keeping these sealed any longer....Here I thought they would be worth something some day
 

Robin Reeve

The Swiss Moron
Staff member
Moderator
Joined
Jul 26, 2003
Messages
20,004
Reaction score
6,137
Location
St-Légier
First name
Robin
Country
llSwitzerland
Please provide ChatGPT answers to your rules questions, so that experienced players can evaluate them.
My suggestion is unanswered.
It would be interesting, though, to have examples that we could examine and evaluate.
Some of the presently debated questions - such as the MP cost for a VCA chamge on a woods-road - would be most interesting cases.
 

von Marwitz

Forum Guru
Joined
Nov 25, 2010
Messages
14,722
Reaction score
11,016
Location
Kraut Corner
Country
llUkraine
My suggestion is unanswered.
It would be interesting, though, to have examples that we could examine and evaluate.
Some of the presently debated questions - such as the MP cost for a VCA chamge on a woods-road - would be most interesting cases.
ChatGPT's answer to the MP cost for VCA change on Woods Road would surely be interesting.

But I reckon that ChatGPT would run out of MPs before emerging with a sound answer to explain the twists and turns of the underlying rules... ;)

von Marwitz
 

Tin_Man

Recruit
Joined
May 14, 2023
Messages
13
Reaction score
25
Country
llUnited States
This whole thread has really went off of the rails. My original post was geared to towards the new player who is struggling to understand the basic mechanics of this game. We all need to admit that ASL rules are written with a sort of "legal-ese" which, until one used to, can be a little intimidating, All I was suggesting was that ChatGtp could be of some use in aiding in the understanding of "basic" rules and mechanics for "the new player".
 
Top