On the issue of whether or not to log "crappy" games into ROAR, I think that everybody is right in a sense. If the point of ROAR is to tell us whether a scenario is balanced under ideal conditions, then we probably don't want to add "crappy" records to ROAR, because there isn't any guarantee that the error that rules mistakes, etc, add is going to be symmetrically distributed around whatever the true balance is--and my guess would be that Srynerson is right that more often than not it won't be! In these sorts of cases, adding more records doesn't making things better.
On the other hand, I don't think that the best purpose of ROAR is to tell us whether a scenario is balanced under ideal conditions. "Ideal conditions" are probably something like "played by experience players with ample time to prepare who didn't make any rules mistakes," and players who can do that probably don't have a lot of need for ROAR to tell them whether a scenario is balanced in the first place! I think that ROAR is probably more helpful to people like me, who aren't good enough to look at a scenario card and know whether it's balanced, and who also aren't good enough to play it without making any avoidable errors. And for us, adding records from people like us makes ROAR more useful.
So imagine a scenario that depended on a perfect understanding of the Overrun rules. ASL pros could play it, and it's perfectly balanced, but if you don't understand all the defensive options available to infantry, it's a dog. All the people like me play it, and one side always wins. This is good information for our fellow dumb people to have, and it's hard to imagine where else we could get this information!
So all in all, I would encourage everybody to submit records of any scenario in which both players were trying to win (I think I would recommend against recording teaching scenarios if the teacher is trying to help his opponent win, but if he's just playing with one hand behind his back, then probably record it).
Also, here is a helpful tool to tell you whether there are enough records to say that a scenario might be unbalanced:
https://graphpad.com/quickcalcs/binomial1.cfm
For "number of successes", enter the recorded wins by either side (doesn't matter which).
For "number of trials", enter the total number of games played (technically the total number of games that didn't end in a tie, but it doesn't really matter).
Leave "probability" at 0.5 and hit calculate.
Then look at the "Two-tail P value" result. This is the probability that a balanced scenario would look like a dog just due to chance. If the number is low, then you might say that it's a dog. In my field (biology), we consider anything less than 0.05 to be low.
(Be careful with scenarios with a lot of playings, however. With these, you can get a low p-value even when the scenario isn't that tilted. For instance, if there are 350 playings of a scenario, and the attacker won 150 times, this has a quite low p-value, but the attacking is still winning 43% of the time, which isn't that unbalanced. So while the p-value gives you evidence that a scenario is unbalanced, it doesn't necessarily tell you how unbalanced it is.)