Consider definitions of recall/precision and true/false positives/negatives #17
Description
One quirk of the OPD is that for any given form, it doesn't provide an exhaustive set of analyses when a form is ambiguous, usually just one of them. So the cases where the model provide these additional (objectively true positive) analyses, it is wrongly considered to be a fail (false positive) just because it happens not to be noted in the OPD. I think we might want to redefine false positives as cases where we are both missing the analysis from the OPD AND we have an unexpected result that is something other than "+?" (two examples: izhiwaned and izhiwanen). So we are failing both to provide the correct analysis AND providing (one or more) wrong one.
If a form has more than one analysis in the OPD, and the FST is missing both, I think we want to count this as two as "false negatives/positives". Similarly if we get both correct, we want to count two "true positives". I guess in summary we want to consider each form-analysis pair from the OPD separately. However, if the FST generates more than one unexpected form, I think we want to just count that as a singular failure to get the correct form (assuming the definition of false positive as above).
False negatives should be (I think) the combination of a missing analysis AND failing to provide an analysis.