@Metric.register("drop") class DropEmAndF1(Metric): | def __init__(self) -> None
Metric takes the best span string computed by a model, along with the answer
strings labeled in the data, and computes exact match and F1 score using the official DROP
evaluator (which has special handling for numbers and for questions with multiple answer spans,
among other things).
class DropEmAndF1(Metric): | ... | def __call__( | self, | prediction: Union[str, List], | ground_truths: List | )
Parametersprediction: ``Union[str, List]``¶
The predicted answer from the model evaluated. This could be a string, or a list of string when multiple spans are predicted as answer.
All the ground truth answer annotations.
class DropEmAndF1(Metric): | ... | def get_metric(self, reset: bool = False) -> Tuple[float, float]
ReturnsAverage exact match and F1 score (in that order) as computed by the official DROP script¶
over all inputs.
class DropEmAndF1(Metric): | ... | def reset(self)