drop_em_and_f1
allennlp_models.rc.metrics.drop_em_and_f1
DropEmAndF1#
@Metric.register("drop")
class DropEmAndF1(Metric):
| def __init__(self) -> None
This Metric
takes the best span string computed by a model, along with the answer
strings labeled in the data, and computes exact match and F1 score using the official DROP
evaluator (which has special handling for numbers and for questions with multiple answer spans,
among other things).
__call__#
class DropEmAndF1(Metric):
| ...
| def __call__(
| self,
| prediction: Union[str, List],
| ground_truths: List
| )
Parametersprediction: ``Union[str, List]``¶
The predicted answer from the model evaluated. This could be a string, or a list of string
when multiple spans are predicted as answer.
ground_truths: List
All the ground truth answer annotations.
get_metric#
class DropEmAndF1(Metric):
| ...
| def get_metric(self, reset: bool = False) -> Tuple[float, float]
ReturnsAverage exact match and F1 score (in that order) as computed by the official DROP script¶
over all inputs.
reset#
class DropEmAndF1(Metric):
| ...
| def reset(self)