drop_em_and_f1

allennlp_models.rc.metrics.drop_em_and_f1

[SOURCE]

---

DropEmAndF1

@Metric.register("drop")
class DropEmAndF1(Metric):
 | def __init__(self) -> None

This Metric takes the best span string computed by a model, along with the answer strings labeled in the data, and computes exact match and F1 score using the official DROP evaluator (which has special handling for numbers and for questions with multiple answer spans, among other things).

__call__

class DropEmAndF1(Metric):
 | ...
 | @overrides
 | def __call__(
 |     self,
 |     prediction: Union[str, List],
 |     ground_truths: List
 | )

Parametersprediction: ``Union[str, List]``

The predicted answer from the model evaluated. This could be a string, or a list of string
when multiple spans are predicted as answer.

ground_truths: List All the ground truth answer annotations.

get_metric

class DropEmAndF1(Metric):
 | ...
 | @overrides
 | def get_metric(self, reset: bool = False) -> Tuple[float, float]

ReturnsAverage exact match and F1 score (in that order) as computed by the official DROP script

over all inputs.

reset

class DropEmAndF1(Metric):
 | ...
 | @overrides
 | def reset(self)