Skip to content





class DropEmAndF1(Metric):
 | def __init__(self) -> None

This Metric takes the best span string computed by a model, along with the answer strings labeled in the data, and computes exact match and F1 score using the official DROP evaluator (which has special handling for numbers and for questions with multiple answer spans, among other things).


class DropEmAndF1(Metric):
 | ...
 | def __call__(
 |     self,
 |     prediction: Union[str, List],
 |     ground_truths: List
 | )

Parametersprediction: ``Union[str, List]``

The predicted answer from the model evaluated. This could be a string, or a list of string
when multiple spans are predicted as answer.

ground_truths: List All the ground truth answer annotations.


class DropEmAndF1(Metric):
 | ...
 | def get_metric(self, reset: bool = False) -> Tuple[float, float]

ReturnsAverage exact match and F1 score (in that order) as computed by the official DROP script

over all inputs.


class DropEmAndF1(Metric):
 | ...
 | def reset(self)