conditional_random_field_wemission

allennlp.modules.conditional_random_field.conditional_random_field_wemission

Conditional random field with emission-based weighting

ConditionalRandomFieldWeightEmission¶

class ConditionalRandomFieldWeightEmission(ConditionalRandomField):
 | def __init__(
 |     self,
 |     num_tags: int,
 |     label_weights: List[float],
 |     constraints: List[Tuple[int, int]] = None,
 |     include_start_end_transitions: bool = True
 | ) -> None

This module uses the "forward-backward" algorithm to compute the log-likelihood of its inputs assuming a conditional random field model.

See, e.g. http://www.cs.columbia.edu/~mcollins/fb.pdf

This is a weighted version of ConditionalRandomField which accepts a label_weights parameter to be used in the loss function in order to give different weights for each token depending on its label. The method implemented here is based on the simple idea of weighting emission scores using the weight given for the corresponding tag.

There are two other sample weighting methods implemented. You can find more details about them in: https://eraldoluis.github.io/2022/05/10/weighted-crf.html

Parameters¶

num_tags : int
The number of tags.
label_weights : List[float]
A list of weights to be used in the loss function in order to give different weights for each token depending on its label. len(label_weights) must be equal to num_tags. This is useful to deal with highly unbalanced datasets. The method implemented here is based on the simple idea of weighting emission scores using the weight given for the corresponding tag.
constraints : List[Tuple[int, int]], optional (default = None)
An optional list of allowed transitions (from_tag_id, to_tag_id). These are applied to viterbi_tags() but do not affect forward(). These should be derived from allowed_transitions so that the start and end transitions are handled correctly for your tag type.
include_start_end_transitions : bool, optional (default = True)
Whether to include the start and end transition parameters.

forward¶

class ConditionalRandomFieldWeightEmission(ConditionalRandomField):
 | ...
 | def forward(
 |     self,
 |     inputs: torch.Tensor,
 |     tags: torch.Tensor,
 |     mask: torch.BoolTensor = None
 | ) -> torch.Tensor

Computes the log likelihood for the given batch of input sequences $(x,y)$

Args: inputs (torch.Tensor): (batch_size, sequence_length, num_tags) tensor of logits for the inputs $x$ tags (torch.Tensor): (batch_size, sequence_length) tensor of tags $y$ mask (torch.BoolTensor, optional): (batch_size, sequence_length) tensor of masking flags. Defaults to None.

Returns: torch.Tensor: (batch_size,) log likelihoods $log P(y|x)$ for each input