Conditional random field with emission- and transition-based weighting
class ConditionalRandomFieldWeightTrans(ConditionalRandomField): | def __init__( | self, | num_tags: int, | label_weights: List[float], | constraints: List[Tuple[int, int]] = None, | include_start_end_transitions: bool = True | ) -> None
This module uses the "forward-backward" algorithm to compute the log-likelihood of its inputs assuming a conditional random field model.
See, e.g. http://www.cs.columbia.edu/~mcollins/fb.pdf
This is a weighted version of
ConditionalRandomField which accepts a
parameter to be used in the loss function in order to give different weights for each
token depending on its label. The method implemented here is based on the simple idea
of weighting emission and transition scores using the weight given for the
There are two other sample weighting methods implemented. You can find more details about them in: https://eraldoluis.github.io/2022/05/10/weighted-crf.html
- num_tags :
The number of tags.
- label_weights :
A list of weights to be used in the loss function in order to give different weights for each token depending on its label.
len(label_weights)must be equal to
num_tags. This is useful to deal with highly unbalanced datasets. The method implemented here is based on the simple idea of weighting emission and transition scores using the weight given for the corresponding tag.
- constraints :
List[Tuple[int, int]], optional (default =
An optional list of allowed transitions (from_tag_id, to_tag_id). These are applied to
viterbi_tags()but do not affect
forward(). These should be derived from
allowed_transitionsso that the start and end transitions are handled correctly for your tag type.
- include_start_end_transitions :
bool, optional (default =
Whether to include the start and end transition parameters.
class ConditionalRandomFieldWeightTrans(ConditionalRandomField): | ... | def forward( | self, | inputs: torch.Tensor, | tags: torch.Tensor, | mask: torch.BoolTensor = None | ) -> torch.Tensor
Computes the log likelihood for the given batch of input sequences $(x,y)$
Args: inputs (torch.Tensor): (batch_size, sequence_length, num_tags) tensor of logits for the inputs $x$ tags (torch.Tensor): (batch_size, sequence_length) tensor of tags $y$ mask (torch.BoolTensor, optional): (batch_size, sequence_length) tensor of masking flags. Defaults to None.
Returns: torch.Tensor: (batch_size,) log likelihoods $log P(y|x)$ for each input