NamespaceSwappingField( self, source_tokens: List[allennlp.data.tokenizers.token.Token], target_namespace: str, ) -> None
NamespaceSwappingField is used to map tokens in one namespace to tokens in another namespace.
It is used by seq2seq models with a copy mechanism that copies tokens from the source
sentence into the target sentence.
- source_tokens :
List[Token]The tokens from the source sentence.
- target_namespace :
strThe namespace that the tokens from the source sentence will be mapped to.
NamespaceSwappingField.as_tensor( self, padding_lengths: Dict[str, int], ) -> torch.Tensor
Given a set of specified padding lengths, actually pad the data in this field and return a torch Tensor (or a more complex data structure) of the correct shape. We also take a couple of parameters that are important when constructing torch Tensors.
- padding_lengths :
Dict[str, int]This dictionary will have the same keys that were produced in
get_padding_lengths. The values specify the lengths to use when padding each relevant dimension, aggregated across all instances in a batch.
NamespaceSwappingField.empty_field(self) -> 'NamespaceSwappingField'
ListField can pad the number of fields in a list (e.g., the number of answer
TextFields), we need a representation of an empty field of each type. This
returns that. This will only ever be called when we're to the point of calling
as_tensor, so you don't need to worry about
count_vocab_items, etc., being called on this empty field.
We make this an instance method instead of a static method so that if there is any state
in the Field, we can copy it over (e.g., the token indexers in
NamespaceSwappingField.get_padding_lengths(self) -> Dict[str, int]
If there are things in this field that need padding, note them here. In order to pad a
batch of instance, we get all of the lengths from the batch, take the max, and pad
everything to that length (or use a pre-specified maximum length). The return value is a
dictionary mapping keys to lengths, like
This is always called after :func:
Given a :class:
Vocabulary, converts all strings in this field into (typically) integers.
Field object, it does not return anything.
Field does not have any strings that need to be converted into indices, you do
not need to implement this method.