token_characters_encoder

allennlp.modules.token_embedders.token_characters_encoder

[SOURCE]

---

TokenCharactersEncoder

@TokenEmbedder.register("character_encoding")
class TokenCharactersEncoder(TokenEmbedder):
 | def __init__(
 |     self,
 |     embedding: Embedding,
 |     encoder: Seq2VecEncoder,
 |     dropout: float = 0.0
 | ) -> None

A TokenCharactersEncoder takes the output of a TokenCharactersIndexer, which is a tensor of shape (batch_size, num_tokens, num_characters), embeds the characters, runs a token-level encoder, and returns the result, which is a tensor of shape (batch_size, num_tokens, encoding_dim). We also optionally apply dropout after the token-level encoder.

We take the embedding and encoding modules as input, so this class is itself quite simple.

Registered as a TokenEmbedder with name "character_encoding".

get_output_dim

class TokenCharactersEncoder(TokenEmbedder):
 | ...
 | def get_output_dim(self) -> int

forward

class TokenCharactersEncoder(TokenEmbedder):
 | ...
 | def forward(self, token_characters: torch.Tensor) -> torch.Tensor