cnn_highway_encoder

allennlp.modules.seq2vec_encoders.cnn_highway_encoder

[SOURCE]

---

CnnHighwayEncoder

@Seq2VecEncoder.register("cnn-highway")
class CnnHighwayEncoder(Seq2VecEncoder):
 | def __init__(
 |     self,
 |     embedding_dim: int,
 |     filters: Sequence[Sequence[int]],
 |     num_highway: int,
 |     projection_dim: int,
 |     activation: str = "relu",
 |     projection_location: str = "after_highway",
 |     do_layer_norm: bool = False
 | ) -> None

The character CNN + highway encoder from Kim et al "Character aware neural language models" with an optional projection.

Registered as a Seq2VecEncoder with name "cnn-highway".

Parameters

• embedding_dim : int
  The dimension of the initial character embedding.
• filters : Sequence[Sequence[int]]
  A sequence of pairs (filter_width, num_filters).
• num_highway : int
  The number of highway layers.
• projection_dim : int
  The output dimension of the projection layer.
• activation : str, optional (default = 'relu')
  The activation function for the convolutional layers.
• projection_location : str, optional (default = 'after_highway')
  Where to apply the projection layer. Valid values are 'after_highway', 'after_cnn', and None.

forward

class CnnHighwayEncoder(Seq2VecEncoder):
 | ...
 | def forward(
 |     self,
 |     inputs: torch.Tensor,
 |     mask: torch.BoolTensor
 | ) -> Dict[str, torch.Tensor]

Compute context insensitive token embeddings for ELMo representations.

Parameters

• inputs : torch.Tensor
  Shape (batch_size, num_characters, embedding_dim) Character embeddings representing the current batch.
• mask : torch.BoolTensor
  Shape (batch_size, num_characters) Currently unused. The mask for characters is implicit. See TokenCharactersEncoder.forward.

Returns

• encoding:
  Shape (batch_size, projection_dim) tensor with context-insensitive token representations.

get_input_dim

class CnnHighwayEncoder(Seq2VecEncoder):
 | ...
 | def get_input_dim(self) -> int

get_output_dim

class CnnHighwayEncoder(Seq2VecEncoder):
 | ...
 | def get_output_dim(self) -> int