Skip to content

cnn_highway_encoder

allennlp.modules.seq2vec_encoders.cnn_highway_encoder

[SOURCE]


CnnHighwayEncoder

@Seq2VecEncoder.register("cnn-highway")
class CnnHighwayEncoder(Seq2VecEncoder):
 | def __init__(
 |     self,
 |     embedding_dim: int,
 |     filters: Sequence[Sequence[int]],
 |     num_highway: int,
 |     projection_dim: int,
 |     activation: str = "relu",
 |     projection_location: str = "after_highway",
 |     do_layer_norm: bool = False
 | ) -> None

The character CNN + highway encoder from Kim et al "Character aware neural language models" with an optional projection.

Registered as a Seq2VecEncoder with name "cnn-highway".

Parameters

  • embedding_dim : int
    The dimension of the initial character embedding.
  • filters : Sequence[Sequence[int]]
    A sequence of pairs (filter_width, num_filters).
  • num_highway : int
    The number of highway layers.
  • projection_dim : int
    The output dimension of the projection layer.
  • activation : str, optional (default = 'relu')
    The activation function for the convolutional layers.
  • projection_location : str, optional (default = 'after_highway')
    Where to apply the projection layer. Valid values are 'after_highway', 'after_cnn', and None.

forward

class CnnHighwayEncoder(Seq2VecEncoder):
 | ...
 | def forward(
 |     self,
 |     inputs: torch.Tensor,
 |     mask: torch.BoolTensor
 | ) -> Dict[str, torch.Tensor]

Compute context insensitive token embeddings for ELMo representations.

Parameters

  • inputs : torch.Tensor
    Shape (batch_size, num_characters, embedding_dim) Character embeddings representing the current batch.
  • mask : torch.BoolTensor
    Shape (batch_size, num_characters) Currently unused. The mask for characters is implicit. See TokenCharactersEncoder.forward.

Returns

  • encoding:
    Shape (batch_size, projection_dim) tensor with context-insensitive token representations.

get_input_dim

class CnnHighwayEncoder(Seq2VecEncoder):
 | ...
 | def get_input_dim(self) -> int

get_output_dim

class CnnHighwayEncoder(Seq2VecEncoder):
 | ...
 | def get_output_dim(self) -> int