Skip to content

input_variational_dropout

allennlp.modules.input_variational_dropout

[SOURCE]


InputVariationalDropout

class InputVariationalDropout(torch.nn.Dropout)

Apply the dropout technique in Gal and Ghahramani, Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning to a 3D tensor.

This module accepts a 3D tensor of shape (batch_size, num_timesteps, embedding_dim) and samples a single dropout mask of shape (batch_size, embedding_dim) and applies it to every time step.

forward

class InputVariationalDropout(torch.nn.Dropout):
 | ...
 | def forward(self, input_tensor)

Apply dropout to input tensor.

Parameters

  • input_tensor : torch.FloatTensor
    A tensor of shape (batch_size, num_timesteps, embedding_dim)

Returns

  • output : torch.FloatTensor
    A tensor of shape (batch_size, num_timesteps, embedding_dim) with dropout applied.