input_variational_dropout

allennlp.modules.input_variational_dropout

[SOURCE]

---

InputVariationalDropout

class InputVariationalDropout(torch.nn.Dropout)

Apply the dropout technique in Gal and Ghahramani, Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning to a 3D tensor.

This module accepts a 3D tensor of shape (batch_size, num_timesteps, embedding_dim) and samples a single dropout mask of shape (batch_size, embedding_dim) and applies it to every time step.

forward

class InputVariationalDropout(torch.nn.Dropout):
 | ...
 | def forward(self, input_tensor)

Apply dropout to input tensor.

Parameters

• input_tensor : torch.FloatTensor
  A tensor of shape (batch_size, num_timesteps, embedding_dim)

Returns

• output : torch.FloatTensor
  A tensor of shape (batch_size, num_timesteps, embedding_dim) with dropout applied.