transformer_module

allennlp.modules.transformer.transformer_module

TransformerModule¶

class TransformerModule(torch.nn.Module):
 | def __init__(self, *args, **kwargs)

Base class to help with generalized loading of pretrained weights.

_huggingface_mapping is an optional mapping for each class, that determines any differences in the module names between the class modules and the huggingface model's modules.

_relevant_module is an optional str or list of str which contains the expected name of the module in the huggingface pretrained model. It can be a list to account for different names in different models. The search is carried out in the order of the list.

get_relevant_module¶

class TransformerModule(torch.nn.Module):
 | ...
 | @classmethod
 | def get_relevant_module(
 |     cls,
 |     pretrained_module: Union[str, torch.nn.Module],
 |     relevant_module: Optional[Union[str, List[str]]] = None,
 |     source="huggingface",
 |     mapping: Optional[Dict[str, str]] = None
 | )

Returns the relevant underlying module given a model name/object.

Parameters:¶

pretrained_module: Name of the transformer model containing the layer, or the actual layer (not the model object). relevant_module: Name of the desired module. Defaults to cls._relevant_module. source: Where the model came from. Default - huggingface. mapping: Optional mapping that determines any differences in the module names between the class modules and the input model's modules. Default - cls._huggingface_mapping

from_pretrained_module¶

class TransformerModule(torch.nn.Module):
 | ...
 | @classmethod
 | def from_pretrained_module(
 |     cls,
 |     pretrained_module: Union[str, torch.nn.Module],
 |     source="huggingface",
 |     mapping: Optional[Dict[str, str]] = None,
 |     **kwargs
 | )

Creates and returns an instance of the class, by using the weights (and the architecture, by default) of the pretrained_module. Optionally, the architecture can be changed by providing arguments.