Skip to content

transformer_module

allennlp.modules.transformer.transformer_module

[SOURCE]

TransformerModule

class TransformerModule(Module)

Base class to help with generalized loading of pretrained weights.

Subclasses should override _from_config() if you want to instantiate them with from_pretrained_module().

_pretrained_mapping

class TransformerModule(Module):
 | ...
 | _pretrained_mapping: Dict[str, str] = {}

An optional mapping for each class that determines any differences in the module names between the class modules and the HuggingFace model's modules. Keys correspond to HuggingFace submodule names, values correspond to submodules names of this module.

_pretrained_relevant_module

class TransformerModule(Module):
 | ...
 | _pretrained_relevant_module: Optional[Union[str, List[str]]] = None

An optional string or list of strings which contains the expected name of the module in the HuggingFace pretrained model. It can be a list to account for different names in different models. The search is carried out in the order of the list.

_pretrained_ignore

class TransformerModule(Module):
 | ...
 | _pretrained_ignore: Optional[List[str]] = None

An optional list of regular expressions that define which weights to ignore from a pretrained state_dict.

_pretrained_allow_missing

class TransformerModule(Module):
 | ...
 | _pretrained_allow_missing: Optional[List[str]] = None

An optional list of regular expressions that specifies which weights are allowed to be missing from a pretrained state dictionary.

_from_config

class TransformerModule(Module):
 | ...
 | @classmethod
 | def _from_config(
 |     cls: Type[_T],
 |     config: "PretrainedConfig",
 |     **kwargs
 | ) -> _T

Instantiate this module from a HuggingFace config. Subclasses should override this method if you want to be able to instantiate them with from_pretrained_module().

from_pretrained_module

class TransformerModule(Module):
 | ...
 | @classmethod
 | def from_pretrained_module(
 |     cls: Type[_T],
 |     model_name: str,
 |     *, load_weights: bool = True,
 |     *, weights_path: Optional[Union[str, PathLike]] = None,
 |     *, auto_config_kwargs: Optional[Dict[str, Any]] = None,
 |     *, mapping: Optional[Dict[str, str]] = None,
 |     *, relevant_module: Optional[Union[str, List[str]]] = None,
 |     *, ignore: Optional[List[str]] = None,
 |     *, allow_missing: Optional[List[str]] = None,
 |     *, strict: bool = True,
 |     **kwargs,
 |     *, ,
 | ) -> _T

Initialize this module from a corresponding model on HuggingFace.

Note

This method is only available for subclasses that implement _from_config(). Otherwise a NotImplementedError will be raised.

Parameters

  • model_name : str
    The model identifier or path.

  • load_weights : bool, optional (default = True)
    Whether to download and load the pretrained weights. If False, the weights are left uninitialized.

  • weights_path : Optional[Union[str, PathLike]], optional (default = None)
    When load_weights is True, this can be set to override the weights file. Otherwise the default weights from the pretrained model are used.

  • auto_config_kwargs : Optional[Dict[str, Any]], optional (default = None)
    Optional key-word arguments to pass to transformers.AutoConfig.from_pretrained() to load the pretrained model's configuration file.

  • mapping : Optional[Dict[str, str]], optional (default = None)
    Optional mapping that determines any differences in the submodule names between this module and the pretrained model from HuggingFace. If not given, the class's default is used: cls._pretrained_mapping.

  • relevant_module : Optional[str], optional (default = None)
    An optional submodule of the HuggingFace module to initialize weights from. This is only relevant when load_weights is True. If not given, the class's default is used: cls._pretrained_relevant_module.

  • ignore : Optional[List[str]], optional (default = None)
    An optional list of regular expressions that define which weights to ignore from a pretrained state_dict. This is only relevant when load_weights is True. If not specified, the class's default is used: cls._pretrained_ignore.

  • allow_missing : Optional[List[str]], optional (default = None)
    An optional list of regular expressions that specifies which weights are allowed to be missing from the pretrained state dictionary. This is only relevant when load_weights is True. If not specified, the class's default is used: cls._pretrained_allow_missing.

  • strict : bool, optional (default = True)
    Whether to load the state_dict in "strict" model. This only applies when load_weights is True.

  • **kwargs : Any
    Key word arguments to pass to cls.from_config() when instantiating the module.