allennlp.common.testing

Utilities and helpers for writing tests.

class allennlp.common.testing.test_case.AllenNlpTestCase(methodName='runTest')

Bases: unittest.case.TestCase

A custom subclass of TestCase that disables some of the more verbose AllenNLP logging and that creates and destroys a temp directory as a test fixture.

FIXTURES_ROOT = PosixPath('/Users/michael/hack/allenai/allennlp/allennlp/tests/fixtures')

MODULE_ROOT = PosixPath('/Users/michael/hack/allenai/allennlp/allennlp')

PROJECT_ROOT = PosixPath('/Users/michael/hack/allenai/allennlp')

TESTS_ROOT = PosixPath('/Users/michael/hack/allenai/allennlp/allennlp/tests')

TOOLS_ROOT = PosixPath('/Users/michael/hack/allenai/allennlp/allennlp/tools')

setUp(self)

Hook method for setting up the test fixture before exercising it.

tearDown(self)

Hook method for deconstructing the test fixture after testing it.

class allennlp.common.testing.model_test_case.ModelTestCase(methodName='runTest')

Bases: allennlp.common.testing.test_case.AllenNlpTestCase

A subclass of AllenNlpTestCase with added methods for testing Model subclasses.

assert_fields_equal(self, field1, field2, name: str, tolerance: float = 1e-06) → None

static check_model_computes_gradients_correctly(model: allennlp.models.model.Model, model_batch: Dict[str, Union[Any, Dict[str, Any]]], params_to_ignore: Set[str] = None, disable_dropout: bool = True)

ensure_batch_predictions_are_consistent(self, keys_to_ignore: Iterable[str] = ())

Ensures that the model performs the same on a batch of instances as on individual instances. Ignores metrics matching the regexp .*loss.* and those specified explicitly.

Parameters

keys_to_ignoreIterable[str], optional (default=())

Names of metrics that should not be taken into account, e.g. “batch_weight”.

ensure_model_can_train_save_and_load(self, param_file: str, tolerance: float = 0.0001, cuda_device: int = -1, gradients_to_ignore: Set[str] = None, overrides: str = '', disable_dropout: bool = True)

Parameters

param_filestr

Path to a training configuration file that we will use to train the model for this test.

tolerancefloat, optional (default=1e-4)

When comparing model predictions between the originally-trained model and the model after saving and loading, we will use this tolerance value (passed as rtol to numpy.testing.assert_allclose).

cuda_deviceint, optional (default=-1)

The device to run the test on.

gradients_to_ignoreSet[str], optional (default=None)

This test runs a gradient check to make sure that we’re actually computing gradients for all of the parameters in the model. If you really want to ignore certain parameters when doing that check, you can pass their names here. This is not recommended unless you’re really sure you don’t need to have non-zero gradients for those parameters (e.g., some of the beam search / state machine models have infrequently-used parameters that are hard to force the model to use in a small test).

overridesstr, optional (default = “”)

A JSON string that we will use to override values in the input parameter file.

disable_dropoutbool, optional (default = True)

If True we will set all dropout to 0 before checking gradients. (Otherwise, with small datasets, you may get zero gradients because of unlucky dropout.)

set_up_model(self, param_file, dataset_file)