evaluate subcommand can be used to
evaluate a trained model against a dataset
and report any metrics calculated by the model.
$ allennlp evaluate --help usage: allennlp evaluate [-h] [--output-file OUTPUT_FILE] [--weights-file WEIGHTS_FILE] [--cuda-device CUDA_DEVICE] [-o OVERRIDES] [--batch-size BATCH_SIZE] [--batch-weight-key BATCH_WEIGHT_KEY] [--extend-vocab] [--embedding-sources-mapping EMBEDDING_SOURCES_MAPPING] [--include-package INCLUDE_PACKAGE] archive_file input_file Evaluate the specified model + dataset positional arguments: archive_file path to an archived trained model input_file path to the file containing the evaluation data optional arguments: -h, --help show this help message and exit --output-file OUTPUT_FILE path to output file --weights-file WEIGHTS_FILE a path that overrides which weights file to use --cuda-device CUDA_DEVICE id of GPU to use (if any) -o OVERRIDES, --overrides OVERRIDES a JSON structure used to override the experiment configuration --batch-size BATCH_SIZE If non-empty, the batch size to use during evaluation. --batch-weight-key BATCH_WEIGHT_KEY If non-empty, name of metric used to weight the loss on a per-batch basis. --extend-vocab if specified, we will use the instances in your new dataset to extend your vocabulary. If pretrained-file was used to initialize embedding layers, you may also need to pass --embedding-sources-mapping. --embedding-sources-mapping EMBEDDING_SOURCES_MAPPING a JSON dict defining mapping from embedding module path to embedding pretrained-file used during training. If not passed, and embedding needs to be extended, we will try to use the original file paths used during training. If they are not available we will use random vectors for embedding extension. --include-package INCLUDE_PACKAGE additional packages to include