@DatasetReader.register("multitask") class MultiTaskDatasetReader(DatasetReader): | def __init__(self, readers: Dict[str, DatasetReader]) -> None
DatasetReader simply collects a dictionary of other
DatasetReaders. It is designed for
a different class (the
MultiTaskDataLoader) to actually read from each of the underlying
dataset readers, and so this really is just a glorified dictionary that we can construct as a
DatasetReader. We throw an error if you try to actually call
read(), because you should be
doing that differently.
Registered as a
DatasetReader with name "multitask".
- readers :
A mapping from dataset name to
DatasetReaderobjects for reading that dataset. You can use whatever names you want for the datasets, but they have to match the keys you use for data files and in other places in the
class MultiTaskDatasetReader(DatasetReader): | ... | def read( | self, | file_paths: Union[PathLike, str, Dict[str, Union[PathLike, str]]], | *, | force_task: Optional[str] = None | ) -> Union[Iterator[Instance], Dict[str, Iterator[Instance]]]