Skip to content





def init_process(
    process_rank: int,
    world_size: int,
    distributed_device_ids: List[int],
    func: Callable,
    func_args: Tuple = None,
    func_kwargs: Dict[str, Any] = None,
    primary_addr: str = "",
    primary_port: int = 29500


def run_distributed_test(
    device_ids: List[int] = None,
    func: Callable = None,
    *, ,

This runs the func in a simulated distributed environment.


  • device_ids : List[int]
    List of devices. There need to be at least 2 devices. Default is [-1, -1].

  • func : Callable
    func needs to be global for spawning the processes, so that it can be pickled.

  • start_method : Optional[str], optional (default = None)
    The start method to use for starting the workers. Defaults to "spawn" for GPU processes and fork otherwise.