sequential.seq_distributor

Sequential distributor that runs one task at a time.

Attributes

logger

Classes

`LimitedSizeDict`	Dictionary for exit info.
`SequentialDistributor`	Executor to run tasks one at a time.
`SequentialWorkerNode`	Get Executor Info for a Task Instance.

Module Contents

sequential.seq_distributor.logger[source]

class sequential.seq_distributor.LimitedSizeDict(*args: int, **kwds: int)[source]

Bases: collections.OrderedDict

Dictionary for exit info.

size_limit[source]

__setitem__(key: Any, value: Any) → None[source]: Set item in dict.

_check_size_limit() → None[source]

class sequential.seq_distributor.SequentialDistributor(cluster_name: str, exit_info_queue_size: int = 1000, *args: tuple, **kwargs: dict)[source]

Bases: jobmon.core.cluster_protocol.ClusterDistributor

Executor to run tasks one at a time.

started = False[source]

_cluster_name[source]

_worker_node_entry_point = 'worker_node_entry_point'[source]

_next_distributor_id = 1[source]

_exit_info[source]

property worker_node_entry_point: str[source]: Path to jobmon worker_node_entry_point.

property cluster_name: str[source]: Return the name of the cluster type.

start() → None[source]: Start the distributor.

stop() → None[source]: Stop the distributor.

abstract get_queueing_errors(distributor_ids: List[str]) → Dict[str, str][source]: Get the task instances that have errored out.

abstract get_array_queueing_errors(distributor_id: int | str) → Dict[int | str, str][source]

get_remote_exit_info(distributor_id: str) → Tuple[str, str][source]: Get exit info from task instances that have run.

get_submitted_or_running(distributor_ids: List[str] | None = None) → Set[str][source]: Check status of running task.

terminate_task_instances(distributor_ids: List[str]) → None[source]

Terminate task instances.

If implemented, return a list of (task_instance_id, hostname) tuples for any task_instances that are terminated.

submit_to_batch_distributor(command: str, name: str, requested_resources: Dict[str, Any]) → str[source]: Execute sequentially.

class sequential.seq_distributor.SequentialWorkerNode[source]

Bases: jobmon.core.cluster_protocol.ClusterWorkerNode

Get Executor Info for a Task Instance.

_distributor_id: str | None = None[source]

_logfile_template[source]

property distributor_id: str | None[source]: Distributor id of the task.

initialize_logfile(log_type: str, log_dir: str, name: str) → str[source]: Error and exit code info from the executor.

static get_exit_info(exit_code: int, error_msg: str) → Tuple[str, str][source]: Exit info, error message.

static get_usage_stats() → Dict[source]: Usage information specific to the exector.