pyslurmutils.concurrent.rest.SlurmRestExecutor#

class pyslurmutils.concurrent.rest.SlurmRestExecutor(url='', user_name='', token='', api_version='', parameters=None, log_directory=None, std_split=False, request_options=None, pre_script=None, post_script=None, python_cmd=None, initializer=None, initargs=None, initkwargs=None, data_directory=None, max_workers=None, max_tasks_per_worker=1, lazy_scheduling=True, conservative_scheduling=False, cleanup_job_artifacts=True)[source]#

Bases: RemoteExecutor

Parameters:

  • url (str)

  • user_name (str)

  • token (str)

  • api_version (str)

  • parameters (Optional[dict])

  • log_directory (Optional[str])

  • std_split (Optional[bool])

  • request_options (Optional[dict])

  • pre_script (Optional[str])

  • post_script (Optional[str])

  • python_cmd (Optional[str])

  • initializer (Optional[callable])

  • initargs (Optional[tuple])

  • initkwargs (Optional[tuple])

  • data_directory (Optional[str])

  • max_workers (Optional[int])

  • max_tasks_per_worker (Optional[int])

  • lazy_scheduling (bool)

  • conservative_scheduling (bool)

  • cleanup_job_artifacts (bool)

execute_context()[source]#
Return type:

Generator[Tuple[Callable[[callable, tuple, dict, Future], Any], Callable[[], str]], None, None]

map(fn, *iterables, timeout=None, chunksize=1)#

Returns an iterator equivalent to map(fn, iter).

Args:
fn: A callable that will take as many arguments as there are

passed iterables.

timeout: The maximum number of seconds to wait. If None, then there

is no limit on the wait time.

chunksize: The size of the chunks the iterable will be broken into

before being passed to a child process. This argument is only used by ProcessPoolExecutor; it is ignored by ThreadPoolExecutor.

Returns:

An iterator equivalent to: map(func, *iterables) but the calls may be evaluated out-of-order.

Raises:
TimeoutError: If the entire result iterator could not be generated

before the given timeout.

Exception: If fn(*args) raises for any values.

shutdown(wait=True, cancel=False)#
Parameters:

  • wait (bool) – wait for all workers to exit

  • cancel (bool) – cancel all pending tasks (running once cannot be cancelled)

submit(task, *args, **kwargs)#
Parameters:

  • task (Callable) – function to be executed remotely

  • args – positional arguments of the function to be executed remotely

  • kwargs – named arguments of the function to be executed remotely

Return type:

Future

Returns:

future object to retrieve the result