src.executors package

Submodules

src.executors.base module

class src.executors.base.BaseExecutor(**kwargs)

Bases: ABC

A base class for a benchmark executor

abstract _execute(func, args, **kwargs)

Executor-specific implementation (see inherited classes)

execute(func, args, **kwargs)

Executes a given function over a list or dict of arguments, and passes arbitrary keyword arguments to the function. The particular meaning of “execution” is defined in _execute() method, and implemented in inherited executor classes.

Parameters:

• func (callable) – The function to be executed.

• args (list | dict) – A list (or dict) of input arguments for func.

• kwargs (dict, optional) – Arbitrary keyword arguments passed to func.

Returns:

results – Results of the execution in the same format as input arguments.

Return type:

list | dict

abstract get_n_workers()

wait_for_workers(nworkers)

src.executors.dask module

class src.executors.dask.DaskGatewayExecutor(**kwargs)

Bases: BaseExecutor

Dask Gateway executor

Searches for an existing Gateway cluster and uses it to parallelize execution over multiple nodes using a batch system defined in Dask Gateway’s backend (e.g. Slurm).

_execute(func, args, **kwargs)

Execute func over args in parallel using distributed.Client::submit().

get_n_workers()

class src.executors.dask.DaskLocalExecutor(**kwargs)

Bases: BaseExecutor

Dask executor with a local cluster

Creates a LocalCluster and uses it to parallelize execution over local CPU cores (same node as the benchmark)

_execute(func, args, **kwargs)

Execute func over args in parallel using distributed.Client::submit().

get_n_workers()

src.executors.futures module

class src.executors.futures.FuturesExecutor(**kwargs)

Bases: BaseExecutor

Futures executor

Uses concurrent.futures to parallelize execution over CPU cores on the same node where the benchmark is launched.

_execute(func, args, **kwargs)

Execute func over args in parallel using concurrent.futures.ThreadPoolExecutor.

get_n_workers()

src.executors.sequential module

class src.executors.sequential.SequentialExecutor(**kwargs)

Bases: BaseExecutor

Simple sequential executor

Processes arguments in a for loop.

_execute(func, args, **kwargs)

Execute func over args in a loop.

get_n_workers()

Module contents