Thread workers¶
Workers run a long task in another thread, so that the GUI stays responsive,
and report back with signals (such as started, returned, errored,
finished, and, for generators, yielded), which can safely update the GUI
from the main thread. Except for
new_worker_qthread, they use Qt's
QRunnable and global
QThreadPool.
The simplest way to use them is the
@thread_worker decorator. Calling a decorated
function returns a worker, which runs the function when it's started (or at
once, if callbacks are given with connect=):
import time
from superqt.utils import thread_worker
@thread_worker
def long_task(n):
time.sleep(n)
return n
worker = long_task(2)
worker.returned.connect(print)
worker.start()
If the function is a generator, the worker is a
GeneratorWorker: it emits yielded for each
value, and can be paused, resumed, and aborted between them. Otherwise, it's a
FunctionWorker.
These tools were written for napari, and moved to superqt. The napari guide to threading describes them in more depth.
WorkerBase
¶
WorkerBase(
func: Callable[_P, _R] | None = None,
SignalsClass: type[
WorkerBaseSignals
] = WorkerBaseSignals,
)
Base class for creating a Worker that can run in another thread.
Parameters:
-
SignalsClass(type, default:WorkerBaseSignals) –A QObject subclass that contains signals, by default WorkerBaseSignals
Attributes:
-
signals(WorkerBaseSignals) –signal emitter object. To allow identify which worker thread emitted signal.
quit
¶
Send a request to abort the worker.
Note
It is entirely up to subclasses to honor this method by checking
self.abort_requested periodically in their worker.work
method, and exiting if True.
run
¶
Start the worker.
The end-user should never need to call this function. But it cannot be made private or renamed, since it is called by Qt.
The order of method calls when starting a worker is:
calls QThreadPool.globalInstance().start(worker)
| triggered by the QThreadPool.start() method
| | called by worker.run
| | |
V V V
worker.start -> worker.run -> worker.work
This is the function that actually gets called when calling
QThreadPool.start(worker). It simply wraps the work()
method, and emits a few signals. Subclasses should NOT override this
method (except with good reason), and instead should implement
work().
work
¶
work() -> Exception | _R
Main method to execute the worker.
The end-user should never need to call this function.
But subclasses must implement this method (See
GeneratorFunction.work for
an example implementation). Minimally, it should check self.abort_requested
periodically and exit if True.
Examples:
start
¶
await_workers
¶
await_workers(msecs: int | None = None) -> None
Ask all workers to quit, and wait up to msec for quit.
Attempts to clean up all running workers by calling worker.quit()
method. Any workers in the WorkerBase._worker_set set will have this
method.
By default, this function will block indefinitely, until worker threads
finish. If a timeout is provided, a RuntimeError will be raised if
the workers do not gracefully exit in the time requests, but the threads
will NOT be killed. It is (currently) left to the user to use their OS
to force-quit rogue threads.
Important
If the user does not put any yields in their function, and the function is super long, it will just hang... For instance, there's no graceful way to kill this thread in python:
This is why it's always advisable to use a generator that periodically yields for long-running computations in another thread.
See this stack-overflow post for a good discussion on the difficulty of killing a rogue python thread:
Parameters:
-
msecs(int, default:None) –Waits up to msecs milliseconds for all threads to exit and removes all threads from the thread pool. If msecs is
None(the default), the timeout is ignored (waits for the last thread to exit).
Raises:
-
RuntimeError–If a timeout is provided and workers do not quit successfully within the time allotted.
FunctionWorker
¶
FunctionWorker(func: Callable[_P, _R], *args, **kwargs)
QRunnable with signals that wraps a simple long-running function.
Note
FunctionWorker does not provide a way to stop a very long-running
function (e.g. time.sleep(10000)). So whenever possible, it is better to
implement your long running function as a generator that yields periodically,
and use the GeneratorWorker instead.
Parameters:
-
func(Callable) –A function to call in another thread
-
*args–will be passed to the function
-
**kwargs–will be passed to the function
Raises:
-
TypeError–If
funcis a generator function and not a regular function.
GeneratorWorker
¶
GeneratorWorker(
func: Callable[_P, Generator[_Y, _S | None, _R]],
*args,
SignalsClass: type[
WorkerBaseSignals
] = GeneratorWorkerSignals,
**kwargs,
)
QRunnable with signals that wraps a long-running generator.
Provides a convenient way to run a generator function in another thread, while allowing 2-way communication between threads, using plain-python generator syntax in the original function.
Parameters:
-
func(callable) –The function being run in another thread. May be a generator function.
-
SignalsClass(type, default:GeneratorWorkerSignals) –A QObject subclass that contains signals, by default GeneratorWorkerSignals
-
*args–Will be passed to func on instantiation
-
**kwargs–Will be passed to func on instantiation
work
¶
work() -> _R | Exception | None
Core event loop that calls the original function.
Enters a continual loop, yielding and returning from the original function. Checks for various events (quit, pause, resume, etc...). (To clarify: we are creating a rudimentary event loop here because there IS NO Qt event loop running in the other thread to hook into)
Convenience functions¶
thread_worker
¶
thread_worker(
function: Callable[_P, Generator[_Y, _S, _R]],
start_thread: bool | None = None,
connect: dict[str, Callable | Sequence[Callable]]
| None = None,
worker_class: type[WorkerBase] | None = None,
ignore_errors: bool = False,
) -> Callable[_P, GeneratorWorker[_Y, _S, _R]]
thread_worker(
function: Callable[_P, _R],
start_thread: bool | None = None,
connect: dict[str, Callable | Sequence[Callable]]
| None = None,
worker_class: type[WorkerBase] | None = None,
ignore_errors: bool = False,
) -> Callable[_P, FunctionWorker[_R]]
thread_worker(
function: Literal[None] = None,
start_thread: bool | None = None,
connect: dict[str, Callable | Sequence[Callable]]
| None = None,
worker_class: type[WorkerBase] | None = None,
ignore_errors: bool = False,
) -> Callable[
[Callable],
Callable[_P, FunctionWorker | GeneratorWorker],
]
thread_worker(
function: Callable | None = None,
start_thread: bool | None = None,
connect: dict[str, Callable | Sequence[Callable]]
| None = None,
worker_class: type[WorkerBase] | None = None,
ignore_errors: bool = False,
)
Decorator that runs a function in a separate thread when called.
When called, the decorated function returns a
WorkerBase. See
create_worker for additional keyword arguments that
can be used
when calling the function.
The returned worker will have these signals:
- started: emitted when the work is started
- finished: emitted when the work is finished
- returned: emitted with return value
- errored: emitted with error object on Exception
It will also have a worker.start() method that can be used to start
execution of the function in another thread. (useful if you need to connect
callbacks to signals prior to execution)
If the decorated function is a generator, the returned worker will also provide these signals:
- yielded: emitted with yielded values
- paused: emitted when a running job has successfully paused
- resumed: emitted when a paused job has successfully resumed
- aborted: emitted when a running job is successfully aborted
And these methods:
- quit: ask the thread to quit
- toggle_paused: toggle the running state of the thread.
- send: send a value into the generator. (This requires that your
decorator function uses the
value = yieldsyntax)
Parameters:
-
function(callable, default:None) –Function to call in another thread. For communication between threads may be a generator function.
-
start_thread(bool, default:None) –Whether to immediaetly start the thread. If False, the returned worker must be manually started with
worker.start(). by default it will beFalseif the_connectargument isNone, otherwiseTrue. -
connect(Dict[str, Union[Callable, Sequence]], default:None) –A mapping of
"signal_name"->callableor list ofcallable: callback functions to connect to the various signals offered by the worker class. by default None -
worker_class(Type[WorkerBase], default:None) –The
WorkerBaseto instantiate, by defaultFunctionWorkerwill be used iffuncis a regular function, andGeneratorWorkerwill be used if it is a generator. -
ignore_errors(bool, default:False) –If
False(the default), errors raised in the other thread will be reraised in the main thread (makes debugging significantly easier).
Returns:
-
callable–function that creates a worker, puts it in a new thread and returns the worker instance.
Examples:
@thread_worker
def long_function(start, end):
# do work, periodically yielding
i = start
while i <= end:
time.sleep(0.1)
yield i
# do teardown
return "anything"
# call the function to start running in another thread.
worker = long_function()
# connect signals here if desired... or they may be added using the
# `connect` argument in the `@thread_worker` decorator... in which
# case the worker will start immediately when long_function() is called
worker.start()
create_worker
¶
create_worker(
func: Callable[_P, Generator[_Y, _S, _R]],
*args,
_start_thread: bool | None = None,
_connect: dict[str, Callable | Sequence[Callable]]
| None = None,
_worker_class: type[GeneratorWorker]
| type[FunctionWorker]
| None = None,
_ignore_errors: bool = False,
**kwargs,
) -> GeneratorWorker[_Y, _S, _R]
create_worker(
func: Callable[_P, _R],
*args,
_start_thread: bool | None = None,
_connect: dict[str, Callable | Sequence[Callable]]
| None = None,
_worker_class: type[GeneratorWorker]
| type[FunctionWorker]
| None = None,
_ignore_errors: bool = False,
**kwargs,
) -> FunctionWorker[_R]
create_worker(
func: Callable,
*args,
_start_thread: bool | None = None,
_connect: dict[str, Callable | Sequence[Callable]]
| None = None,
_worker_class: type[GeneratorWorker]
| type[FunctionWorker]
| None = None,
_ignore_errors: bool = False,
**kwargs,
) -> FunctionWorker | GeneratorWorker
Convenience function to start a function in another thread.
By default, uses FunctionWorker for functions and GeneratorWorker for
generators, but a custom WorkerBase subclass may be provided. If so, it must be a
subclass of WorkerBase, which defines a standard set of signals and a run method.
Parameters:
-
func(Callable) –The function to call in another thread.
-
_start_thread(bool, default:None) –Whether to immediaetly start the thread. If False, the returned worker must be manually started with
worker.start(). by default it will beFalseif the_connectargument isNone, otherwiseTrue. -
_connect(Dict[str, Union[Callable, Sequence]], default:None) –A mapping of
"signal_name"->callableor list ofcallable: callback functions to connect to the various signals offered by the worker class. by defaultNone -
_worker_class(type of `GeneratorWorker` or `FunctionWorker`, default:None) –The
WorkerBaseto instantiate, by defaultFunctionWorkerwill be used iffuncis a regular function, andGeneratorWorkerwill be used if it is a generator. -
_ignore_errors(bool, default:False) –If
False(the default), errors raised in the other thread will be reraised in the main thread (makes debugging significantly easier). -
*args–will be passed to
func -
**kwargs–will be passed to
func
Returns:
-
worker(WorkerBase) –An instantiated worker. If
_start_threadwasFalse, the worker will have a.start()method that can be used to start the thread.
Raises:
-
TypeError–If a worker_class is provided that is not a subclass of WorkerBase.
-
TypeError–If _connect is provided and is not a dict of
{str: callable}
Examples:
new_worker_qthread
¶
new_worker_qthread(
Worker: type[WorkerProtocol],
*args,
_start_thread: bool = False,
_connect: dict[str, Callable] | None = None,
**kwargs,
)
Convenience function to start a worker in a QThread.
thread, not as the actual code or object that runs in that thread. The QThread object is created on the main thread and lives there.
Worker objects which derive from QObject are the things that actually do the work. They can be moved to a QThread as is done here.
Mostly ignorable detail
While the signals/slots syntax of the worker looks very similar to standard "single-threaded" signals & slots, note that inter-thread signals and slots (automatically) use an event-based QueuedConnection, while intra-thread signals use a DirectConnection. See Signals and Slots Across Threads
Parameters:
-
Worker(QObject) –QObject type that implements a
work()method. The Worker should also emit a finished signal when the work is done. -
_start_thread(bool, default:False) –If True, thread will be started immediately, otherwise, thread must be manually started with thread.start().
-
_connect(dict, default:None) –Optional dictionary of {signal: function} to connect to the new worker. for instance: _connect = {'incremented': myfunc} will result in: worker.incremented.connect(myfunc)
-
*args–will be passed to the Worker class on instantiation.
-
**kwargs–will be passed to the Worker class on instantiation.
Returns:
-
worker(WorkerBase) –The created worker.
-
thread(QThread) –The thread on which the worker is running.
Examples:
Create some QObject that has a long-running work method:
class Worker(QObject):
finished = Signal()
increment = Signal(int)
def __init__(self, argument):
super().__init__()
self.argument = argument
@Slot()
def work(self):
# some long running task...
import time
for i in range(10):
time.sleep(1)
self.increment.emit(i)
self.finished.emit()
worker, thread = new_worker_qthread(
Worker,
"argument",
_start_thread=True,
_connect={"increment": print},
)