gevent.threadpool
- A pool of native threads¶ThreadPool
(maxsize, hub=None)[source]¶Note
The method apply_async()
will always return a new
greenlet, bypassing the threadpool entirely.
Caution
Instances of this class are only true if they have unfinished tasks.
apply
(func, args=None, kwds=None)¶Rough equivalent of the apply()
builtin function,
blocking until the result is ready and returning it.
The func
will usually, but not always, be run in a way
that allows the current greenlet to switch out (for example,
in a new greenlet or thread, depending on implementation). But
if the current greenlet or thread is already one that was
spawned by this pool, the pool may choose to immediately run
the func
synchronously.
Note
As implemented, attempting to use
Threadpool.appy()
from inside another function that
was itself spawned in a threadpool (any threadpool) will
cause the function to be run immediately.
Changed in version 1.1a2: Now raises any exception raised by func instead of dropping it.
apply
(func, args=None, kwds=None)¶Rough quivalent of the apply()
builtin function blocking until
the result is ready and returning it.
The func
will usually, but not always, be run in a way
that allows the current greenlet to switch out (for example,
in a new greenlet or thread, depending on implementation). But
if the current greenlet or thread is already one that was
spawned by this pool, the pool may choose to immediately run
the func
synchronously.
Any exception func
raises will be propagated to the caller of apply
(that is,
this method will raise the exception that func
raised).
apply_async
(func, args=None, kwds=None, callback=None)¶A variant of the apply()
method which returns a Greenlet
object.
When the returned greenlet gets to run, it will call apply()
,
passing in func, args and kwds.
If callback is specified, then it should be a callable which accepts a single argument. When the result becomes ready callback is applied to it (unless the call failed).
This method will never block, even if this group is full (that is,
even if spawn()
would block, this method will not).
Caution
The returned greenlet may or may not be tracked
as part of this group, so joining
this group is
not a reliable way to wait for the results to be available or
for the returned greenlet to run; instead, join the returned
greenlet.
Tip
Because ThreadPool
objects do not track greenlets, the returned
greenlet will never be a part of it. To reduce overhead and improve performance,
Group
and Pool
may choose to track the returned
greenlet. These are implementation details that may change.
apply_cb
(func, args=None, kwds=None, callback=None)¶apply()
the given func(*args, **kwds), and, if a callback is given, run it with the
results of func (unless an exception was raised.)
The callback may be called synchronously or asynchronously. If called
asynchronously, it will not be tracked by this group. (Group
and Pool
call it asynchronously in a new greenlet; ThreadPool
calls
it synchronously in the current greenlet.)
apply_e
(expected_errors, function, args=None, kwargs=None)[source]¶Deprecated since version 1.1a2: Identical to apply()
; the expected_errors
argument is ignored.
imap
(func, *iterables, maxsize=None) → iterable¶An equivalent of itertools.imap()
, operating in parallel.
The func is applied to each element yielded from each
iterable in iterables in turn, collecting the result.
If this object has a bound on the number of active greenlets it can
contain (such as Pool
), then at most that number of tasks will operate
in parallel.
maxsize (int) –
If given and not-None, specifies the maximum number of finished results that will be allowed to accumulate awaiting the reader; more than that number of results will cause map function greenlets to begin to block. This is most useful if there is a great disparity in the speed of the mapping code and the consumer and the results consume a great deal of resources.
Note
This is separate from any bound on the number of active parallel tasks, though they may have some interaction (for example, limiting the number of parallel tasks to the smallest bound).
Note
Using a bound is slightly more computationally expensive than not using a bound.
Tip
The imap_unordered()
method makes much better
use of this parameter. Some additional, unspecified,
number of objects may be required to be kept in memory
to maintain order by this function.
An iterable object.
Changed in version 1.1b3: Added the maxsize keyword parameter.
Changed in version 1.1a1: Accept multiple iterables to iterate in parallel.
imap_unordered
(func, *iterables, maxsize=None) → iterable¶The same as imap()
except that the ordering of the results
from the returned iterator should be considered in arbitrary
order.
This is lighter weight than imap()
and should be preferred if order
doesn’t matter.
See also
imap()
for more details.
map
(func, iterable)¶Return a list made by applying the func to each element of the iterable.
See also
map_async
(func, iterable, callback=None)¶A variant of the map() method which returns a Greenlet object that is executing the map function.
If callback is specified then it should be a callable which accepts a single argument.
ThreadPoolExecutor
(max_workers)[source]¶A version of concurrent.futures.ThreadPoolExecutor
that
always uses native threads, even when threading is monkey-patched.
The Future
objects returned from this object can be used
with gevent waiting primitives like gevent.wait()
.
Caution
If threading is not monkey-patched, then the Future
objects returned by this object are not guaranteed to work with
as_completed()
and wait()
.
The individual blocking methods like result()
and exception()
will always work.
New in version 1.2a1: This is a provisional API.
Initializes a new ThreadPoolExecutor instance.
execute the given calls.
thread_name_prefix: An optional name prefix to give our threads. initializer: A callable used to initialize worker threads. initargs: A tuple of arguments to pass to the initializer.
Next page: gevent.time
– Makes sleep gevent aware