@@ -12,6 +12,7 @@ The following signals are implemented by Huey:
12
12
* ``SIGNAL_CANCELED ``: task was canceled due to a pre-execute hook raising
13
13
a :py:class: `CancelExecution ` exception.
14
14
* ``SIGNAL_COMPLETE ``: task has been executed successfully.
15
+ * ``SIGNAL_ENQUEUED ``: task has been enqueued (**see note **).
15
16
* ``SIGNAL_ERROR ``: task failed due to an unhandled exception.
16
17
* ``SIGNAL_EXECUTING ``: task is about to be executed.
17
18
* ``SIGNAL_EXPIRED ``: task expired.
@@ -33,6 +34,14 @@ The following signals will include additional arguments:
33
34
* ``SIGNAL_ERROR ``: includes a third argument ``exc ``, which is the
34
35
``Exception `` that was raised while executing the task.
35
36
37
+ .. note ::
38
+ Signals are run within the context of the consumer **except ** that the
39
+ ``SIGNAL_ENQUEUED `` signal will also run within the context of your
40
+ application code (since your application code will typically enqueue
41
+ tasks). Recall that signal handlers are run sequentially and synchronously,
42
+ so be careful about introducing overhead in them -- particularly when they
43
+ may be run by the application process.
44
+
36
45
To register a signal handler, use the :py:meth: `Huey.signal ` method:
37
46
38
47
.. code-block :: python
@@ -88,8 +97,10 @@ Here is a simple example of a task execution we would expect to succeed:
88
97
>>> result = add(1, 2)
89
98
>>> result.get(blocking=True)
90
99
91
- The consumer would send the following signals :
100
+ The following signals would be fired :
92
101
102
+ * ``SIGNAL_ENQUEUED `` - the task has been enqueued (happens in the application
103
+ process).
93
104
* ``SIGNAL_EXECUTING `` - the task has been dequeued and will be executed.
94
105
* ``SIGNAL_COMPLETE `` - the task has finished successfully.
95
106
@@ -102,10 +113,13 @@ Here is an example of scheduling a task for execution after a short delay:
102
113
103
114
The following signals would be sent:
104
115
116
+ * ``SIGNAL_ENQUEUED `` - the task has been enqueued (happens in the **application **
117
+ process).
105
118
* ``SIGNAL_SCHEDULED `` - the task is not yet ready to run, so it has been added
106
119
to the schedule.
107
- * After 10 seconds, the consumer will run the task and send
108
- the ``SIGNAL_EXECUTING `` signal.
120
+ * After 10 seconds, the consumer will re-enqueue the task as it is now ready to
121
+ run, sending the ``SIGNAL_ENQUEUED `` (in the **consumer ** process!).
122
+ * Then the consumer will run the task and send the ``SIGNAL_EXECUTING `` signal.
109
123
* ``SIGNAL_COMPLETE ``.
110
124
111
125
Here is an example that may fail, in which case it will be retried
@@ -124,11 +138,14 @@ automatically with a delay of 10 seconds.
124
138
Assuming the task failed the first time and succeeded the second time, we would
125
139
see the following signals being sent:
126
140
141
+ * ``SIGNAL_ENQUEUED `` - task has been enqueued.
127
142
* ``SIGNAL_EXECUTING `` - the task is being executed.
128
143
* ``SIGNAL_ERROR `` - the task raised an unhandled exception.
129
144
* ``SIGNAL_RETRYING `` - the task will be retried.
130
145
* ``SIGNAL_SCHEDULED `` - the task has been added to the schedule for execution
131
146
in ~10 seconds.
147
+ * ``SIGNAL_ENQUEUED `` - 10s have elapsed and the task is ready to run and has
148
+ been re-enqueued.
132
149
* ``SIGNAL_EXECUTING `` - second try running task.
133
150
* ``SIGNAL_COMPLETE `` - task succeeded.
134
151
@@ -141,6 +158,7 @@ What happens if we revoke the ``add()`` task and then attempt to execute it:
141
158
142
159
The following signal will be sent:
143
160
161
+ * ``SIGNAL_ENQUEUED `` - the task has been enqueued for execution.
144
162
* ``SIGNAL_REVOKED `` - this is sent before the task enters the "executing"
145
163
state. When a task is revoked, no other signals will be sent.
146
164
@@ -169,8 +187,9 @@ Performance considerations
169
187
--------------------------
170
188
171
189
Signal handlers are executed **synchronously ** by the consumer as it processes
172
- tasks. It is important to use care when implementing signal handlers, as one
173
- slow signal handler can impact the overall responsiveness of the consumer.
190
+ tasks (with the exception of ``SIGNAL_ENQUEUED ``). It is important to use care
191
+ when implementing signal handlers, as one slow signal handler can impact the
192
+ overall responsiveness of the consumer.
174
193
175
194
For example, if you implement a signal handler that posts some data to REST
176
195
API, everything might work fine until the REST API goes down or stops being
@@ -183,3 +202,9 @@ handles. Signal handlers are called by the consumer workers, which (depending
183
202
on how you are running the consumer) may be separate processes, threads or
184
203
greenlets. As a result, care should be taken to ensure proper initialization
185
204
and cleanup of any resources you plan to use in signal handlers.
205
+
206
+ Lastly, take care when implementing ``SIGNAL_ENQUEUED `` handlers, as these may
207
+ run in your application-code (e.g. whenever your application enqueues a task),
208
+ **or ** by the consumer process (e.g. when re-enqueueing a task for retry, or
209
+ when enqueueing periodic tasks, when moving a task from the schedule to the
210
+ queue, etc).
0 commit comments