How to run CELERY_BEAT_SCHEDULE tasks at different intervals per tenant?

[royroyee]

Is there a way to stagger CELERY_BEAT_SCHEDULE tasks so that they run at different times per tenant, instead of all tenants executing tasks simultaneously?

When tasks run at the same time across tenants, resource load increases significantly as the tenant count grows. Looking for a straightforward solution to vary task execution times per tenant without complex configurations or major structural changes.

Any guidance or examples on achieving this would be greatly appreciated

[maciej-gol]

Unfortunately, I'm not aware of any simple configuration for this :( I could look into this when I'm done implementing the DB scheduler for the tenants. What you are basically looking at is to add a jitter to every scheduler entry.

[aayush-kosh]

@maciej-gol How's the DB Scheduler for tenants coming along? When do you think we'll be able to merge it?

[maciej-gol]

@aayush-kosh do you have any suggestion on how to add jitter to a schedule? My best solution to this is on the user side - to add a random jitter/sleep to your tasks. Otherwise, it is completely not obvious how to offset a periodic task that uses a cron or interval schedule without being extremely confusing to users that the task has actually launched on a different time than specified.

[aayush-kosh]

Hi @maciej-gol

We managed to get this working. Let me know if you need help with something.

[maciej-gol]

Its ok, just wanted to hear if you have any troubles. Will be releasing the PR then! Thanks! śr., 29 sty 2025, 07:50 użytkownik aayush-kosh ***@***.***> napisał:

…

Hi @maciej-gol <https://github.com/maciej-gol> We managed to get this working. Let me know if you need help with something. — Reply to this email directly, view it on GitHub <#129 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAOGQTIOBAPSFCJXVNPKT5L2NB22RAVCNFSM6AAAAABRKQHUL6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMRQHA2DEMJYGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>