Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Job status cache for gbasf2 #174

Open
meliache opened this issue Oct 17, 2022 · 0 comments
Open

Job status cache for gbasf2 #174

meliache opened this issue Oct 17, 2022 · 0 comments
Labels
enhancement New feature or request gbasf2 Concerns the gbasf2/grid b2luigi wrapper good first issue Good for newcomers help wanted Extra attention is needed

Comments

@meliache
Copy link
Collaborator

meliache commented Oct 17, 2022

The get_job_status call in the Gbasf2Process is rather costly and the gbasf2/dirac servers can only handle limited requests at a time. Therefore it would be good to create a job status cache by implementing a sub-class of BatchJobStatusCache, similar as to what has been done for htcondor and LSF.

It's possible to check the status of multiple projects with a singlegb2_job_status call and I'm sure we can do this with out custom script as well. We can then fill the cache will the statuses of all jobs in all user projects. If we inherit from BatchJobStatusCache, then by default the cache gets re-filled every 20 seconds (the time-to-live, TTL), but for the grid I would suggest to set a longer TTL.

@meliache meliache added enhancement New feature or request help wanted Extra attention is needed good first issue Good for newcomers gbasf2 Concerns the gbasf2/grid b2luigi wrapper labels Oct 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request gbasf2 Concerns the gbasf2/grid b2luigi wrapper good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant