Job status cache for gbasf2 #174
Labels
enhancement
New feature or request
gbasf2
Concerns the gbasf2/grid b2luigi wrapper
good first issue
Good for newcomers
help wanted
Extra attention is needed
The
get_job_status
call in theGbasf2Process
is rather costly and the gbasf2/dirac servers can only handle limited requests at a time. Therefore it would be good to create a job status cache by implementing a sub-class ofBatchJobStatusCache
, similar as to what has been done for htcondor and LSF.It's possible to check the status of multiple projects with a single
gb2_job_status
call and I'm sure we can do this with out custom script as well. We can then fill the cache will the statuses of all jobs in all user projects. If we inherit fromBatchJobStatusCache
, then by default the cache gets re-filled every 20 seconds (the time-to-live, TTL), but for the grid I would suggest to set a longer TTL.The text was updated successfully, but these errors were encountered: