Jobs retention

[thoven87]

• Adding job retention configuration
• Make job priority static vars instead of static func

[adam-fowler]

A couple of comments

[adam-fowler]

Maybe this could be

enum RetainData {
    case retain(for: Duration)
    case doNotRetain
}

Anything with doNotRetain gets deleted immediately

[adam-fowler]

Sorry I'm wondering if we do this another way. Basically the retention policy is retain or not, without a timeout. Then we add a function for users to call.

func purgeRetainedJobsEarlierThan(cancelled: Date, finished: Date, failed: Date) {

Actually you could just update the cleanup function.

Then users could add their own scheduled job. We could even write the job for them

struct PurgeOldJobs: JobParameters {
    let jobQueue: JobQueue<PostgresJobQueue>
}

jobQueue.registerJob(parameters: PurgeOldJobs.self) { parameters,_ in
    parameters.jobQueue.cleanUp(
        cancelledJobs: .remove(olderThan: .now - 4*24*60*60),
        finishedJobs: .remove(olderThan: .now - 7*24*60*60)
    )
}

jobSchedule = JobSchedule(job: PurgeOldJobs.self, .onMinutes(0,10,20,30,40,50))

I know this ends up being more work for the user, but it makes it a lot more configurable

[thoven87]

Sorry I'm wondering if we do this another way. Basically the retention policy is retain or not, without a timeout. Then we add a function for users to call.

func purgeRetainedJobsEarlierThan(cancelled: Date, finished: Date, failed: Date) {

Actually you could just update the cleanup function.

Then users could add their own scheduled job. We could even write the job for them

struct PurgeOldJobs: JobParameters {
    let jobQueue: JobQueue<PostgresJobQueue>
}

jobQueue.registerJob(parameters: PurgeOldJobs.self) { parameters,_ in
    parameters.jobQueue.cleanUp(
        cancelledJobs: .remove(olderThan: .now - 4*24*60*60),
        finishedJobs: .remove(olderThan: .now - 7*24*60*60)
    )
}

jobSchedule = JobSchedule(job: PurgeOldJobs.self, .onMinutes(0,10,20,30,40,50))

I know this ends up being more work for the user, but it makes it a lot more configurable

I think this works better and also maybe introduce a job rescue (orphan jobs :))?