Removing repeatable jobs by data passed in queue.add

[yairk]

If I understand correctly, repeatable jobs are created via queue.add() by passing the repeat option as described e.g. here.

When calling getRepeatableJobs, however, all of the data that was passed to queue.add() is not returned. Instead, I only get the key, name, id, etc.

In my use case, I am creating a repeatable job while passing a type field in the job's data. If I then want to cancel all of the repeatable jobs of that type, how would I go about doing that? The only way I could so far come up with is to get all of the delayed jobs, filter the ones with the right type, and then retrieve all the repeatable jobs and "somehow" correlate their keys/ids with the delayed jobs.

Is there a better way to achieve this? If not, what's the safest way to correlate a delayed job with its repeatable job?

Bull version

1.14.8

[manast]

The ergonomics of repeatable jobs are not very nice atm, however can't you use the "name" field and correlate using that?

[yairk]

@manast thank you for the response! I could do that, yes - it's just that currently I'm using different name values for different jobs. I wanted to give all of them the same type (e.g. SystemScheduled) and then be able to cancel all repeatable jobs of type SystemScheduled.

Are there short-term plans for improving the ergonomics of repeatable jobs?

[croossin]

I am also having trouble working around repeatable jobs. I simply want to cancel particular repeatable jobs by some sort of known ID. It seems that the APIs make it quite challenging to do so. Why can't we simply call something like job.remove() and it knows to remove that particular job?

@yairk Were you able to curate some sort of solution?

[croossin]

When I call getRepeatableJobs this is the array that is returned:

[
  {
    key: '__default__::::* * * * *',
    name: '__default__',
    id: null,
    endDate: null,
    tz: null,
    cron: '* * * * *',
    every: null,
    next: 1623336240000
  }
]

It seems weird that none of the information is passed through? However, the job object passed into the process function holds a lot of valuable information:

Job {
  opts: {
    repeat: { count: 9, cron: '* * * * *' },
    jobId: 'repeat:06ff7e943c0b9a35a957a28ad1155a1b:1623279600000',
    delay: 59987,
    timestamp: 1623279540013,
    prevMillis: 1623279600000,
    removeOnComplete: 100,
    attempts: 1,
    backoff: undefined
  },
...

Why can we not simply remove by said jobId in the opts? cc @manast

[manast]

@croossin because the jobId is for one iteration of the repeatable job. A repeatable job is some metadata that is used to determine when the next job should be added to the queue.

[yairk]

I am also having trouble working around repeatable jobs. I simply want to cancel particular repeatable jobs by some sort of known ID. It seems that the APIs make it quite challenging to do so. Why can't we simply call something like job.remove() and it knows to remove that particular job?

@yairk Were you able to curate some sort of solution?

@croossin unfortunately I was unable to come up with a decent solution other than correlation based on name as suggested by @manast.

[manast]

I made a small change where the repeat key is also stored in the Job returned by queue##add (or when getting any repeatable job for that matter), I know it is not a great solution but maybe it makes it easier for you: #2106

[croossin]

@manast Great - Is this an example of how to remove a repeatable job by said key? If not, do you mind showing us an example?

[manast]

Yes, that the example, you just use job.opts.repeat.key to remove the repeatable job.

[annieTechno]

But there is a problem. When removing a job with key one last delayed job is still not removed, issue here