Skip to content

Split jobs to multiple listeners #645

@Erethon

Description

@Erethon

At the moment we only have one listener worker that runs all the jobs. While this works, it's not ideal because certain long procedures (like evaluation) can take a long time and more import jobs (issue creation for example) can be stuck waiting for evals to finish.

Django-pgpubsub doesn't have a way to exclude certain channels, you can only specify channels to listen to with --channels otherwise it listens on all channels. We can either:
a) extend django-pgpubsub with the functionality to exclude channels
b) specify all channels for each listener manually, making sure we update the listeners when new channels are added or removed.

I think option (a) makes more sense long term.

Related to #409, but more generic.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions