Skip to content

Switch to RQ for background jobs #53

@jace

Description

@jace

Imgee currently uses Celery for background jobs while the rest of HasGeek's code uses RQ. For the sake of maintainability, Imgee should switch to RQ as well.

Imgee currently uses blocking requests when an image is being processed in the background – an earlier iteration returned a placeholder thumbnail until the real one was available. The blocking approach can be implemented with RQ by wrapping RQ with a queue helper:

  1. For every request that needs a background job (thumbnail requested, but one is not in the database), first create a cache key that is deterministic – based on the image id, dimensions and transformations required ("scaled" is only one of many transformations possible)
  2. Consult Redis for whether there is a running RQ job id for this key
  3. If no job, queue the job and save the key to Redis with a TTL of 24 hours
  4. Poll RQ until the job returns a result (should be a redirect URL), sleeping 1 second between polls
  5. Remove Redis cache key and return result to the caller

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions