Feature Description

Hyperswitch uses scheduler services to manage asynchronous workflows like payment sync, refund processing, dispute handling, and outgoing webhook retries. These tasks are stored and tracked in process_tracker table and processed by producer/consumer services via Redis streams.

Aim is to introduce a Playground environment for rapid testing and validation of new features before production rollout. To establish proper isolation for scheduler services, we need to segregate scheduler tasks between environments.

Possible Implementation

This will introduce a new column - application_source in the process_tracker table to identify where the tasks were originally created. Tasks created in Playground will be marked with application_source='playground' and processed by dedicated Playground consumer. This ensures complete isolation from production workflows and enables independent testing and deployment of scheduler changes.

Database Changes:

• Add application_source VARCHAR(32) column to process_tracker table
• Add index on (application_source, status, schedule_time) for query performance

Application Layer:

• Set application_source='playground' when Playground application creates tasks
• Set application_source='main' when Main application creates tasks

Producer (Single Deployment in Main Cluster):

• Fetch all tasks from process_tracker table
• Route tasks to different Redis streams based on application column:
  • Tasks with application='playground' → "playground_scheduler_stream"
  • Tasks with application='main' or NULL → "scheduler_stream"

Consumers (Separate Deployments):

• Main consumers read from "scheduler_stream"
• Playground consumers read from "playground_scheduler_stream"
• Both use same business logic, just different stream configurations

Have you spent some time checking if this feature request has been raised before?

• I checked and didn't find a similar issue

Have you read the Contributing Guidelines?

• I have read the Contributing Guidelines

Are you willing to submit a PR?

Yes, I am willing to submit a PR!