Feat/graceful shutdown handler

[Muaysh02]

Here is a comprehensive summary of what was implemented:

1. The Core Issue

In production (process.env.NODE_ENV !== "test"), the application registered duplicate signal handlers for SIGTERM and SIGINT inside src/index.ts.

While the system initialized the GracefulShutdownManager and called installShutdownHandlers(), it immediately followed with an inline shutdown handler registered via process.on(...). This overriding handler immediately called server.close(() => process.exit(0)) upon receiving a signal. As a result, the process would exit as soon as the HTTP server closed, bypassing the remaining phases of the GracefulShutdownManager sequence (which was responsible for closing database pools, stopping WebSockets, pausing background schedulers, stopping the outbox publisher, and disconnecting from Redis).

2. The Solution (Code Changes)

• Signal Handler Cleanup: Removed the local shutdown function and its duplicate process.on('SIGTERM') and process.on('SIGINT') registrations in src/index.ts. This leaves installShutdownHandlers() as the single production registration point.
• Unified Shutdown Sequence: Signals are now routed exclusively to GracefulShutdownManager.shutdown(). This executes the following clean, ordered sequence before exiting:
  1. server_close: Express HTTP server stops accepting new connections, allowing in-flight HTTP requests to complete.
  2. ws_drain: Open WebSocket clients are closed gracefully (code 1000) with a 5-second hard termination fallback.
  3. listener_stop: Event consumers are paused at their current offsets.
  4. scheduler_drain: Schedulers are stopped and current running jobs are allowed to finish.
  5. outbox_stop: Outbox publisher is safely shut down.
  6. invalidation_bus_stop: Cross-replica cache invalidation listener connection is closed.
  7. pool_close: All three database connection pools (pool, workerPool, replicaPool) are cleanly closed via pool.end().
  8. redis_close: Redis client is cleanly disconnected via disconnect().
  9. Exit: The process exits with code 0 (or 1 if any phase throws an error or the grace period expires).

3. Unit Test Integration

We added a sequence verification test in src/tests/gracefulShutdown.test.ts to guarantee order execution:

• Mocks were introduced to track call execution order for server.close(), pool.end(), redis.disconnect(), and process.exit().
• The test asserts that the HTTP server close finishes first (draining in-flight requests) before database pool termination begins, and that process.exit(0) is only triggered after all resources are closed.

4. Documentation Updates

• README.md: Added a new ## Graceful Shutdown section documenting how the backend handles SIGTERM/SIGINT signals, and linking to the primary documentation.
• docs/graceful-shutdown.md: Updated the architecture guide with a new ## Signal Handling in Production section detailing the entrypoint signal hooks.
• pr-description.md: Updated with the PR description and a Closes #<this-issue> tag.

closes #562

[drips-wave]

@Muaysh02 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits