Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ end

group :development do
gem "ammeter"
gem "pg"
gem "benchmark-ips", "~> 2.15"
gem "concurrent-ruby-ext"
gem "connection_pool"
Expand Down
2 changes: 2 additions & 0 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,7 @@ GEM
parser (3.3.11.1)
ast (~> 2.4.1)
racc
pg (1.6.3)
pp (0.6.3)
prettyprint
prettyprint (0.2.0)
Expand Down Expand Up @@ -312,6 +313,7 @@ DEPENDENCIES
cucumber
database_cleaner-redis (~> 2.0)
debug
pg
puma
rack-test
rake (~> 13.4)
Expand Down
64 changes: 63 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -372,10 +372,11 @@ single successful recovery probe will resume traffic flow.

### Data Store

Stoplight officially supports three data stores:
Stoplight officially supports three data stores, plus one experimental adapter:
- In-memory data store
- Redis
- Valkey
- PostgreSQL (experimental)

By default, Stoplight uses an in-memory data store:

Expand Down Expand Up @@ -417,6 +418,66 @@ Stoplight.configure do |config|
end
```

#### PostgreSQL (experimental)

> **Experimental**: This adapter is new and may change in future releases. Production use is possible but not yet covered by the maintenance policy.

Stoplight can use [PostgreSQL][] as a persistent data store. This adapter requires no additional runtime dependencies beyond the `pg` gem and uses the application Ruby clock, so there is no clock-skew concern between Stoplight and the database.

```ruby
require "pg"

Stoplight.configure do |config|
config.data_store = Stoplight::DataStore::Postgres.new(PG.connect(ENV["DATABASE_URL"]))
end
```

For multi-threaded applications (e.g. Puma or Sidekiq), pass a `ConnectionPool<PG::Connection>` instead of a bare connection — this is strongly recommended for production:

```ruby
require "pg"
require "connection_pool"

pool = ConnectionPool.new(size: 5, timeout: 3) { PG.connect(ENV["DATABASE_URL"]) }

Stoplight.configure do |config|
config.data_store = Stoplight::DataStore::Postgres.new(pool)
end
```

The schema must be created before use. In a Rails application, run the provided generator:

```sh
bin/rails generate stoplight:postgres:install
bin/rails db:migrate
```

For non-Rails applications, the DDL is available as the `Stoplight::Infrastructure::Postgres::DataStore::Schema::SQL` constant — execute it against your database once during setup.

> **Warning — Rails schema format requirement:** The adapter installs pgSQL functions (via the migration above). Rails' default `schema_format = :ruby` (`db/schema.rb`) cannot represent database functions — only tables. If you keep the default format, `db:schema:load`, `db:prepare`, and any fresh-database setup that loads from `schema.rb` (common in CI and test environments) will create the `stoplight_*` tables but **not** the functions. The adapter will then raise `PG::UndefinedFunction` at runtime.
>
> To capture the functions, set the following in `config/application.rb`:
>
> ```ruby
> config.active_record.schema_format = :sql
> ```
>
> With `:sql` format, Rails dumps to `db/structure.sql` instead, which preserves pgSQL functions and makes `db:schema:load` / `db:prepare` fully reproduce the schema.
>
> **Prefer to keep the `:ruby` schema format?** Add the [`fx`](https://github.com/teoljungberg/fx) gem to your app (the function equivalent of `scenic` for views). With `fx` loaded, Rails' schema dumper emits the Stoplight functions into `db/schema.rb` as `create_function` statements, so `db:schema:load` / `db:prepare` reproduce them without switching to `:sql`. This requires no changes to Stoplight — `fx` discovers the functions by introspection. (This is the same options matrix `logidze` offers: `schema_format = :sql`, or `fx` + the default `:ruby`.)
>
> If you keep `:ruby` format **without** `fx`, the functions are only installed when you run `bin/rails db:migrate` — they are **not** replayed by `db:schema:load`.
>
> **Seeing `PG::UndefinedFunction: function stoplight_*` does not exist?** Your database was loaded from `schema.rb` without the functions. Switch to `schema_format = :sql`, add the `fx` gem, or run `bin/rails db:migrate` to install them.

As with the Redis adapter, if the PostgreSQL connection fails Stoplight automatically falls back to the in-memory data store and invokes the configured `error_notifier`.

**Known limitations (experimental):**

- **Thread safety / connections.** A bare `PG::Connection` is *not* thread-safe; in a multi-threaded server (e.g. Puma) pass a `ConnectionPool<PG::Connection>` so each thread checks out its own connection. A single shared bare connection will raise `another command is already in progress` under concurrency (which then trips the in-memory failover).
- **Failover suspends distributed guarantees.** While PostgreSQL is unavailable, the breaker falls back to a per-process in-memory store (same trade-off as the Redis adapter). During that window circuit state is not shared across processes, the single-recovery-prober guarantee does not hold, and a `stoplight_locks` row held when the outage began is released only by its TTL.
- **Sub-second window boundaries.** This adapter counts events with microsecond precision (`occurred_at >= window_start`), whereas the in-memory/Redis adapters bucket by whole seconds. At the exact window edge the same event stream can yield slightly different counts across backends, so a circuit configured with a threshold right at the boundary may decide differently on PostgreSQL.

#### DragonflyDB Support

Although Stoplight does not officially support [DragonflyDB], it can be used with it. For details, you may refer to the official [DragonflyDB documentation].
Expand Down Expand Up @@ -636,6 +697,7 @@ Fowler’s [CircuitBreaker][] article.
[complete list of contributors]: https://github.com/bolshakov/stoplight/graphs/contributors
[CircuitBreaker]: http://martinfowler.com/bliki/CircuitBreaker.html
[Redis]: https://redis.io/
[PostgreSQL]: https://www.postgresql.org/
[Git Flow wiki page]: https://github.com/bolshakov/stoplight/wiki/Git-Flow
[Valkey]: https://valkey.io/
[Ruby Maintenance Branches]: https://www.ruby-lang.org/en/downloads/branches/
Expand Down
4 changes: 4 additions & 0 deletions UPGRADING.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
## Unreleased
- Added an experimental PostgreSQL data store adapter (`Stoplight::DataStore::Postgres`). Create the schema with `bin/rails generate stoplight:postgres:install` then `bin/rails db:migrate` (Rails apps). For non-Rails apps, execute the DDL from `Stoplight::Infrastructure::Postgres::DataStore::Schema::SQL`. See the README "PostgreSQL (experimental)" section.
- **Rails users:** The PostgreSQL adapter relies on pgSQL functions that Rails' default Ruby schema dumper (`schema_format = :ruby`, `db/schema.rb`) cannot capture. Set `config.active_record.schema_format = :sql` in `config/application.rb` so that `db/structure.sql` is used instead — this ensures `db:schema:load`, `db:prepare`, and CI database setup all install the functions. Alternatively, keep the default `:ruby` format and add the [`fx`](https://github.com/teoljungberg/fx) gem — its schema dumper emits the functions into `db/schema.rb` as `create_function` statements (same options matrix as `logidze`). If you keep `:ruby` *without* `fx`, the functions are only present after running `bin/rails db:migrate`; loading from `schema.rb` alone will cause `PG::UndefinedFunction` errors at runtime.

## Stoplight 6.0
- Removed Light#with() method
- Removed Light's `#with_data_store`, `#with_cool_off_time`, `#with_threshold`, `#with_window_size`, `#with_notifiers`,
Expand Down
36 changes: 36 additions & 0 deletions lib/generators/stoplight/postgres/USAGE
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
Description:
Generates a database migration that creates the five tables required by the
Stoplight PostgreSQL data store and installs the nine pgSQL functions used
by the adapter for atomic circuit-breaker operations:

stoplight_events — rolling event log with composite PK
stoplight_metadata — per-light error/success counters
stoplight_recovery_metrics — metrics tracked during half-open recovery
stoplight_states — circuit breaker state and timestamps
stoplight_locks — distributed advisory locks

The pgSQL function definitions are embedded directly into the generated
migration at generation time, keeping migrations self-contained and
reproducible (no runtime SQL file reads required).

Examples:
rails generate stoplight:postgres:install

Creates db/migrate/<timestamp>_create_stoplight_tables.rb
The migration creates all five tables AND installs the nine pgSQL functions.

rails generate stoplight:postgres:install --update

Creates db/migrate/<timestamp>_update_stoplight_functions.rb
The migration only re-installs the pgSQL functions (CREATE OR REPLACE is
idempotent). Use this after upgrading the stoplight gem to pick up any
function changes without recreating the tables.

rails db:migrate

Applies whichever migration(s) are pending.

Note:
The Stoplight PostgreSQL data store NEVER issues DDL at runtime.
This generator is the supported way to create or update the schema in
Rails apps.
92 changes: 92 additions & 0 deletions lib/generators/stoplight/postgres/install_generator.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# frozen_string_literal: true

begin
require "rails/generators"
require "rails/generators/migration"
rescue LoadError
raise <<~WARN
Currently generators are only available for Rails applications
WARN
end

require "stoplight/infrastructure/postgres/data_store/schema"
require "stoplight/infrastructure/postgres/data_store/functions"

# steep:ignore:start
module Stoplight
module Generators
module Postgres
class InstallGenerator < ::Rails::Generators::Base # :nodoc:
include ::Rails::Generators::Migration

class_option :update, type: :boolean, default: false,
desc: "Generate a migration that only refreshes the pgSQL functions"

case (root = __dir__)
when String
source_root File.expand_path("templates", root)
else
raise "cannot find templates root"
end

# Required by Rails::Generators::Migration. Returns the next migration
# version number. When ActiveRecord is available (Rails app), delegates
# to AR's counter so timestamps stay monotonic; otherwise falls back to
# a plain UTC timestamp string.
def self.next_migration_number(dirname)
if defined?(::ActiveRecord::Generators::Base)
::ActiveRecord::Generators::Base.next_migration_number(dirname)
else
Time.now.utc.strftime("%Y%m%d%H%M%S")
end
end

def create_migration_file
if options[:update]
migration_template(
"update_stoplight_functions.rb.erb",
"db/migrate/update_stoplight_functions.rb"
)
else
migration_template(
"create_stoplight_tables.rb.erb",
"db/migrate/create_stoplight_tables.rb"
)
end
end

private

# Returns the ActiveRecord::Migration version string (e.g. "8.0") derived
# from the running Rails version, without requiring activerecord to be
# loaded. Used in the migration template as <%= migration_version %>.
def migration_version
"#{Rails::VERSION::MAJOR}.#{Rails::VERSION::MINOR}"
end

# Returns the concatenated SQL of all pgSQL function definitions.
# Embedded into the generated migration at generation time so migrations
# remain self-contained and reproducible (logidze-style).
#
# Thor reads ERB template files with File.binread, so the ERB output buffer
# is ASCII-8BIT. Any non-ASCII characters in the interpolated SQL (e.g.,
# em-dashes in comments) cause an Encoding::CompatibilityError. We encode
# to ASCII, replacing non-ASCII code points with their closest ASCII
# equivalent. SQL syntax is always ASCII; only comments may contain Unicode.
def stoplight_functions_sql
Stoplight::Infrastructure::Postgres::DataStore::Functions.sql
.encode("ASCII", invalid: :replace, undef: :replace, replace: "-")
end

# Returns the canonical tables DDL (text / timestamptz columns). Embedded
# into the migration so the generated tables always match the column types
# the pgSQL functions expect. See stoplight_functions_sql for the ASCII note.
def stoplight_schema_sql
Stoplight::Infrastructure::Postgres::DataStore::Schema::SQL
.encode("ASCII", invalid: :replace, undef: :replace, replace: "-")
end
end
end
end
end
# steep:ignore:end
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# frozen_string_literal: true

class CreateStoplightTables < ActiveRecord::Migration[<%= migration_version %>]
def up
# Tables and pgSQL functions are embedded from the gem's canonical DDL at
# generation time, so the migration is self-contained and the column types
# (text / timestamptz) always match what the functions expect.
safety_assured do
execute <<~SQL
<%= stoplight_schema_sql.gsub("\n", "\n ").rstrip %>
SQL

execute <<~SQL
<%= stoplight_functions_sql.gsub("\n", "\n ").rstrip %>
SQL
end
end

def down
safety_assured do
execute <<~SQL
DROP FUNCTION IF EXISTS stoplight_get_metrics CASCADE;
DROP FUNCTION IF EXISTS stoplight_record_failure CASCADE;
DROP FUNCTION IF EXISTS stoplight_record_recovery_probe_failure CASCADE;
DROP FUNCTION IF EXISTS stoplight_record_recovery_probe_success CASCADE;
DROP FUNCTION IF EXISTS stoplight_record_success CASCADE;
DROP FUNCTION IF EXISTS stoplight_release_lock CASCADE;
DROP FUNCTION IF EXISTS stoplight_transition_to_green CASCADE;
DROP FUNCTION IF EXISTS stoplight_transition_to_red CASCADE;
DROP FUNCTION IF EXISTS stoplight_transition_to_yellow CASCADE;
SQL

drop_table :stoplight_locks
drop_table :stoplight_states
drop_table :stoplight_recovery_metrics
drop_table :stoplight_metadata
drop_table :stoplight_events
end
end
end
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# frozen_string_literal: true

class UpdateStoplightFunctions < ActiveRecord::Migration[<%= migration_version %>]
def up
# Re-installs all pgSQL functions used by the Stoplight PostgreSQL adapter.
# CREATE OR REPLACE is idempotent — safe to run multiple times.
# Embedded at generation time so this migration is self-contained.
safety_assured do
execute <<~SQL
<%= stoplight_functions_sql.gsub("\n", "\n ").rstrip %>
SQL
end
end

def down
raise ActiveRecord::IrreversibleMigration,
"CREATE OR REPLACE FUNCTION has no clean inverse. " \
"To remove the functions, drop them manually or roll back to a prior install migration."
end
end
18 changes: 18 additions & 0 deletions lib/stoplight/data_store.rb
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,24 @@ def initialize(redis, warn_on_clock_skew: true)
end
end

class Postgres < Base
# @!attribute connection
# @return [::PG::Connection, ConnectionPool<::PG::Connection>]
attr_reader :connection

# @!attribute warn_on_clock_skew
# @return [Boolean]
attr_reader :warn_on_clock_skew

# @param connection [::PG::Connection, ConnectionPool<::PG::Connection>]
# @param warn_on_clock_skew [Boolean] (true) accepted for interface parity with Redis;
# the Postgres adapter uses the database clock, so cross-node skew does not apply.
def initialize(connection, warn_on_clock_skew: true)
@warn_on_clock_skew = warn_on_clock_skew
@connection = connection
end
end

class Memory < Base
end
end
Expand Down
2 changes: 1 addition & 1 deletion lib/stoplight/infrastructure/fail_safe/data_store.rb
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ def acquire_recovery_lock(config)
#
def release_recovery_lock(recovery_lock_token)
case recovery_lock_token
in Redis::DataStore::RecoveryLockToken
in Redis::DataStore::RecoveryLockToken | Postgres::DataStore::RecoveryLockToken
fallback = proc do |error|
error_notifier.call(error) if error
end
Expand Down
Loading