Ractor support (supersedes #365)

[rubys]

Supersedes #365, rebased onto main. The original commits' authors are credited as co-authors.

What this does

Declares the C extension Ractor-safe when SQLite is built thread-safe, so each Ractor can open and use its own connections. Passing a Database between Ractors still raises Ractor::Error (see RACTOR.md).

Changes beyond #365

• Statement status symbols filled at Init. setup_stmt_stat_symbols was lazy, and its "already filled?" check only looked at slot 0, so a second Ractor could see a partly filled table. It now runs once in init_sqlite3_statement.
• define_aggregator works outside the main Ractor. The proxy class kept the template and name in class instance variables, which raises Ractor::IsolationError in a non-main Ractor. They are now captured in closures (define_singleton_method). No API change.
• VERSION_INFO is deeply frozen, so any Ractor can read it.
• Dropped the FULLMUTEX "shareable database" part. SQLite's serialized mode makes the C calls safe, but a Database also holds Ruby state (registered functions, aggregators, collations, handlers) that can't be shared between Ractors. Marking it shareable would expose that state.
• Ractor tests run in a child process with a timeout. On Ruby 4.0, once a process has started any Ractor, ObjectSpace.each_object returns nothing, which broke test_aggregate_instances_are_released_after_each_query whenever the Ractor tests ran first. A child process isolates that, and turns a hang into a failure instead of a stalled job. This may account for the hangs seen on Initial Ractor support #365. I only tested on 4.0.5, so CI will tell for the other Rubies.

Testing

On Ruby 4.0.5, arm64-darwin:

• The full suite passes with three different seeds (360 runs). Linting is clean on the changed files.
• Reverting the define_aggregator or VERSION_INFO change makes the matching new test fail.
• A standalone stress script passes with 2, 8, 16 and 32 Ractors. Each Ractor uses its own in-memory connection with every callback type (trace, authorizer, function, aggregator, collation, statement timeout, statement stats), and all Ractors also make contended writes to one shared WAL file.

stress script

# Stress sqlite3 from multiple Ractors. Each Ractor owns its connections;
# nothing but strings and integers crosses a Ractor boundary.
#
#   ruby ractor_stress.rb [ractors=8] [iterations=200]

require "sqlite3"
require "tmpdir"

Warning[:experimental] = false

RACTORS = Integer(ARGV[0] || 8)
ITERS = Integer(ARGV[1] || 200)
dir = Dir.mktmpdir
shared = File.join(dir, "shared.db")

SQLite3::Database.new(shared) do |db|
  db.execute "PRAGMA journal_mode=WAL"
  db.execute "CREATE TABLE log (r INTEGER, i INTEGER, note TEXT)"
end

class Summer
  def initialize = @sum = 0
  def step(v) = @sum += v.to_i
  def finalize = @sum
end

class ReverseCollation
  def compare(a, b) = b <=> a
end

def workload(id, iters, shared)
  checks = 0
  iters.times do |i|
    # private in-memory database exercising every callback path
    SQLite3::Database.new(":memory:", results_as_hash: (i.odd?)) do |db|
      traced = 0
      db.trace { |_sql| traced += 1 }
      db.statement_timeout = 10_000
      db.authorizer = ->(*) { true }
      db.create_function("double", 1) { |fn, x| fn.result = x.to_i * 2 }
      db.define_aggregator("summer", Summer.new)
      db.collation("rev", ReverseCollation.new)

      db.execute "CREATE TABLE t (a INTEGER, s TEXT)"
      db.transaction do
        stmt = db.prepare("INSERT INTO t VALUES (?, ?)")
        50.times { |n| stmt.execute(n, "s#{n}") }
        stmt.close
      end

      sum = db.get_first_value("SELECT summer(double(a)) FROM t")
      raise "aggregate #{sum}" unless sum == 2450

      first = db.get_first_value("SELECT s FROM t ORDER BY s COLLATE rev LIMIT 1")
      raise "collation #{first}" unless first == "s9"

      stmt = db.prepare("SELECT * FROM t WHERE a > ?")
      rows = stmt.execute(10).to_a
      raise "rows #{rows.size}" unless rows.size == 39
      raise "stat" unless stmt.stat(:vm_steps).is_a?(Integer) && stmt.stat.key?(:sorts)
      stmt.close

      begin
        db.execute "SELECT * FROM nope"
        raise "expected SQLException"
      rescue SQLite3::SQLException
      end

      raise "trace" unless traced > 0
      checks += 1
    end

    # shared file database: one connection per Ractor, contended writes
    SQLite3::Database.new(shared) do |db|
      db.busy_timeout = 5000
      db.execute "INSERT INTO log VALUES (?, ?, ?)", [id, i, "r#{id}"]
      db.get_first_value("SELECT count(*) FROM log")
    end
  end
  raise "version_info" unless SQLite3::VERSION_INFO[:gem][:version] == SQLite3::VERSION
  checks
end

t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
ractors = RACTORS.times.map do |id|
  Ractor.new(id, ITERS, shared) { |id, iters, shared| workload(id, iters, shared) }
end
results = ractors.map(&:value)
elapsed = Process.clock_gettime(Process::CLOCK_MONOTONIC) - t0

total = SQLite3::Database.new(shared).then { |db| db.get_first_value("SELECT count(*) FROM log").tap { db.close } }
expected = RACTORS * ITERS
puts "ruby #{RUBY_VERSION} sqlite3 #{SQLite3::VERSION} (libsqlite #{SQLite3::SQLITE_LOADED_VERSION})"
puts "#{RACTORS} ractors x #{ITERS} iterations: checks=#{results.sum} log rows=#{total}/#{expected} in #{elapsed.round(2)}s"
exit(results.sum == expected && total == expected ? 0 : 1)

A related observation (not addressed here)

If the main Ractor exits while worker Ractors are still in the middle of queries, the process can hang during exit. Ruby's exit-time finalizer frees a leftover Statement, and sqlite3_finalize then waits on a connection mutex held by a Ractor that has stopped. I haven't reduced it to a minimal repro yet, so I don't know whether it belongs here or in Ruby. I'll follow up separately.

🤖 Generated with Claude Code

[flavorjones]

Thanks for the contribution, I'll review when I get a chance.