rl: opt-in EMA-based low-update trigger for greedy selection (experimental)

[kimjune01]

What this PR does

Adds an opt-in mechanism to track exponential moving averages of per-production |ΔQ| during RL updates, and to force greedy selection on a slot when the EMA of every RL rule on that slot falls below a configured threshold.

Three new parameters, all off by default:

• chunk-gate (on/off, default off) — enables the mechanism
• chunk-gate-threshold (default 0.01) — EMA threshold below which a rule is treated as low-update. The value is an exposed default, not a recommended universal setting.
• chunk-gate-ema-decay (default 0.95) — smoothing factor for the EMA update

Default configuration leaves existing code paths inactive. chunk-gate=on is required for the new logic to have any effect.

Intent

The mechanism lets users experiment with low recent TD-update magnitude as an opt-in trigger for greedy selection, so that chunking has a deterministic result to compile over. Low EMA of |ΔQ| does not imply policy optimality or full convergence. It is a local heuristic about recent update magnitude.

The override currently intercepts the exploration policy step in run_preference_semantics() and forces greedy choice when a slot's RL rules are all below threshold. It does not audit interactions with non-RL preferences, tie handling, or the broader preference semantics, and may affect learned policy behavior in ways this PR has not measured.

Open for discussion; not a merge proposal.

Changes

File	Change
production.h	Add rl_ema_delta_q field to production struct
production.cpp, rete.cpp, reinforcement_learning.cpp	Initialize new field to 1.0 at all creation sites
reinforcement_learning.h	Declare 3 new params + rl_slot_converged()
reinforcement_learning.cpp	Update EMA in rl_perform_update(), add param init, implement rl_slot_converged()
decide.cpp	In run_preference_semantics(), check EMA state before exploration policy; if all RL rules on the slot are below threshold, select greedily

Tests

All pass locally (M4 Pro, macOS, 16s clean build):

• testRLConvergenceGate — agent with chunk-gate ON, 50 decisions, completes successfully
• testRLConvergenceGateOff — same agent with chunk-gate OFF, regression test
• testRLConvergenceGateParams — verifies new params accepted by command parser
• Existing RL/chunking tests (Chunk_RL_Proposal, RL_Variablization, testPreferenceSemantics, testLearn) all pass

Background reference

• Laird, J. E. (2022). "Introduction to the Soar Cognitive Architecture." §4 discusses composition between stochastic RL selection and deterministic chunking.