Skip to content

Commit c3cbcae

Browse files
committed
docs: document race condition between config changes and reward claims (TRST-L-2)
Add comprehensive documentation for TRST-L-2 race condition where configuration changes (reducing eligibility period or enabling validation) can cause indexers to permanently lose rewards if their claim transactions are in-flight when the change occurs. Changes include: - NatSpec warnings on setEligibilityPeriod() and setEligibilityValidation() - Contract-level security warning in RewardsEligibilityOracle - Operational considerations section in documentation with mitigation strategies - Monitoring guidance for operators and indexers No code changes - mitigation relies on operational practices.
1 parent 771f9a9 commit c3cbcae

File tree

2 files changed

+116
-2
lines changed

2 files changed

+116
-2
lines changed

packages/issuance/contracts/eligibility/RewardsEligibilityOracle.md

Lines changed: 103 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -161,6 +161,107 @@ It might initially seem safer to allow indexers by default unless an oracle expl
161161

162162
In general to be rewarded for providing service on The Graph, there is expected to be proof provided of good operation (such as for proof of indexing). While proof should be required to receive rewards, the system is designed for participants to have confidence is being able to adequately prove good operation (and in the case of oracles, be seen by at least one observer) that is sufficient to allow the indexer to receive rewards. The oracle model is in general far more suited to collecting evidence of good operation, from multiple independent observers, rather than any observer being able to establish that an indexer is not providing good service.
163163

164+
## Operational Considerations
165+
166+
### Race Conditions with Configuration Changes
167+
168+
Configuration changes can create race conditions with in-flight reward claim transactions, potentially causing indexers to permanently lose rewards.
169+
170+
When an indexer submits a transaction to claim rewards through the RewardsManager:
171+
172+
1. The indexer is eligible at the time of transaction submission
173+
2. The transaction enters the mempool and waits for execution
174+
3. A configuration change occurs (e.g., reducing `eligibilityPeriod` or enabling `eligibilityValidation`)
175+
4. The transaction executes after the indexer is no longer eligible
176+
5. **The indexer is denied rewards** resulting in permanent loss for the indexer
177+
178+
This occurs because the RewardsManager's `takeRewards()` function returns 0 rewards for ineligible indexers, but the calling contract (Staking or SubgraphService) still marks the allocation as processed.
179+
180+
Circumstances potentially leading to this race condition:
181+
182+
1. **Reducing eligibility period** (`setEligibilityPeriod`):
183+
- Shortening the eligibility window may cause recently-approved indexers to become ineligible
184+
- Indexers near the end of their eligibility period become ineligible immediately
185+
186+
2. **Enabling eligibility validation** (`setEligibilityValidation`):
187+
- Switching from disabled (all eligible) to enabled (oracle-based)
188+
- Indexers without recent oracle renewals become ineligible immediately
189+
190+
3. **Oracle update delays**:
191+
- If oracles do not renew an indexer's eligibility before it expires
192+
- Combined with network congestion delaying claim transactions
193+
194+
4. **Network conditions**:
195+
- High gas prices causing indexers to delay transaction submission
196+
- Network congestion delaying transaction execution
197+
- Multiple blocks between submission and execution
198+
199+
#### Mitigation Strategies
200+
201+
Operators and indexers should implement these practices:
202+
203+
**For Operators:**
204+
205+
1. **Announce configuration changes in advance**:
206+
- Publish planned changes to eligibility period or validation state
207+
- Provide sufficient notice (e.g., 24-48 hours) before executing changes
208+
- Use governance forums, Discord, or official communication channels
209+
210+
2. **Implement two-step process for critical changes**:
211+
- First transaction: Announce the pending change with a delay period
212+
- Second transaction: Execute the change after the delay
213+
- This is a governance/operational practice, not enforced by the contract
214+
215+
3. **Avoid sudden reductions in eligibility**:
216+
- When reducing eligibility period, consider gradual reductions
217+
- Monitor pending transactions in the mempool before making changes
218+
- Time changes for periods of low network activity
219+
220+
4. **Coordinate with oracle operations**:
221+
- Ensure oracles are actively renewing indexer eligibility
222+
- Verify oracle health before enabling eligibility validation
223+
- Monitor `lastOracleUpdateTime` to detect oracle failures
224+
225+
**For Indexers:**
226+
227+
1. **Monitor eligibility status closely**:
228+
- Regularly check `isEligible()` and `getEligibilityRenewalTime()`
229+
- Calculate when eligibility will expire (`renewalTime + eligibilityPeriod`)
230+
- Set up alerts for approaching expiration
231+
232+
2. **Claim rewards with sufficient margin**:
233+
- Don't wait until the last moment of eligibility period
234+
- Account for network congestion and gas price volatility
235+
- Consider claiming more frequently rather than in large batches
236+
237+
3. **Watch for configuration change announcements**:
238+
- Monitor governance communications and proposals
239+
- Subscribe to operator announcements
240+
- Plan claim transactions around announced changes
241+
242+
4. **Use appropriate gas pricing**:
243+
- During announced configuration changes, use higher gas prices
244+
- Ensure transactions execute quickly during critical windows
245+
- Monitor transaction status and resubmit if necessary
246+
247+
5. **Understand the risk**:
248+
- Be aware that rewards can be permanently lost due to race conditions
249+
- Factor this risk into reward claiming strategies
250+
251+
#### Monitoring and Detection
252+
253+
Operators should monitor:
254+
255+
- `RewardsDeniedDueToEligibility` events
256+
- Time between configuration changes and claim transactions
257+
258+
Indexers should monitor:
259+
260+
- Their own eligibility status via `isEligible()`
261+
- `EligibilityPeriodUpdated` events
262+
- `EligibilityValidationUpdated` events
263+
- `IndexerEligibilityRenewed` events for their address
264+
164265
## Events
165266

166267
```solidity
@@ -193,8 +294,8 @@ The system is deployed with reasonable defaults but can be adjusted as required.
193294

194295
### Normal Operation
195296

196-
1. Oracles periodically call `renewIndexerEligibility()` to renew eligibility for indexers
197-
2. Reward systems call `isEligible()` to check indexer eligibility
297+
1. Oracle nodes periodically call `renewIndexerEligibility()` to renew eligibility for indexers
298+
2. Reward Manager calls `isEligible()` to check indexer eligibility
198299
3. Operators adjust parameters as needed via configuration functions
199300
4. The operation of the system is monitored and adjusted as needed
200301

packages/issuance/contracts/eligibility/RewardsEligibilityOracle.sol

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,11 @@ import { BaseUpgradeable } from "../common/BaseUpgradeable.sol";
1919
* @dev Note: If the eligibility period is set to an extremely large value that exceeds the current
2020
* block timestamp, all indexers (including those never registered) will be eligible.
2121
* @custom:security-contact Please email [email protected] if you find any bugs. We might have an active bug bounty program.
22+
* @custom:security-warning Configuration changes (eligibility period, validation toggle) can create race
23+
* conditions with in-flight reward claim transactions. When configuration changes make an indexer ineligible
24+
* between transaction submission and execution, the indexer permanently loses those rewards.
25+
* Operators should announce configuration changes in advance and consider implementing
26+
* a two-step process (announce delay, then execute) for changes that reduce eligibility.
2227
*/
2328
contract RewardsEligibilityOracle is
2429
BaseUpgradeable,
@@ -133,6 +138,10 @@ contract RewardsEligibilityOracle is
133138
* @dev Only callable by accounts with the OPERATOR_ROLE
134139
* @param eligibilityPeriod New eligibility period in seconds
135140
* @return True if the state is as requested (eligibility period is set to the specified value)
141+
* @custom:warning Configuration changes can affect in-flight reward claim transactions. Reducing the
142+
* eligibility period may cause indexers to lose rewards if their claim transactions execute after
143+
* they become ineligible due to the configuration change. Consider announcing configuration changes
144+
* in advance and using a two-step process (announce, then execute) for changes that reduce eligibility.
136145
*/
137146
function setEligibilityPeriod(uint256 eligibilityPeriod) external override onlyRole(OPERATOR_ROLE) returns (bool) {
138147
RewardsEligibilityOracleData storage $ = _getRewardsEligibilityOracleStorage();
@@ -171,6 +180,10 @@ contract RewardsEligibilityOracle is
171180
* @dev Only callable by accounts with the OPERATOR_ROLE
172181
* @param enabled True to enable eligibility validation, false to disable
173182
* @return True if successfully set (always the case for current code)
183+
* @custom:warning Enabling eligibility validation can affect in-flight reward claim transactions.
184+
* Indexers who submitted claim transactions while validation was disabled may lose rewards if
185+
* validation is enabled before their transactions execute and they are not marked as eligible.
186+
* Consider announcing this change in advance to allow indexers to adjust their claiming behavior.
174187
*/
175188
function setEligibilityValidation(bool enabled) external override onlyRole(OPERATOR_ROLE) returns (bool) {
176189
RewardsEligibilityOracleData storage $ = _getRewardsEligibilityOracleStorage();

0 commit comments

Comments
 (0)