#892 gives the session of a domain one place to be stopped and started from: serviceStateLock serialises disable(), enable(), shutdown(), the fractional reconfiguration and the recovery from a failed replay, and sessionGeneration says which session a thread stopped, so a recovery does not start a session back which something else has already replaced.
ExternalChangelogDomain.applyConfigurationChange() (opendj-server-legacy/src/main/java/org/opends/server/replication/plugin/ExternalChangelogDomain.java:95) reaches the session by another road: domain.changeConfig(Set, Set) -> ReplicationDomain.restartService(), which is disableService() + enableService() with neither the lock nor the counter. A replay thread waiting out its backoff in restartSession() has stopped the session under the lock and checks both the generation and isListenerShuttingDown() before starting it again; against this path only the second half of that guard is doing any work, and the interleaving where the ECL thread has stopped the session and not yet started it leaves both of them calling enableService() on the same broker.
The same is true of changeConfig(ReplicationDomainCfg) and readAssuredConfig(), which the javadoc of sessionGeneration names rather than closes.
The tidy fix is for the pairing to live where the session does: a restartService() in ReplicationDomain which takes the lock and bumps a generation owned next to sessionLock, so that every restart in the hierarchy is countable and the subclass needs neither a second lock nor the isListenerShuttingDown() probe.
Found while reviewing #892.
#892 gives the session of a domain one place to be stopped and started from:
serviceStateLockserialisesdisable(),enable(),shutdown(), the fractional reconfiguration and the recovery from a failed replay, andsessionGenerationsays which session a thread stopped, so a recovery does not start a session back which something else has already replaced.ExternalChangelogDomain.applyConfigurationChange()(opendj-server-legacy/src/main/java/org/opends/server/replication/plugin/ExternalChangelogDomain.java:95) reaches the session by another road:domain.changeConfig(Set, Set)->ReplicationDomain.restartService(), which isdisableService()+enableService()with neither the lock nor the counter. A replay thread waiting out its backoff inrestartSession()has stopped the session under the lock and checks both the generation andisListenerShuttingDown()before starting it again; against this path only the second half of that guard is doing any work, and the interleaving where the ECL thread has stopped the session and not yet started it leaves both of them callingenableService()on the same broker.The same is true of
changeConfig(ReplicationDomainCfg)andreadAssuredConfig(), which the javadoc ofsessionGenerationnames rather than closes.The tidy fix is for the pairing to live where the session does: a
restartService()inReplicationDomainwhich takes the lock and bumps a generation owned next tosessionLock, so that every restart in the hierarchy is countable and the subclass needs neither a second lock nor theisListenerShuttingDown()probe.Found while reviewing #892.