[#1011] Wait out the connection limit an account is given of its own

[vharseko]

Fixes #1011.

The defect

CachedConnection.isWorthRetrying(SQLException, ConnectDialect) is the one gate that decides whether a connect the database refused is waited out or reported: the borrow loop of the pool asks it, and so does the catalog connect of JDBCStorage. It answers yes for SQLState 53300/57P03 and for the vendor codes the dialect lists — and those codes are the limits of the server. The limit an account is given of its own falls through them, so a database momentarily out of connections for this account is indistinguishable from a password it will never accept: the borrow fails at once instead of waiting its deadline out, and a borrow is not something JDBCStorage.write() replays — it is taken outside the loop on purpose, so the operation fails with it.

Measured against the images the suites use, both refusals arrive with nothing but the vendor code to go on:

refusal	driver	class / state / code
GRANT … MAX_USER_CONNECTIONS 1, second connect	mysql-connector-j 9.2	SQLSyntaxErrorException / 42000 / 1226
GRANT … MAX_CONNECTIONS_PER_HOUR 1, second connect	mysql-connector-j 9.2	SQLSyntaxErrorException / 42000 / 1226
profile SESSIONS_PER_USER 1, second connect	ojdbc8 23.7	SQLException / 61000 / 2391
global max_user_connections=1 (already retried)	mysql-connector-j 9.2	SQLSyntaxErrorException / 42000 / 1203

What changed

• mysql 1226 is waited out, unless the resource it names is granted per hour. The same code carries the MAX_USER_CONNECTIONS of a grant — the connections this account may hold at once, which a connection of this pool going back clears, exactly as the limit of the server does — and the resources granted per hour, which only the top of the hour clears. Waiting one of those out would cost every borrow the whole pool.timeout, a worker thread parked in each, for as long as the hour lasts. They are told apart by the resource the server names in the message: it is filled in as a literal of its own rather than translated with the rest of the text. A message that names none — a proxy that rewrote it, a driver that kept the code and dropped the text — is taken for the concurrent limit: that is the one an account is given in practice, and the wait it costs is bounded by the deadline of the borrow, while reading it as permanent fails an operation a connection of ours would have served. That is accountLimitCodes, asked of the message of the same link the code came from.
• oracle ORA-02391 and ORA-00018 join ORA-00020. ORA-02391 is the SESSIONS_PER_USER of the account's profile, the per-account sibling of the processes of the instance; ORA-00018 is the sessions of the instance, the global sibling that was missing beside it. All three are cleared by a session ending.
• mysql 1129 is left out, and the reason is in the code. A host the server blocked after too many failed connects stays blocked until an administrator flushes the host cache; waiting a borrow's deadline out would only hide the one message naming the remedy behind a timeout. It belongs with the password that is not accepted.
• ConnectDialect.isWorthRetrying takes the SQLException rather than its code alone, since one of these verdicts now reads the message beside it.

Not in this change

• postgresql needs nothing: a role's and a database's CONNECTION LIMIT are both reported as 53300, which this gate already waits out.
• sql server has no per-login limit of this kind; 17809 is the server's own and is already listed. The azure throttling codes (40501, 49918-49920) are a different family and not this issue.
• mariadb is not a line in this table: jdbc:mariadb: is a dialect this class does not know at all, so such a url has no connect bound and no read bound either — which reportUnknownDialect warns about since [#872] Bound the connect of the JDBC pool and report a connect it cannot make #876. Giving it a dialect is its own change, and the driver is not one this server ships.
• JDBCStorage.scopeOf reads a stamp connect refused with 1226 or 1203 as FailureScope.TREE, so the table of that tree is remembered as unstampable for the life of the backend where a momentary limit was met. Same root cause — mysql puts these in the syntax error class — different predicate and a different consequence (the comment is a diagnostic aid), so it belongs in an issue of its own rather than in this change.

Verification

• Measured rather than read. The table above was taken against the images the suites use - mysql:9.2 with mysql-connector-j 9.2 and gvenzl/oracle-free:23.26.2-slim-faststart with ojdbc8 - by granting an account MAX_USER_CONNECTIONS 1, then MAX_CONNECTIONS_PER_HOUR 1, then a profile with SESSIONS_PER_USER 1, and opening one connection more than each allows. ORA-00018 is the one code here that is read and not measured: an instance cannot be brought below its own session floor to provoke it - oracle recomputes sessions upwards from processes - which is the footing ORA-00020 and 12516-12520 stand on already.
• CachedConnectionTestCase - six new tests, watched failing first: the per-account limit of mysql, the same code naming an hourly resource, the same code naming no resource at all, a host the server blocked, and the two oracle session limits. Four of them fail on master; the two that must stay "reported at once" - the hourly resource and the blocked host - pass there too and are what pins this change to the width it has.
• MySqlTestCase.testAPerAccountConnectionLimitIsWorthRetrying - new, and watched failing against master's classification with the connection the server actually refused: expected [true] but found [false] on the verdict, the message reading User 'limited1011' has exceeded the 'max_user_connections' resource (current value: 1). It needed sequential = true on the class, which PgSqlTestCase and OracleTestCase carry already: TestListener asks that of the class a running test is declared by, so a suite made of inherited tests alone only meets the rule when it declares one of its own.
• Suites, all green: CachedConnectionTestCase 105, MySqlTestCase 79, PgSqlTestCase 79, JDBCStorageRetryTest 67, JDBCStatementBoundTestCase 44, jdbc.EncryptedTestCase 35, cassandra.EncryptedTestCase 35, CatalogConnectionTestCase 13, StampConnectionTestCase 5.
• OracleTestCase and MsSqlTestCase are not among them - neither comes up on this machine - so the oracle codes are covered by the unit tests and the measurement above, and left to CI beyond that. The oracle suite could not carry a test of its own either way: it connects as the application account, which may create neither a profile nor a user.