Support external write to rest catalog

[sfc-gh-abozkurt]

We need to accept rest catalog of source of truth to enable external reads. The diff algorithm between our internal catalogs and the rest catalog should be as follows:

Diff algorithm:

1. At the start fetch latest metadata from rest catalog (source of truth),
2. Verify no external ddl happened by checking schema and partition specs in the current metadata,
3. Try fetching the last synced snapshot from the metadata, if exists,
   3.1 If not exists (expired and removed), we sync our internal catalog from scratch (removing all files in the catalog) in separate transaction, and throw conflict error
4. Calculate the data files diff between our internal catalogs and the latest synced snapshot,
5. Send rest catalog requests with snapshot assertion, which fails with 409 in case of conflicts (external writes)
   5.1 Sync external writes in separate transaction in case of 409 errors and then throw error.
6. Track the last snapshot, that is successfully synced to rest catalog, in table.

Failure scenarios:

• external ddl => table needs to be recreated
• external write => sync from rest catalog and throw conflict error
• rest catalog request succeeds but tx aborts => no problem, we calculate the diff from the last synced snapshot id, tracked in table
• tx succeeds but rest catalog request fails => not possible, we abort tx when request fails

TODOS

• What happens our tx fails but ddl change request succeeds? In the next iteration we detect it as external ddl and error. (acceptable till we support applying ddl diff maybe)
• check if applying external writes cause lock conflicts with tables
• add tests including isolation tests (external writes by spark)