Replace environment and restHost with endpoint

CHANGELOG.md

@@ -16,6 +16,10 @@ the full mapping table.
 - Clients are constructed through `Ably\PubSub\Server::createHttpClient()`, which declares the server side on the wire. It accepts everything the constructor accepted: an options array, an API key string, a token string, or a `ClientOptions` instance. A client constructed directly declares no side, and is rejected on accounts with monthly-active-user pricing enabled.
 - Removed `AblyRest::setAblyAgentHeader()` and `AblyRest::setLibraryFlavourString()`, replaced by the per-client `agents` client option.
 - Removed `ably-loader.php`; Composer's autoloader is the only supported install path.
+- Added the `endpoint` client option (REC1, REC2), which is now the only way to choose where the client connects. The default primary domain is `main.realtime.ably.net`, with fallback hosts `main.[a-e].fallback.ably-realtime.com`.
+- Removed the `environment` and `restHost` client options in favour of `endpoint`. They are now ignored if passed, so a client still setting `environment` connects to production.
+- Setting `port` or `tlsPort` no longer disables the default fallback hosts; a hostname `endpoint` does.
+- `ClientOptions::getPrimaryRestHost()` is now `getPrimaryDomain()`. `Defaults::$restHost`, `Defaults::$realtimeHost` and `Defaults::getEnvironmentFallbackHosts()` are replaced by `Defaults::$endpoint`, `Defaults::getPrimaryDomain()` and `Defaults::getEndpointFallbackHosts()`.
 - Removed the `demo/` Heroku application and its `Procfile`.
 - The minimum supported PHP version is now 8.1; the SDK is tested on 8.1 through 8.5.
 

CONTRIBUTING.md

@@ -3,6 +3,6 @@
 1. Fork it
 2. Create your feature branch (`git checkout -b my-new-feature`)
 3. Commit your changes (`git commit -am 'Add some feature'`)
-4. Ensure you have added suitable tests and the test suite is passing (run `vendor/bin/phpunit`)
+4. Ensure you have added suitable tests and the test suite is passing (run `vendor/bin/phpunit`). Tests run against the `nonprod:sandbox` endpoint; set `ABLY_ENDPOINT` to use another.
 4. Push to the branch (`git push origin my-new-feature`)
 5. Create a new Pull Request

UPDATING.md

@@ -6,7 +6,7 @@ Version 2.0.0 ships from a new package, `ably/pubsub-server`, under a new namesp
 
 Under monthly-active-user pricing the platform has to classify every connection as device-side or server-side. The new package declares that automatically, on every request, in the `Ably-Agent` header; the old constructor cannot declare anything. That is the forcing function for this migration: once monthly-active-user pricing is live, `new Ably\AblyRest(...)` from `ably/ably-php` is rejected on accounts where it is enabled.
 
-The client `Server::createHttpClient()` returns is the same REST client as before, renamed from `AblyRest` to `PubSubHttpClient`. Channels, message publishing, history, presence, authentication, push admin, crypto and every `ClientOptions` key behave exactly as they did in 1.x. For most applications the migration is confined to the `composer require` line, the `use` statements, and the constructor call.
+The client `Server::createHttpClient()` returns is the same REST client as before, renamed from `AblyRest` to `PubSubHttpClient`. Channels, message publishing, history, presence, authentication, push admin and crypto behave exactly as they did in 1.x. The one change to `ClientOptions` is how the client chooses where to connect: the `endpoint` option replaces `environment` and `restHost` (see [Endpoint](#endpoint-replaces-environment-and-resthost)). For most applications the migration is confined to the `composer require` line, the `use` statements, the constructor call, and any host options.
 
 ### Mapping
 
@@ -20,6 +20,7 @@ The client `Server::createHttpClient()` returns is the same REST client as befor
 | `require 'ably-loader.php';` | removed — use Composer's autoloader (`vendor/autoload.php`) |
 | PHP 7.2 – 8.0 | PHP `^8.1` (tested on 8.1 – 8.5) |
 | `\Ably\AblyRest` type hints | `\Ably\PubSub\PubSubHttpClient` ([091d](https://ably.atlassian.net/wiki/spaces/product/pages/5363957781)) |
+| `'environment' => 'sandbox'` / `'restHost' => '…'` | `'endpoint' => …` (see [Endpoint](#endpoint-replaces-environment-and-resthost)) |
 
 Every class moves namespace, and `AblyRest` is the only one that also changes name, so the rename is mechanical: replace the prefix `Ably\` with `Ably\PubSub\` throughout — including in type hints, `catch` blocks and fully-qualified string class names — and then `AblyRest` with `PubSubHttpClient`.
 
@@ -41,6 +42,42 @@ $ably->channel('test-channel')->publish('test-event', 'hello world');
 
 `createHttpClient()` accepts everything the 1.x constructor accepted: an options array, an API key string, a token string, or a `ClientOptions` instance. A `ClientOptions` instance you pass in is copied rather than mutated.
 
+### Endpoint replaces environment and restHost
+
+The `endpoint` client option is now the only way to choose where the client connects. `environment` and `restHost` are removed. Like any other unknown key, they are now ignored, so a client still passing `'environment' => 'sandbox'` connects to production: replace them before upgrading.
+
+| 1.x | 2.0 |
+| --- | --- |
+| no host options | no change; traffic moves from `rest.ably.io` to `main.realtime.ably.net` |
+| `'environment' => 'sandbox'` | `'endpoint' => 'nonprod:sandbox'` |
+| `'environment' => 'acme'` (dedicated cluster) | `'endpoint' => 'acme'` |
+| `'restHost' => 'localhost'` | `'endpoint' => 'localhost'` |
+| `'restHost' => 'rest.example.com'` | `'endpoint' => 'rest.example.com'` |
+| custom `'fallbackHosts' => […]` | unchanged |
+
+How `endpoint` resolves:
+
+| `endpoint` | Primary domain | Default fallback hosts |
+| --- | --- | --- |
+| unset (`main`) | `main.realtime.ably.net` | `main.[a-e].fallback.ably-realtime.com` |
+| `[id]`, e.g. `acme` | `[id].realtime.ably.net` | `[id].[a-e].fallback.ably-realtime.com` |
+| `nonprod:[id]`, e.g. `nonprod:sandbox` | `[id].realtime.ably-nonprod.net` | `[id].[a-e].fallback.ably-realtime-nonprod.com` |
+| a hostname: contains `.` or `::`, or is `localhost` | the value as given | none |
+
+A non-empty `fallbackHosts` still replaces the default fallback hosts. Setting `port` or `tlsPort` no longer disables the default fallback hosts; only a hostname `endpoint` does.
+
+```php
+// 1.x
+$ably = new AblyRest(['key' => getenv('ABLY_API_KEY'), 'environment' => 'sandbox']);
+
+// 2.0
+$ably = Server::createHttpClient(['key' => getenv('ABLY_API_KEY'), 'endpoint' => 'nonprod:sandbox']);
+```
+
+`ClientOptions::getPrimaryRestHost()` is renamed `getPrimaryDomain()`, and `Defaults::$restHost`, `Defaults::$realtimeHost` and `Defaults::getEnvironmentFallbackHosts()` are replaced by `Defaults::$endpoint`, `Defaults::getPrimaryDomain()` and `Defaults::getEndpointFallbackHosts()`.
+
+Firewall allowlists and outbound proxies must allow `*.realtime.ably.net` and `*.fallback.ably-realtime.com` in place of `rest.ably.io` and `*.ably-realtime.com`.
+
 ### Declaring the side
 
 Construct through the door. `new Ably\PubSub\PubSubHttpClient(...)` still works — the library and its own tests use it — but it declares no side, and will be rejected on monthly-active-user-enabled accounts just as the 1.x constructor is. The door produces:
@@ -67,6 +104,7 @@ The `agents` option is per-client, unlike the process-global static setters it r
 * `AblyRest::setAblyAgentHeader()` and `AblyRest::$agents` — replaced by the per-client `agents` option.
 * `AblyRest::setLibraryFlavourString()` — already deprecated in 1.x; replaced by the same option.
 * `ably-loader.php`, the hand-rolled autoloader — Composer is the only supported install path.
+* The `environment` and `restHost` client options — replaced by `endpoint`. They are ignored if passed.
 * The `demo/` Heroku application and its `Procfile`.
 * PHP 7.2 – 8.0 support.
 
@@ -75,7 +113,7 @@ The `agents` option is per-client, unlike the process-global static setters it r
 ### Unchanged
 
 * The REST client and its whole surface: `channel()`, `channels`, publishing, message history, presence and presence history, `auth`, token requests and token issuing, `push` admin, `stats`, `time()`, and crypto.
-* Every `ClientOptions` key, and the array / key-string / token-string / `ClientOptions` forms of the constructor argument.
+* Every `ClientOptions` key other than the host options above, and the array / key-string / token-string / `ClientOptions` forms of the constructor argument.
 * Message and error semantics, including `AblyException` and its codes.
 * Requirements: `ext-json`, `ext-curl`, `ext-openssl`, and `rybakit/msgpack` for the msgpack protocol.
 

src/Defaults.php

@@ -5,29 +5,63 @@ class Defaults {
     const API_VERSION = '2';
     const LIB_VERSION = '2.0.0';
 
-    static $restHost = "rest.ably.io";
-    static $realtimeHost = "realtime.ably.io";
+    static $endpoint = 'main';
     static $port = 80;
     static $tlsPort = 443;
 
     static $internetCheckUrl = "https://internet-up.ably-realtime.com/is-the-internet-up.txt";
     static $internetCheckOk = "yes\n";
 
     static $fallbackHosts = [
-        'a.ably-realtime.com',
-        'b.ably-realtime.com',
-        'c.ably-realtime.com',
-        'd.ably-realtime.com',
-        'e.ably-realtime.com',
+        'main.a.fallback.ably-realtime.com',
+        'main.b.fallback.ably-realtime.com',
+        'main.c.fallback.ably-realtime.com',
+        'main.d.fallback.ably-realtime.com',
+        'main.e.fallback.ably-realtime.com',
     ];
 
-    static function getEnvironmentFallbackHosts($environment) {
-        return [
-            $environment."-a-fallback.ably-realtime.com",
-            $environment."-b-fallback.ably-realtime.com",
-            $environment."-c-fallback.ably-realtime.com",
-            $environment."-d-fallback.ably-realtime.com",
-            $environment."-e-fallback.ably-realtime.com"
-        ];
+    const NONPROD_PREFIX = 'nonprod:';
+
+    /**
+     * An endpoint is taken as a hostname rather than a routing policy id when
+     * it contains a `.` or `::`, or is `localhost` (REC1b2).
+     */
+    static function isHostname($endpoint) {
+        return strpos($endpoint, '.') !== false
+            || strpos($endpoint, '::') !== false
+            || $endpoint === 'localhost';
+    }
+
+    /**
+     * Resolves the primary domain from an `endpoint` client option (REC1).
+     */
+    static function getPrimaryDomain($endpoint) {
+        if (self::isHostname($endpoint)) {
+            return $endpoint; // REC1b2
+        }
+        if (strpos($endpoint, self::NONPROD_PREFIX) === 0) {
+            return substr($endpoint, strlen(self::NONPROD_PREFIX)).'.realtime.ably-nonprod.net'; // REC1b3
+        }
+        return $endpoint.'.realtime.ably.net'; // REC1b4
+    }
+
+    /**
+     * Resolves the default fallback domains for an `endpoint` client option (REC2c).
+     */
+    static function getEndpointFallbackHosts($endpoint) {
+        if (self::isHostname($endpoint)) {
+            return []; // REC2c2
+        }
+        if (strpos($endpoint, self::NONPROD_PREFIX) === 0) {
+            $id = substr($endpoint, strlen(self::NONPROD_PREFIX));
+            return self::fallbackHostsFor($id, 'ably-realtime-nonprod.com'); // REC2c3
+        }
+        return self::fallbackHostsFor($endpoint, 'ably-realtime.com'); // REC2c1, REC2c4
+    }
+
+    private static function fallbackHostsFor($id, $domain) {
+        return array_map(function ($letter) use ($id, $domain) {
+            return $id.'.'.$letter.'.fallback.'.$domain;
+        }, ['a', 'b', 'c', 'd', 'e']);
     }
 }

src/Host.php

@@ -8,7 +8,7 @@ class Host {
 
     public function __construct($clientOptions)
     {
-        $this->primaryHost = $clientOptions->getPrimaryRestHost();
+        $this->primaryHost = $clientOptions->getPrimaryDomain();
         $this->fallbackHosts = $clientOptions->getFallbackHosts();
         $this->hostCache = new HostCache($clientOptions->fallbackRetryTimeout);
     }

src/Models/ClientOptions.php

@@ -35,10 +35,14 @@ class ClientOptions extends AuthOptions {
     public $useBinaryProtocol = true;
 
     /**
-     * @var string alternate server domain
-     * For development environments only.
+     * @var string|null where the client connects (REC1), default `main`.
+     * One of: a routing policy id such as `main` or a dedicated cluster id,
+     * which resolves to `[id].realtime.ably.net`; `nonprod:[id]`, which
+     * resolves to `[id].realtime.ably-nonprod.net`; or a hostname (anything
+     * containing `.` or `::`, or `localhost`), which is used as given and has
+     * no default fallback hosts.
      */
-    public $restHost;
+    public $endpoint;
 
     /**
      * @var integer Allows a non-default Ably non-TLS port to be used.
@@ -53,19 +57,13 @@ class ClientOptions extends AuthOptions {
     public $tlsPort;
 
     /**
-     * @var string optional prefix to be prepended to $restHost
-     * Example: 'sandbox' -> 'sandbox-rest.ably.io'
-     */
-    public $environment;
-
-    /**
-     * @var string[] fallback hosts, used when connection to default host fails, populated automatically
+     * @var string[] fallback hosts, used when connection to the primary domain fails; when empty, derived from $endpoint
      */
     public $fallbackHosts = [];
 
     /**
      * @var integer – default 600000 (10 minutes) the period in milliseconds
-     * before HTTP requests are retried against the default endpoint
+     * before HTTP requests are retried against the primary domain
      */
     public $fallbackRetryTimeout = 600000;
 
@@ -151,35 +149,27 @@ public static function normalizeConstructorArgument( $options ) {
         return $options;
     }
 
-    private function isProductionEnvironment() {
-        return empty($this->environment) || strcasecmp($this->environment, "production") == 0;
-    }
-
-    private function isDefaultPort() {
-        return $this->tls ? $this->tlsPort == Defaults::$tlsPort : $this->port == Defaults::$port;
-    }
-
     private function activePort() {
         return $this->tls ? $this->tlsPort : $this->port;
     }
 
-    private function isDefaultRestHost() {
-        return $this->restHost == Defaults::$restHost;
-    }
-
-    public function getPrimaryRestHost() {
-        if ($this->isDefaultRestHost()) {
-            return $this->isProductionEnvironment() ? $this->restHost : $this->environment.'-'.$this->restHost;
-        }
-        return $this->restHost;
+    /**
+     * The domain both REST requests and, in SDKs that have one, the realtime
+     * connection use (REC1, RSC25).
+     */
+    public function getPrimaryDomain() {
+        return Defaults::getPrimaryDomain($this->endpoint);
     }
 
+    /**
+     * An explicit, non-empty `fallbackHosts` replaces the defaults (REC2a2);
+     * otherwise the defaults are derived from `endpoint` (REC2c).
+     */
     public function getFallbackHosts() {
-        $fallbacks = $this->fallbackHosts ?? [];
-        if (empty($this->fallbackHosts) && $this->isDefaultRestHost() && $this->isDefaultPort()) {
-            $fallbacks = $this->isProductionEnvironment() ? Defaults::$fallbackHosts : Defaults::getEnvironmentFallbackHosts($this->environment);
+        if (!empty($this->fallbackHosts)) {
+            return $this->fallbackHosts;
         }
-        return $fallbacks;
+        return Defaults::getEndpointFallbackHosts($this->endpoint);
     }
 
     public function getHostUrl($host) {
@@ -188,8 +178,8 @@ public function getHostUrl($host) {
 
     public function __construct( $options = [] ) {
         parent::__construct( $options );
-        if (empty($this->restHost)) {
-            $this->restHost = Defaults::$restHost;
+        if (empty($this->endpoint)) {
+            $this->endpoint = Defaults::$endpoint;
         }
         if (empty($this->port)) {
             $this->port = Defaults::$port;

src/PubSubHttpClient.php

@@ -219,7 +219,7 @@ public function requestInternal( $method, $path, $headers = [], $params = [], $r
             $hostUrl = $this->options->getHostUrl($host). $path;
             try {
                 $updatedHeaders = $mergedHeaders;
-                if ($host != $this->options->getPrimaryRestHost()) { // set hostHeader for fallback host (RSC15j)
+                if ($host != $this->options->getPrimaryDomain()) { // set hostHeader for fallback host (RSC15j)
                     $updatedHeaders[] = "Host: " . $host;
                 }
                 $response = $this->http->request( $method, $hostUrl, $updatedHeaders, $params );

tests/ChannelIdempotentTest.php

@@ -168,9 +168,9 @@ public function testIdempotentLibraryGeneratedPublish() {
             'idempotentRestPublishing' => true,
             'httpClass' => 'tests\HttpMockIdempotent',
             'fallbackHosts' => [
-                self::$ably->options->getPrimaryRestHost(),
-                self::$ably->options->getPrimaryRestHost(),
-                self::$ably->options->getPrimaryRestHost(),
+                self::$ably->options->getPrimaryDomain(),
+                self::$ably->options->getPrimaryDomain(),
+                self::$ably->options->getPrimaryDomain(),
             ],
         ] ) );
 

tests/ChannelMessagesTest.php

@@ -468,7 +468,7 @@ public function testEncodingInteroperabilityRawToAbly() {
 
         $defaultOpts = new \Ably\PubSub\Models\ClientOptions( self::$defaultOptions );
         $http = new \Ably\PubSub\Http( $defaultOpts ); // initialize http class for raw requests with default timeouts
-        $server = 'https://' . $defaultOpts->getPrimaryRestHost();
+        $server = 'https://' . $defaultOpts->getPrimaryDomain();
 
         $messages = [];
         foreach ($fixture->messages as $i => $testMsgData) {
@@ -514,7 +514,7 @@ public function testEncodingInteroperabilityAblyToRaw() {
 
         $defaultOpts = new \Ably\PubSub\Models\ClientOptions( self::$defaultOptions );
         $http = new \Ably\PubSub\Http( $defaultOpts ); // initialize http class for raw requests with default timeouts
-        $server = 'https://' . $defaultOpts->getPrimaryRestHost();
+        $server = 'https://' . $defaultOpts->getPrimaryDomain();
 
         $messages = [];
         foreach ($fixture->messages as $i => $testMsgData) {