AWS sigv4 validation fails when chef is being a reverse proxy

[tomhughes]

Chef Server Version

Chef server 15.9.38 upgraded from 15.1.7 which was working.

Platform Details

Ubuntu 22.04

Configuration

Standalone, configured as follows:

api_fqdn "chef.openstreetmap.org"
default_orgname "openstreetmap"

addons['install'] = false

lb['api_fqdn'] = "chef.openstreetmap.org"
lb['web_ui_fqdn'] = "chef.openstreetmap.org"

nginx['non_ssl_port'] = false
nginx['ssl_port'] = 4443
nginx['ssl_certificate'] = '/etc/ssl/certs/chef.openstreetmap.org.pem'
nginx['ssl_certificate_key'] = '/etc/ssl/private/chef.openstreetmap.org.key'
nginx['server_name'] = "chef.openstreetmap.org"

opscode_erchef['base_resource_url'] = "https://chef.openstreetmap.org"

Scenario

Running chef server on port 4443 behind a reverse proxy that exposes it on port 443 using the following apache config:

<VirtualHost *:443>
        ServerName chef.openstreetmap.org
        ServerAdmin webmaster@openstreetmap.org

        CustomLog /var/log/apache2/chef.openstreetmap.org-access.log combined_extended
        ErrorLog /var/log/apache2/chef.openstreetmap.org-error.log

        SSLEngine on
        SSLProxyEngine on
        SSLCertificateFile /etc/ssl/certs/chef.openstreetmap.org.pem
        SSLCertificateKeyFile /etc/ssl/private/chef.openstreetmap.org.key

        ProxyPassMatch ^/.*\.git/ !
        ProxyPass / https://chef.openstreetmap.org:4443/
        ProxyPreserveHost on
</VirtualHost>

Steps to Reproduce

Attempt to upload or download cookbook content to/from bookshelf.

Expected Result

It works.

Actual Result

It fails with a not authorized response.

Analysis

I believe the following summarises what is happening:

• Knife uses a host header value of chef.openstreetmap.org:443 and signs the request using that value
• Apache proxies that to port 4443 and preserves chef.openstreetmap.org:443
• The chef nginx instance proxies that to bookshelf but rewrites the host header to chef.openstreetmap.org:4443
• Bookshelf attempts signature validation using chef.openstreetmap.org:4443 and the alternate of chef.openstreetmap.org with the port deleted and both fail

The culprit appears to be

chef-server/omnibus/files/server-ctl-cookbooks/infra-server/templates/default/nginx/nginx_chef_api_lb.conf.erb

Line 31 in 7940d04

proxy_set_header Host <%= @helper.host_header_var(@server_proto) %>;

combined with

chef-server/omnibus/files/server-ctl-cookbooks/infra-server/libraries/nginx_erb.rb

Line 14 in 7940d04

def host_header_var(proto)

which sets the host header to the port the chef server is running on.

Changing that template to use proxy_set_header Host $http_host; to preserve the host sent by the client appears to make things work again.