Skip to content

Storage backend compatibility

Andrew Gaul edited this page Apr 7, 2017 · 30 revisions

Some storage backends do not support all the features of S3:

Atmos

  • does not support multi-part upload

Azure

  • limits container names to 63 characters and restricts character set to lowercase alphanumeric and dash
  • maximum multipart upload size of 195 GB, see JCLOUDS-1223
  • does not support Expires header but does support Cache-Control header
  • does not support object-level access control
  • does not allow arbitrary list markers, partially emulated by mapping the last key to the opaque marker
  • does not return ETag as an MD5 hash, instead returning opaque data

B2

  • bucket names must be between 6 and 50 characters and consistent of only letters, number, and hyphens
  • requires buffering PUTs to calculate required SHA1 hash
  • does not return ETag as an MD5 hash, instead returning SHA1 hash
  • does not support blob-level access control
  • does not support conditional GET
  • does not support copying objects
  • does not support Content-Disposition, Content-Encoding, Content-Language, Content-MD5, or Expiry headers
  • multipart upload requires at least 2 parts

filesystem

  • does not support listing in-progress multipart uploads
  • does not support object metadata on Mac OS X, see JDK-8030048
  • does not support object metadata using Docker and aufs, see docker/docker#1070

Google Cloud Storage

  • does not allow arbitrary list markers, partially emulated by mapping the last key to the opaque marker
  • does not return ETag as an MD5 hash, instead returning opaque data
  • limits multi-part uploads to 32 parts

OpenStack Swift

  • does not support Content-Language headers
  • does not support copying objects with the If-None-Match header
  • does not support deleting user metadata keys, only replacing them
  • does not support object-level access control
  • exposes multipart upload parts as regular keys

Clone this wiki locally