S3 client for ExStorageService and other S3-compatible object storage.
Build a client config once and pass it to every call. The config captures its endpoint, region, credentials and transport, so in-flight listings and multipart uploads keep their connection even if application settings change later.
{:ok, s3} =
ExStorageServiceSdk.new(
endpoint: "http://localhost:9000",
region: "us-east-1",
access_key_id: "AKIA...",
secret_access_key: "..."
)
{:ok, %{etag: etag}} = ExStorageServiceSdk.put_object(s3, "bucket", "a b+c%.txt", "hello")
{:ok, %{body: "hello"}} = ExStorageServiceSdk.get_object(s3, "bucket", "a b+c%.txt")Operations
call/4 runs any operation in the pinned S3 service model (AWS CLI 2.31.18) with
model-shaped input and output maps, for example
call(s3, "ListObjectsV2", %{"Bucket" => "b", "EncodingType" => "url"}).
The helpers in this module cover common object and multipart operations and
return response metadata alongside the result.
Errors
Every failure is {:error, %ExStorageServiceSdk.Error{}}:
category: :service- the server answered with an error.http_statusdistinguishes 404, 412, 416 and so on.codeis the S3 error code (for example"NoSuchKey"or"PreconditionFailed"), or"HTTP<status>"when the response had no body, as for HEAD requests.category: :transport- connection, TLS or timeout failure.codeis an atom.category: :protocol- the server answered with something that is not valid S3, including an HTTP 200 response that carries an S3<Error>document.
Retries
Reads (GET, HEAD, List) and idempotent multipart steps (UploadPart,
AbortMultipartUpload) retry transient failures up to three attempts in total.
PutObject, CopyObject, DeleteObject and CompleteMultipartUpload are never
replayed automatically. If such a request may have reached the server but its
outcome is unknown, the call fails with code: :indeterminate. For conditional
writes, reconcile by reading the object (for example comparing its ETag) before
deciding to try again, because a replayed If-None-Match: * write would fail
with 412 even when the first attempt succeeded.
Summary
Functions
Aborts a multipart upload, discarding its uploaded parts.
Runs any modeled S3 operation.
Completes a multipart upload. parts is a list of {part_number, etag}; it is
sorted by part number before sending.
Copies input["CopySource"] to bucket/key.
Starts a multipart upload and returns its result map, including "UploadId".
Deletes an object. Deleting a missing key succeeds, as in S3.
Reads an object into memory.
Starts a GET and returns the open ExStorageServiceSdk.S3.Response with its
body still on the wire. Consume it with
config.transport.consume(response, acc, reducer, opts) (which always closes
it), or release it with config.transport.close(response).
Reads object metadata. Returns %{status, etag, headers, result}. A missing object
fails with %Error{http_status: 404}.
Lists one page of keys.
Builds a client config.
Lazily streams result pages of a paginated operation, following continuation
tokens. Each element is {:ok, page} or a final {:error, error}.
Uploads body (a binary) in one request. input accepts model members such as
"ContentType", "Metadata", "IfMatch" and "IfNoneMatch".
Uploads one part (a binary). Returns %{etag: ..., ...} for completion.
Functions
Aborts a multipart upload, discarding its uploaded parts.
@spec call(ExStorageServiceSdk.Config.t(), String.t(), map(), keyword()) :: {:ok, map()} | {:ok, map(), map()} | {:error, ExStorageServiceSdk.Error.t()}
Runs any modeled S3 operation.
input uses the model's member names ("Bucket", "Key", "IfMatch", ...).
Pass return_response: true in opts to receive
{:ok, result, %{status: ..., headers: ..., request_id: ...}}.
Completes a multipart upload. parts is a list of {part_number, etag}; it is
sorted by part number before sending.
Copies input["CopySource"] to bucket/key.
Prefer the structured source %{"Bucket" => b, "Key" => k} (optionally
"VersionId"): the SDK encodes it, so any exact key is safe. A string source is
also accepted but is ambiguous for keys that contain ?versionId=. Input also
accepts "CopySourceIfMatch", "CopySourceIfNoneMatch", "MetadataDirective",
"TaggingDirective" and the other modeled copy members.
Starts a multipart upload and returns its result map, including "UploadId".
Deletes an object. Deleting a missing key succeeds, as in S3.
@spec get_object( ExStorageServiceSdk.Config.t(), String.t(), String.t(), map(), keyword() ) :: {:ok, map()} | {:error, ExStorageServiceSdk.Error.t()}
Reads an object into memory.
Returns %{body: binary, status: 200 | 206, etag: ..., headers: ..., result: map}.
A "Range" input yields status 206. Bodies larger than :max_body_bytes
(default 8 MiB) fail; use get_object_stream/5 for large objects.
@spec get_object_stream( ExStorageServiceSdk.Config.t(), String.t(), String.t(), map(), keyword() ) :: {:ok, ExStorageServiceSdk.S3.Response.t()} | {:error, ExStorageServiceSdk.Error.t()}
Starts a GET and returns the open ExStorageServiceSdk.S3.Response with its
body still on the wire. Consume it with
config.transport.consume(response, acc, reducer, opts) (which always closes
it), or release it with config.transport.close(response).
Reads object metadata. Returns %{status, etag, headers, result}. A missing object
fails with %Error{http_status: 404}.
Lists one page of keys.
Exact keys are preserved: unless the caller sets "EncodingType", the request
asks for EncodingType=url and the SDK decodes keys and prefixes in the result,
so Unicode, spaces, + and % round-trip. A caller that sets "EncodingType"
receives the server's encoded values unchanged. Pass the returned
"NextContinuationToken" as "ContinuationToken", or use pages/4.
@spec new(keyword()) :: {:ok, ExStorageServiceSdk.Config.t()} | {:error, ExStorageServiceSdk.Error.t()}
Builds a client config.
Options
:endpoint- required HTTP(S) base URL, for example"https://minio.local:9000".:region- signing region. Defaults to"us-east-1".:access_key_id,:secret_access_key,:session_token- static credentials.:credential_provider- zero-arity function returning{:ok, %ExStorageServiceSdk.Credentials{}}, called before each signed request. Use it instead of static credentials for rotating keys.:unsigned- send anonymous requests. Defaults tofalse.:path_style- use path-style addressing (/bucket/key). Defaults totrue; setfalsefor virtual-hosted buckets (bucket.host/key).:tls-:sslclient options. Defaults to[verify: :verify_peer].:timeout,:connect_timeout,:read_idle_timeout,:write_idle_timeout- milliseconds.:transport- module implementingExStorageServiceSdk.Transport.:options- map of protocol options, for example%{request_headers: [{"x-custom", "1"}]}.
@spec pages(ExStorageServiceSdk.Config.t(), String.t(), map(), keyword()) :: Enumerable.t()
Lazily streams result pages of a paginated operation, following continuation
tokens. Each element is {:ok, page} or a final {:error, error}.
ExStorageServiceSdk.pages(s3, "ListObjectsV2", %{"Bucket" => "b"})
|> Enum.flat_map(fn {:ok, page} -> page["Contents"] || [] end)Uploads body (a binary) in one request. input accepts model members such as
"ContentType", "Metadata", "IfMatch" and "IfNoneMatch".
Uploads one part (a binary). Returns %{etag: ..., ...} for completion.