Internals: S3

How @opentf/esrun-s3 is put together, and why. The decision and what was rejected are in docs/DECISIONS.md (D211).

A package over fetch

S3 is HTTP, so the client is JavaScript over fetch. It inherits the transport the runtime already has: the process trust store and --tls-ca, mutual TLS, --proxy, HTTP/2 and connection pooling. Reaching the endpoint is --allow-net, scoped to its host. The package adds no capability.

The AWS SDK does not run here. On a server it sends through node:http, and esrun does not provide node: builtins.

The bytes stored are the bytes read

S3 returns the Content-Encoding an object was stored with on every GET, whatever the request accepted. A fetch that decodes by that header would hand back different bytes from the ones stored: they would not match the object's size or ETag, and a range of them could not be decoded at all. Every read therefore passes decompress: false (D210), and the body arrives as stored.

Signing

Requests are signed with SigV4 in the Authorization header. Two places S3 differs from the generic algorithm are handled before the signer runs:

• The path is encoded once, with / kept. Other AWS services encode it twice.

• The path is never normalized. a//b and a/b are different keys.

The signing key changes once a day per region, so the last one is kept rather than paying four HMACs on every request.

The signer is checked against the AWS signing test suite (from awslabs/aws-c-auth, pinned and committed under packages/s3/test/fixtures). The unit tests also run against a fake S3 that re-derives every signature from the request it received: method, raw path, query, signed headers and the body's hash. A request signed one way and sent another fails there, as it would against S3.

Every body is hashed

A body is held in memory before it is sent, and signed with its SHA-256. Only a presigned URL is signed as UNSIGNED-PAYLOAD, because its body is not known when it is signed.

That is what makes every request retryable: a body already in memory can be sent again. It is also what bounds a multipart upload. One part is filled while at most queueSize are in flight, so an upload of any size holds at most partSize × (queueSize + 1) bytes: 30 MiB at the defaults.

S3's streaming signature (aws-chunked) would avoid buffering a part, but a part that has already streamed past cannot be retried, and a part is 5 MiB by default.

A body that ends before the first part fills is one PUT. Nothing is created on the server until a part is full, so a small upload costs one request, not three.

What is refused before sending

Listing exact keys

S3 lists keys in XML, and XML cannot carry some characters a key may contain. The XML reader also trims whitespace at the edges of text, which would change a key such as " draft ". So every list asks for encoding-type=url and decodes the keys it gets back. A key with spaces at both ends lists as itself.

Large copies

AWS copies at most 5 GB in one CopyObject; a larger copy is a multipart upload whose parts are ranges of the source (UploadPartCopy). Providers differ here: Google Cloud Storage copies up to 5 TiB in one request and does not implement UploadPartCopy, and Cloudflare R2 implements UploadPartCopy but not x-amz-copy-source-if-match on it. So copy() always sends the single copy first, and turns to ranges only when it is refused with a 400 and a HEAD shows the source is over 5 GiB. It keys on the size rather than the refusal's wording, which is each provider's own.

The parts must all come from one source. A version id pins them where the bucket has versions. Where it does not, the ETag read before the first part is read again before completing, and a mismatch aborts the copy. That catches a source replaced during the copy on every provider, R2 included. It cannot catch one replaced and then restored between the two reads.

Reads resume

A request is retried before its body starts. A body that breaks after that, by a reset connection or an answer that ends short of its Content-Length, is resumed: the client asks for the rest with Range: bytes=<next>-<last> and If-Match on the ETag of the first answer, and checks that the 206 starts at the byte it asked for. Offsets are absolute, taken from Content-Range, so a suffix slice (slice(-n)) resumes as exactly as a whole object. The same retry budget applies again to the body.

If-Match is what makes a resume safe. Without it, an object overwritten between two pieces would come back as the start of one version and the end of another, with nothing to show for it. With it, the read fails with PreconditionFailed. An answer with no ETag is not resumed, since nothing would tie the rest to it.

Retries

A part that still fails after its retries fails the upload, and the multipart upload is aborted so no parts are left on the server. If the abort also fails, the original error is what the caller sees, and a bucket lifecycle rule (AbortIncompleteMultipartUpload) cleans up.

Credentials

Credentials are given, or produced by a function. Expiring credentials are refreshed five minutes before they lapse, and concurrent requests share one refresh. A credential may be a Secret from runtime:process's env. It is unmasked only when a request is signed, and runtime:process is loaded only then, so a program that passes plain strings needs no env grant.

Providers

The providers speak AWS's protocols as the AWS SDK for JavaScript implements them, with two rules of their own.

• Instance metadata is IMDSv2 only. The SDK falls back to IMDSv1 when the token request fails. IMDSv1 answers any request without a token, which is how a server-side request forgery reads an instance's credentials. A refused token is reported instead, since it usually means a container cannot cross the instance's hop limit.

• A token never travels in plain text. A credentials URL must be HTTPS, loopback, or one of AWS's link-local addresses (169.254.169.254, 169.254.170.2, 169.254.170.23, [fd00:ec2::23]), the rule the SDK applies to container endpoints, here applied to STS and instance metadata too. A container token containing a line break is refused, since it would become a second header.

None is chosen automatically. A chain that tries each in turn would decide where a program's credentials come from by what happens to answer first.

Testing