Internals: sockets

How runtime:net gets bytes on and off a TCP socket, what each socket costs while it is open, where the sharp edges are — and why the UDP half of the module is built the opposite way.

This page explains behaviour rather than listing signatures — for those, see the runtime:net reference — and it covers raw sockets only. The inbound HTTP server has its own page; it does not share this code, only the reasoning behind parts of it.

Why every socket owns two tasks

The obvious implementation of read() is to poll the socket from inside the op and return when bytes arrive. That does not work here.

Ops are driven by the JavaScript event loop, and a read that has to wait for the peer would be polled only when the loop happened to come round again — which, if the loop is idle waiting for exactly that read, is never. The socket has to be driven by the reactor, not by the isolate.

So each socket is split in half and handed to two spawned tasks:

    guest read()  ←── mpsc(8) ←──  reader task ──→ AsyncRead half
    guest write() ──→ mpsc(8) ──→  writer task ──→ AsyncWrite half

The ops become channel operations, which are cheap and always make progress, and the actual I/O runs on the runtime's reactor threads. It is the same shape the HTTP client uses, for the same reason.

Each direction holds 8 chunks of backpressure. A reader whose channel fills stops reading the socket, so the pressure reaches the peer through TCP's own window rather than through a growing buffer here. The reader's own buffer is 64KB per socket, which — with the two channels — is the floor of what an idle open socket costs.

The consequences worth knowing

A read checks the receiver out, and gets it back either way. It takes the receiver out of the registry for the duration of the wait, because the registry's lock cannot be held across an await. The put-back is a destructor rather than a code path, so it happens whether the read completes, is abandoned by the caller, or panics — an embedder can wrap a read in tokio::time::timeout or race it in a select! without silently killing the socket. The same guard covers accept, and the equivalents in the HTTP and WebSocket providers.

EOF is the reader task ending. It drops its sender, recv() yields None, and the read resolves to null. There is no separate end-of-stream flag to keep in sync.

shutdown() is dropping a sender. Clearing the write channel's sender ends the writer task's recv(), which shuts the write half down — a FIN to the peer, with the read half still live. That is half-close, and it falls out of ownership rather than needing a state machine.

close() is dropping the slot. Both channel ends go, both tasks end.

TLS

connect({ secureTransport: "on" }) completes the handshake before the socket is handed back, so a guest never sees a half-open TLS socket. The address information is read off the raw TCP stream before the handshake consumes it.

Trust anchors are the bundled Mozilla set (webpki-roots), built once per process and shared. That means no platform certificate store is read and no I/O is done to find one — a run is portable and deterministic in a way a system-store lookup would not be. The trade is that a corporate root installed on the machine is not trusted; an embedder overrides the store, a CLI user cannot yet.

Two details that exist for reasons worth stating:

The crypto provider is named explicitly. Both ring and aws-lc-rs end up linked in this binary, which makes rustls's process-default provider ambiguous — ClientConfig::builder() would panic rather than pick. aws-lc-rs is selected by name at every construction site.

Client configs are memoized by ALPN list. Building one re-parses the entire root store, which is far too expensive to do per connection. The offered ALPN set is the only per-connect input to the config, so it is the cache key; a hit is an Arc refcount bump.

Server-side TLS

listen({ secureTransport: "on", cert, key }) terminates TLS on accept. The certificate and key are parsed once, at bind time — an unusable certificate fails listen() rather than every later handshake, because a port that accepts connections and then rejects all of them looks like a working server that nothing can reach.

Handshakes for accepted connections run inside the accept task, in a FuturesUnordered, rather than in spawned tasks holding channel senders. Two things fall out of that: a slow handshake cannot head-of-line-block the next accept, and no handshake in progress can keep the accept channel alive past a close.

A handshake that fails ends that connection and nothing else — the guest never sees it, because it never became a socket. It is logged at debug on the runtime::net target with its peer and the reason:

RUST_LOG=runtime::net=debug esrun server.js

debug rather than warn because any peer can produce one on demand, and a level people run with would let a scanner set your log volume. Accept errors stay at warn; those are the listening socket's problem, not a client's. Worth turning on when a TLS listener is accepting connections and serving nothing: that is what a certificate no client will accept looks like from the server side, and it is otherwise indistinguishable from a port nobody is calling.

Upgrading in place: startTls

secureTransport: "starttls" opens plaintext and upgrades later — the SMTP and PostgreSQL shape, where the protocol negotiates in the clear and then switches.

This is harder than it looks, because the reader and writer tasks now own the two halves of the stream and TLS needs them back. Each task therefore parks on a reclaim channel alongside its normal work, with the reclaim branch biased to win:

1. startTls sends each task a one-shot sender.

2. The reader stops and hands its half back. A cancelled read loses nothing — no bytes were consumed.

3. The writer flushes what is still queued before handing its half back, so nothing the guest wrote before the upgrade is lost.

4. The halves are rejoined and wrapped in a TLS connector.

There is one more subtlety with real consequences. Between the peer's go-ahead and the handshake, the peer may already have sent handshake bytes, and the reader task may have buffered them without the guest ever reading them. Those bytes are drained out of the read channel and replayed ahead of the live socket, so the TLS handshake sees the stream the peer actually sent. Dropping them would produce a handshake failure that looks like a broken peer.

Only a plaintext client socket carries reclaim handles. An accepted socket or an already-TLS socket has none, and asking to upgrade one is an error rather than a silent no-op.

Listening

The accept loop is the same design as the HTTP server's, for the same reason: an error from accept is never fatal. ECONNABORTED, EMFILE/ENFILE and EINTR are ordinary on a busy public port and say nothing about the listening socket, so every error is retried behind a delay that doubles from 5ms to 1s and resets on the next accepted connection. See the HTTP page for the full reasoning.

One task owns the sole channel sender, which is what makes listener.close() work: aborting the task drops the sender, so an accept already parked on recv() resolves to None instead of waiting forever. Removing the receiver from the registry is not enough on its own — a parked accept has already checked it out.

The accept channel holds 8 connections. Unlike the HTTP server, there is no maxConnections: a runtime:net listener accepts as fast as the guest calls accept(), and the guest's own loop is the backpressure.

UDP: no tasks, no channels

bind() is the other half of the module and it is built the opposite way. There are no reader and writer tasks, no mpsc in the middle: a datagram socket is held as a shared UdpSocket, and each receive() awaits recv_from on it directly.

The task-per-socket design exists because a TCP stream delivers bytes whether or not anyone is asking, so something has to be reading. A datagram socket already has that buffer — it is the kernel's receive queue — and a second queue in front of it would only add a place for datagrams to be dropped that the program cannot see. So there is exactly one queue, and it is the one the OS bounds.

That choice costs one thing, and it is worth stating because it is the only non-obvious piece of machinery here: a parked receive() holds the socket alive itself, so dropping the registry's handle is not enough to end it. close() therefore sets a flag and rings a Notify, and a receive registers for the bell before it reads the flag. A close landing between those two steps is then seen by one or the other, and never by neither. Without that ordering, a receive on a socket nobody is sending to would wait forever for a datagram that can no longer arrive.

Two more things fall out of the shape:

Message boundaries are the API. One receive() is one datagram, whatever its length — including a zero-length one, which is a message and not an end of stream. A datagram longer than 65,507 bytes is truncated by the OS exactly as it is for any other receiver, and nothing here reports how much was lost.

Concurrent receives race, and that is correct. Several outstanding receive() calls share the socket and whichever is polled first takes the next datagram. There is no checked-out receiver to lose, because there is no channel.

What a datagram costs

Release build, five runtimes, from bench/run.sh — udp_echo is 10 000 request/response round trips over loopback with 64-byte payloads, udp_send is 50 000 fire-and-forget datagrams of 512 bytes. Reproduce with WORKLOADS="udp_echo udp_send" bench/run.sh; the charts are on Benchmarks.

Two different results, and the difference between them is the interesting part.

Sending is cheap — 4.0 µs per datagram, second of the five and ahead of both Node and Deno. A send is one op crossing and one send_to, with nothing between the guest and the syscall.

A round trip costs more, and it is our own shape that costs it: four op crossings and two loop wakeups per exchange, with a promise resolved for every one of them. Node delivers a received datagram to a callback with no promise involved, which is most of the 2 µs between us; Bun's socket is native down to the same callback.

Pooling the receive buffers took that row from 34.9 µs to 30.9 — the 64 KiB allocation per datagram was worth more than a tenth of a round trip. What is left is the promise-per-datagram shape, and it is a design consequence rather than a defect: receive() returning a promise is what makes datagram handling composable with await, timeouts and Promise.race, which a callback does not give without adapting it back. receiveMany does not close it — a strict request/response exchange has only ever one datagram to take — so closing it further would mean making the op crossing itself cheaper, which is a change to the runtime rather than to this module.

Receive buffers are pooled

A datagram socket receives small datagrams by the thousand, and a 64 KiB allocation per datagram was most of the cost of receiving one. Buffers are borrowed from a small per-socket pool for the length of one receive and returned by a destructor — so an abandoned receive gives its buffer back too — while the datagram that leaves is a fresh, exactly-sized copy. The pool keeps four; a burst of concurrent receives may take more, and the surplus is dropped rather than retained.

The buffer is deliberately one byte longer than the largest datagram IPv4 can deliver. That is what makes truncation observable: a datagram that fills the buffer exactly is one that did not fit, because no real IPv4 datagram is that long. It surfaces as truncated: true rather than being fixed — the rest of the message is gone at the OS, and a program parsing a prefix as a whole message is the failure worth naming.

What keeps the loop alive

A pending receive() is not what keeps the process running; a counter is. The receive op is registered unref'd, and a per-agent counter — shared with workers, since "referenced host handles" is one question — is incremented while a referenced socket has a receive in flight.

The split exists because a parked receive cannot be taken back. If the receive itself were the reason the loop runs, unref() would not take effect until a datagram arrived — and on the socket a program is unref'ing, none ever does. The counter is asked afresh every time the loop wonders whether to stop, so the answer changes immediately. It is the same design worker_recv uses, for the same reason.

One consequence differs from Node: a bound socket with nothing in flight is already not a reason to stay alive. That matches listen() here, and it is the answer that hangs less rather than more.

The v4/v6 split

ttl, multicastTtl and multicastLoopback are carried by different socket options on IPv4 and IPv6, and setting the wrong one is not an error the OS reports — it silently does nothing. The address family therefore decides, once, at the bind. Asking for broadcast on an IPv6 socket is an error rather than a flag that sets nothing, because IPv6 has no broadcast at all, and an IPv6 multicast membership names its interface by index while an IPv4 one names it by address.

Options are set at the bind, not later: SO_REUSEPORT and SO_REUSEADDR are meaningless after it, and the rest are configuration a program decides once. Group membership is the exception, and is a method, because a responder joins and leaves groups while it runs.

Capabilities

Reaching out and being reachable are separate privileges, so they are separate capabilities with separate allowlists:

The check happens before the socket is created — a denied listen claims no port and leaves nothing behind.

A UDP socket is checked against both. It is a server and a client at once, so gating it on one grant would be a hole in whichever was chosen: under Net alone a program could bind a port and receive inbound traffic, and under NetListen alone it could reach any host on the internet. The consequence is that a program which only sends still needs listen — an ephemeral source port is a port, and the replies that come back to it are inbound. And because the destination is an argument rather than a property of the socket, --allow-net is consulted on every send, not once at the bind.

A host is judged as written, before resolution. That is deliberate: a name is a name, so a denied name cannot be smuggled past the list by an attacker-controlled DNS answer, and an allowed IP never silently admits a name that happens to resolve to it. It also means allowing example.com does not allow the address it resolves to, if the guest connects by address.

What is not bounded

Worth knowing before putting a runtime:net service on a public port:

• No connect timeout, and no way to cancel one. connect() waits as long as the OS does, which on a dropped SYN is around two minutes on Linux. There is no signal option, and close() on a socket whose connect is still pending awaits that connect rather than aborting it. A guest can stop waiting — Promise.race([socket.opened, timeout]) — and carry on, but the attempt runs to the OS timeout underneath.

• No read, write or idle timeout. A socket that goes quiet stays open until a peer or the guest closes it.

• No TLS handshake timeout on connect. The server side of listen has none either — unlike the HTTP server, which bounds it at 10s.

• No connection cap on a listener, and no per-peer limit.

• No socket count limit. The descriptor limit is the ceiling.

• Nothing bounds a UDP socket's queue but the OS. The kernel's receive buffer is the only backpressure; a guest that stops calling receive() has datagrams dropped by the OS, silently, because that is what UDP is.

None of these are oversights in the sense of being unnoticed; they are the gap between a raw socket API and a server framework. Most are things a guest can impose itself by racing a timer against socket.opened or a read — the exception is the connect attempt, which cannot be cancelled from guest code at all. The HTTP server bounds all of this because a guest there never sees the connection to bound it.

See also

• runtime:net API reference — signatures, options, errors

• Sockets guide — how to use it

• Internals: the HTTP server — the inbound server's own path

• Internals: the fetch client — the outbound HTTP direction

• Internals: WebSockets — the same actor pattern, framed

• Security model — how capabilities are granted and denied