JSON Lines (JSONL)

JSONL is a log that happens to be stored in a file: one JSON value per line, appended forever, read from either end. runtime:serialization treats it that way — there is no whole-document parse, only JSONL.DecoderStream and JSONL.EncoderStream, so a file larger than memory costs a line at a time.

Reading

JSONL.DecoderStream is a web TransformStream: it takes text chunks, splits them on newlines, and emits each complete line as a parsed JavaScript value.

JavaScript

import { JSONL } from "runtime:serialization";
import { file } from "runtime:fs";

// Read a massive JSONL file natively from the file system
const decoder = new JSONL.DecoderStream({
  skipInvalid: true // Continue processing even if lines are corrupt
});

// Optionally log skipped/corrupted lines without crashing the pipeline
decoder.onError(err => {
  console.warn(`Corrupt JSONL at line ${err.line}: ${err.raw}`);
  console.error(err.cause);
});

const stream = file("users.jsonl")
  .stream()
  .pipeThrough(decoder);

for await (const user of stream) {
  // Process each parsed JavaScript object sequentially
  console.log("Loaded record:", user.id);
}

onError reports only what skipInvalid skipped. Without it a malformed line errors the stream — a SyntaxError naming the line number — and the callback never fires, which is the right default for a file that is supposed to be well-formed.

Writing

Writing is handled by piping JavaScript objects through the JSONL.EncoderStream. This is typically used to append records continuously to a log.

Example 1: an access log

This is probably the most common production use case. You can continuously append events to a log file as they occur.

JavaScript

import { JSONL } from "runtime:serialization";
import { serve } from "runtime:http";
import { file } from "runtime:fs";

const log = new JSONL.EncoderStream();

log.pipeTo(file("access.log").writable({ append: true }));

serve(async (request) => {
  await log.write({
    timestamp: Date.now(),
    method: request.method,
    path: new URL(request.url).pathname,
  });
  return new Response("ok");
});

write() resolves once the record is queued; close() flushes the last line and closes the destination.

Example 2: an API response

Suppose you're collecting GitHub events to back up to a persistent log.

JavaScript

const response = await fetch("https://api.github.com/users/octocat/events");
const events = await response.json();

for (const event of events) {
  await log.write({
    id: event.id,
    type: event.type,
    createdAt: event.created_at,
  });
}

Example 3: a database export

Very common for database exports, analytics backups, and AI training dataset generation.

JavaScript

const users = await db.query(`
  SELECT id, email, created_at
  FROM users
`);

// `Rows` is async-iterable and streams a batch at a time, so a table larger
// than memory exports at the cost of a batch.
for await (const user of users) {
  await log.write(user);
}

await log.close();