Storage #

View this page as Markdown

On this page

Storage gives your application named file areas, called storage areas, on Turn Zero Cloud. The service writes, reads, lists, and deletes files, and supports conditional writes. It keeps each account's and each environment's files apart. Store files gives the task step by step.

The client package, @turnzero/storage, comes from turnzero library take --entry storage (turnzero.cmd in Windows PowerShell), not the npm registry. The command downloads and installs a local copy; Use the library explains the required CLI credential and where the files go.

If your backend does not use the package, it can call the file routes itself with its platform credential. Call storage without end-user sign-in shows the complete fetch example.

Using storage takes four parts, in this order. A minimal upload service joins all four in one service you can copy.

  1. Add {"kind": "object_storage"} to the manifest's services list. This declares that the application uses storage; it creates no area unless its areas list names one. For example, { "kind": "object_storage", "areas": [{ "name": "receipts", "account_keyed": true }] } declares the receipts area with account_keyed set to true.
  2. Run turnzero library take --entry storage. It downloads the package, creates the application's copy in app/lib/storage/ or lib/storage/, and runs npm install. Include that copy in the deployment.
  3. Build the client over a transport that adds the platform credential. Before using an area, declare it in the manifest's areas list, with the client's mintArea at backend startup, or with declare_storage_area. An area's declarations are fixed by its first declaration.
  4. Use the client to read and write files. The example calls mintArea at each start with the same declarations, including the application. A differing declaration is refused as differing_redeclaration, which the client reports as declarations_differ.

Use it when #

Use Storage for documents, uploaded images, and other files. Your backend controls each end user's access and chooses the file names, because the service does not separate files by end user. Files are separate per environment: one declared area has a partition for each environment, so the same name can refer to a different file in development and in production.

To serve a new or replaced file without redeploying, have your backend read it with get(area, name) and return the bytes, or mint a download grant for the browser. Each backend request to read the file or mint the grant counts as a backend action and a storage call. Data transfer counts the file bytes at both the storage read and the backend response; a browser download under a grant counts them once.

Keep any file your application needs again in an area. What your code writes to the running copy's own disk is lost whenever the copy is replaced or stops, as at a deploy, a restart, or an idle stop.

A small JSON value or list that the application reads and replaces whole can live in Storage. Use a conditional write with the version read, or require an unused name for its first write. A small key-value guestbook shows this pattern.

For records you search, sort, transact on, or change individually from concurrent requests, use a Database table. Keep key-value records gives the scaffold command for a key column and a value column. Turn Zero Cloud has no separate key-value storage service.

A shared database is one monthly item for the application; stored files are charged by size (Stay inside the Free Allowance).

What it provides #

  • Named areas with fixed declarations, each bound to one application of the account.
  • Reads, paged listings, deletes, and writes that include an acting identity, the X-Acting-Identity header Store files describes.
  • Conditional writes, which require a version you read earlier or require the name to be unused.
  • A client, StorageWireClient, which The client describes. A deployed transport reads the storage origin from TURNZERO_CLOUD_GATEWAY_URL, or TURNZERO_CLOUD_API where the platform sets no separate gateway origin, and the credential from TURNZERO_CLOUD_APP_TOKEN. Both are among the settings a deployed copy receives (A deployed app's settings).
  • Short-lived transfer grants that let a browser upload or download one file, each recording the environment it was issued for.
  • Configurable request and page limits, with named refusals.
  • A test double, createStorageDouble at @turnzero/storage/testing, which takes the place of the service behind the unchanged client. Test with the double shows it in use.

Every call reaches one environment's files in the area:

  • A call under your application's platform credential reaches the environment the credential is bound to. The platform refuses a header that gives another environment with 403 environment_mismatch.
  • A call under a session or a minted token gives the environment in the header x-turnzero-cloud-environment. A file route, a list, or a grant request without it is refused with 400 environment_required. A script that uploads a deploy artifact under a minted token sets it to the environment the deploy goes to: production on an application with one environment, development on one with two.
  • An upload under a short-lived grant, from mint_upload_grant or the grant the turnzero command receives for a deploy, sends no such header, because the grant fixes the environment when it is minted. The command uploads a deploy's zip this way.

The platform's routes do not keep earlier versions of a file, separate files by end user automatically, or give access levels on a shared area. The version a write returns supports concurrency checks, and the platform does not keep the older version of the file. Store files describes the current routes, permissions, and grant options.

Storage keeps no fields of your own on a file: keep a caption or original file name in a Database row keyed by the file's name, which can change without rewriting the file. Storage keeps the acting identity your backend passes on each write as the file's actor, which records who wrote it.

Availability #

Version 0.23.2 is the package's current version, and this page describes it. Until the library's next publish, read_library can return an earlier version, the one you can take, so this page can describe a change that version lacks. The entry includes the testing subpath.

Every area is bound to one application of the account. Under your application's platform credential, a declaration that leaves out the application member binds that application. A declaration under a session or an account-scoped token must include application, or the platform refuses it with application_required.

The platform declares each area that the manifest's object_storage entry lists at each submission. It binds the area to the application and keeps no versions. The storage routes and declare_storage_area work for a declared area with or without the entry.

The library entry contains the package's product statement and detailed contract, its integration guide, and the compiled client. Turn Zero Cloud's storage routes are live.

Account supplies the end-user identities your backend can use for access checks and attribution. Database stores structured records beside the files.

The client #

A minimal upload service shows the client in a complete service, and a test runs it.

Construction #

The constructor new StorageWireClient(transport) builds the client. Your transport, transport(path, init), adds the storage origin and your credential to each request and returns the response; the path starts /storage/v0/. The package ships no transport and reads no setting itself. The deployed transport is a few lines over fetch, which A minimal upload service gives in src/storage.mjs. For a transport under a session or a minted token, the helper withEnvironment(transport, environment) wraps it so it sends x-turnzero-cloud-environment on every request except the area's declaration.

Methods and results #

  • The method mintArea(area, { accountKeyed, versionKeeping, application }) declares the area, or declares it again with the same declarations, and returns { minted }. Set versionKeeping to false, the one value the platform accepts. Leave out application under the platform credential, which binds its own.
  • The method put(area, name, bytes, { identity, contentType, ifVersion, mustNotExist }) returns the new { version }. The option mustNotExist requires an unused name, and ifVersion requires the stored version to match. When mustNotExist is true, ifVersion is ignored.
  • Every put and delete_ requires identity, including a write made for a visitor who is not signed in. Pass a nonempty string of your own that says who acted, such as anonymous for a public upload form. The string grants no access.
  • The method get(area, name, { ifNoneMatch }) returns { bytes, meta }, or { notModified: true, meta } where ifNoneMatch gives the current version.
  • The method getMetadata(area, name) returns the file's meta: version, size, contentType, created, lastWritten, and identity, the acting identity that wrote it.
  • The method delete_(area, name, { identity }) deletes the file. Deleting a missing name succeeds.
  • The method list(area, { prefix, cursor, limit }) returns { names, cursor }. Despite its name, names contains one object per file, with its name, size, version, contentType, created, lastWritten, and identity, using the same field names as getMetadata. The cursor member is the cursor for the following page, passed back as the cursor option, or null on the last. The result also carries next, the same value under its earlier name.

The client takes and returns file bytes as Uint8Array.

Failures #

The client throws an Error whose code is one of these client codes. They can differ from the platform's error names:

Code When
no_identity A put or delete_ was called without identity. The client refuses it before it calls the platform, with a message that starts with unattributed write. Left uncaught in a request handler, it becomes your application's own 500.
invalid_name A file name contains a path part equal to . or .., separated by / or \, or an area name is exactly . or ... A put also rejects a backslash anywhere in the file name and a dot at the end of any part, including the last. These checks run before the client calls the platform, which may otherwise store the file under a different name. Reads and deletion still send these names, so previously stored files remain accessible.
exists, version_moved A conditional write found the name already in use (exists for mustNotExist) or a different stored version (version_moved for ifVersion): HTTP 412, with status 412, refusal version_mismatch, and the stored version in version. An ifVersion write to a missing file also throws version_moved, with status 404, refusal no_such_file, and version null.
no_area, no_file The area is not declared, or the area holds no file of that name.
declarations_differ The area was declared again with other declarations.
mint_failed, put_failed, get_failed, meta_failed, delete_failed, list_failed The platform refused the call for another reason. The message begins with the HTTP status and a colon. It then uses detail, or the refusal name if detail is absent. If the response is not JSON or has neither field, it uses fixed fallback text. Handle the client's own code rather than matching its message.

An error built from a service response carries status, the HTTP status. If the response body's error is a string, refusal carries that name. Handle code, or inspect refusal when one client code covers several platform refusals; do not match the message text.

A small key-value guestbook #

Use this for a small guestbook whose messages are read and replaced together. The key is messages.json in the guestbook area; its value is one JSON list.

At backend startup, build a StorageWireClient as Construction shows, then pass it to openGuestbook below. Use your application's platform credential; the area declaration binds to that application.

Save this as guestbook.mjs. Every list reads the stored file, so a replacement backend reads the messages already stored there.

// guestbook.mjs
export async function openGuestbook(storage) {
  const area = 'guestbook';
  const name = 'messages.json';
  await storage.mintArea(area, { accountKeyed: false, versionKeeping: false });
  const encoder = new TextEncoder();
  const decoder = new TextDecoder('utf-8', { fatal: true });

  async function snapshot() {
    let file;
    try {
      file = await storage.get(area, name);
    } catch (error) {
      if (error.code === 'no_file') return { messages: [], version: null };
      throw error;
    }
    const messages = JSON.parse(decoder.decode(file.bytes));
    if (!Array.isArray(messages) || messages.some((value) => typeof value !== 'string')) {
      throw new Error('Stored guestbook is not a list of messages.');
    }
    if (!file.meta.version) throw new Error('Stored guestbook has no version.');
    return { messages, version: file.meta.version };
  }

  return {
    async list() {
      return (await snapshot()).messages;
    },
    async add(message, identity) {
      if (typeof message !== 'string' || !message.trim()) {
        throw new TypeError('A message must be nonempty text.');
      }
      for (let attempt = 0; attempt < 5; attempt += 1) {
        const { messages, version } = await snapshot();
        try {
          await storage.put(area, name, encoder.encode(JSON.stringify([...messages, message])), {
            identity,
            contentType: 'application/json',
            ...(version === null ? { mustNotExist: true } : { ifVersion: version }),
          });
          return;
        } catch (error) {
          if (error.code !== 'exists' && error.code !== 'version_moved') throw error;
        }
      }
      throw Object.assign(new Error('Guestbook is busy; try again.'), { code: 'guestbook_busy' });
    },
  };
}

After await openGuestbook(client), keep the returned helper for your routes. A read route calls list(); a write route calls add(message, identity) after checking the visitor's access. Choose the acting identity in your backend, such as anonymous for an open guestbook. Render each message as text, never HTML.

The first write uses mustNotExist. If another request creates the file first, exists makes the helper read it again. A replacement uses the version just read as ifVersion. On version_moved, it reads again and appends to the newer list, rather than overwriting another request's message. Deletion between the read and write takes the same retry.

After five conflicting writes, add throws guestbook_busy; ask the visitor to try again. Other storage failures are returned to your route unchanged. Invalid stored JSON is an error, never an empty guestbook to overwrite.

Each update reads and writes the whole list. Keep it small; use a Database table when you need to search messages, page through them, or update records separately.

Each file in a listing includes its created and last-written times and the acting identity that wrote it, so a listing alone dates every file. A listing is still in name order.

The client reads one file per call. To read every file under a prefix, call list for each page, following cursor, and then get once for each name. Records stored one per file, such as a guestbook's messages, fit better in a Database table, where one query reads them.

For a gallery or a feed that shows each file's caption or prompt, newest first, keep one Database row for each file. The row contains the file's name, the caption or prompt, and the created time:

CREATE TABLE gallery (
  file_name text PRIMARY KEY,
  prompt text NOT NULL,
  created_at timestamptz NOT NULL DEFAULT now()
);

One query reads a page, newest first with ORDER BY, and within a time window with WHERE:

SELECT file_name, prompt, created_at
FROM gallery
WHERE created_at >= $1 AND created_at < $2
ORDER BY created_at DESC
LIMIT 24;

Write and delete in this order, so a page never lists a file your service has not stored:

  • Put the file first, and write the row after the put returns.
  • Delete the row first, and then the file.
  • For a file a browser uploads through an upload grant, write the row when your backend mints the grant. Until the upload is made, a get through the client throws no_file, and a download through a grant returns 404 no_such_file, so show the file as pending or skip it.

Each image still costs one storage call: a get through your service, or the mint of a download grant. The row saves the extra read that a caption kept in a file of its own would cost. It does not save the image reads.

Without a Database row, your application can name each file so that the listing's name order is newest first. Keep the files under a prefix of their own, such as gallery/. Each name starts with a countdown: a fixed instant after every write, such as 9999999999999 milliseconds, less the write's time, zero-padded to 13 digits. A unique suffix of ASCII characters follows it, because two writes can share a millisecond:

// The name a gallery image is stored under, newest first in a listing.
const END = 9999999999999; // an instant, in milliseconds, after every write
const galleryName = (now = Date.now()) =>
  `gallery/${String(END - now).padStart(13, '0')}-${crypto.randomUUID()}.png`;

A list with the prefix gallery/ then lists the newest file first. Keep the width fixed: names are compared character by character, so a shorter countdown would sort out of time order.

Test with the double #

The factory createStorageDouble() from @turnzero/storage/testing keeps areas and files in memory behind the real client. Build the client over double.transport, or stub the global fetch with double.fetch where your own transport wraps it. The double records each call in calls and each file in files. A declaration that leaves out application binds the application set by the double's application option, app by default, as the platform credential binds its own. With application: null, the double acts as a call under a session or an account-scoped token, and such a declaration is refused with application_required.

The double ships as lib/testing.js beside the client in the Storage package's library entry. Your committed package copy includes it, so every clone and CI run can use it without a Turn Zero credential. A signed-in AI tool with no CLI token can read that file through read_library. Set entry to storage and file to features/storage/lib/testing.js; it imports nothing else (Use the library).

The double's reset() forgets every area, file, call, and script. Code that mints its area once per process, as a server does at its first file request, must mint it again after a reset. Otherwise a test that resets the double between cases gets undeclared_area. A test runs this sample over the double.

import assert from 'node:assert/strict';
import { StorageWireClient } from '@turnzero/storage';
import { createStorageDouble } from '@turnzero/storage/testing';

const storage = createStorageDouble();
const client = new StorageWireClient(storage.transport);
const bytes = (text) => new TextEncoder().encode(text);

await client.mintArea('notes', { accountKeyed: false, versionKeeping: false });
const { version } = await client.put('notes', 'first.txt', bytes('hello'), { identity: 'user-1', contentType: 'text/plain' });
const read = await client.get('notes', 'first.txt');
assert.equal(new TextDecoder().decode(read.bytes), 'hello');
assert.equal(read.meta.version, version);
// A write that requires the name to be unused is refused once the file exists.
await assert.rejects(client.put('notes', 'first.txt', bytes('again'), { identity: 'user-1', mustNotExist: true }), { code: 'exists' });

The client throws each refusal with its storage code in code, as Failures lists. Where a refusal is the test's subject, assert the code instead of applying the remedy. These come from the double's own setup:

Refusal Cause Remedy
mint_failed, its message starting 400: An area was declared with versionKeeping: true, or without application on a double created with application: null (application_required). Declare versionKeeping: false, and give application where the double represents a session.
no_area A call named an area not declared since reset(). Declare the area in each test.
invalid_name A file name has a part that is . or .., or a name given to put holds a backslash or a part that ends with a dot. The client checks this before the identity, and the double refuses such a name with 400 invalid_request, as the platform does. Rename the file or the area. For a write, separate the parts with a forward slash, and end no part with a dot.
put_failed, get_failed, list_failed, and the other _failed codes A scripted refusal, a body past requestBytes, or a list cursor that matches no file. Keep the body within the limit, and pass the cursor the previous page returned.
mint_failed or another _failed code, its message starting 400: The request holds a NUL character A file name, a list prefix, or another part of a storage call's address contained a NUL character (U+0000), or an area declaration contained a NUL character or an unpaired UTF-16 surrogate. The platform refuses the same text. Send the text without the character.

File routes #

The storage routes are HTTPS on the platform's origin, under /storage/v0/, and the client calls them for you. This section lists what each route returns, following the package's detailed contract.

Send the credential in Authorization: Bearer <credential>. For a file PUT, send the file's bytes directly as the body, without JSON, base64, or form encoding. Its Content-Type is stored as the file's content_type; if you omit it, the type is application/octet-stream. A file GET that returns bytes uses that type in its Content-Type header.

Storage's own refusals return error and detail, as the JSON example shows. Examples of extra fields include current_version, credential_kind and scope. The platform's credential guard can refuse the request first; its responses also carry contract_version. For a raw HTTP call, handle error in either form. The client's code follows Failures.

PUT /storage/v0/areas/{area} takes a JSON object with these members, using the JSON field names rather than the client's camelCase options:

Member Required Value
account_keyed Yes A boolean: true for files belonging to individual end users; prefix each file name with that user's ID. Use false for files shared by the whole application.
version_keeping Yes false. The platform keeps no prior file versions and refuses true.
application Under an account-wide credential The application's identifier. An application's platform credential or an application-scoped token may omit it to use its own application.

An area declaration takes no environment header. File reads, writes, listings, deletes, and grant requests reach one environment, under the credential and header rules in What it provides.

Route Returns
PUT /storage/v0/areas/{area} 201 { area, outcome: "created" } on a first declaration, and 200 with outcome: "unchanged" on the same declarations again. A differing declaration, the application included, is refused 409 differing_redeclaration. A declaration that leaves out application under an account-wide credential is refused 400 application_required. A body that is not a JSON object, or a malformed member, is refused 400 invalid_request, and version_keeping: true 400 version_keeping_unsupported.
PUT /storage/v0/areas/{area}/files/{name} 200 with the ETag header and the file's fields as JSON: name, area, size, content_type, created_at, last_written_at, actor, and version. A write without X-Acting-Identity is refused 400 identity_required. A failed condition is refused 412 version_mismatch, with current_version, and an If-Match on a missing file 404 no_such_file.
GET /storage/v0/areas/{area}/files/{name} 200 with the bytes and the headers ETag, Content-Type, X-Created-At, X-Last-Written-At, and X-Actor. With ?meta=1, the file's fields as JSON and the ETag header. 304 with the ETag header and no body where If-None-Match names the current version. A missing file is refused 404 no_such_file.
DELETE /storage/v0/areas/{area}/files/{name} 200 { deleted }, for a missing file too. A delete without X-Acting-Identity is refused 400 identity_required.
GET /storage/v0/areas/{area}/files?prefix=&cursor=&limit= Returns { files, cursor }, ordered by name and limited to the prefix. Each file has the fields above. limit is capped at the page bound; the last page has no cursor. Pass the returned cursor unchanged. It contains no account, area, or environment details. The platform checks its format and a digest of those values plus the prefix. It also checks that the cursor's filename begins with the prefix and is valid for a stored-file read, allowing names that new writes now reject. The filename is not authenticated: changing it can only move within that same listing. Any other cursor is refused 400 invalid_cursor; restart from the first page.

Every route can also return these:

Response When
400 invalid_request The path, query, or file name breaks one of the five rules below. The test double does not apply the stored-name length and segment limits.
404 undeclared_area A route below the area names an area that is not declared.
413 request_too_large The body is larger than the request bound.
404 unknown_route The path is outside the storage routes.
405 method_not_allowed The route does not serve the method.

A 400 invalid_request can have any of these causes:

  1. The path or query contains a NUL character.
  2. A file name has a . or .. part; forward slashes and backslashes both separate parts.
  3. On a write, the file name contains a backslash or a part ending in a dot.
  4. The full stored name, including the account, area, environment, and file name, exceeds 1,024 characters. The refusal's detail gives the room left. Writes count UTF-16 code units, so an emoji counts two. Reads, metadata reads, and deletes count code points, so a stored file stays reachable.
  5. The full stored name exceeds 254 slash-separated parts.

A write is conditional through one of two request headers:

Header On Effect
If-Match A file PUT The write lands only where the stored version is the one named.
If-None-Match: * A file PUT The write lands only where the name holds no file.
If-None-Match A file GET A read naming the current version returns 304 with no body.

Each write sets a new random version, so versions show no order. Read the times instead. The credential, the environment header, a reserved area name, and the stored-data quota have refusals of their own, which Store files lists.