Logging #
On this page
Logging gives your application a lasting record of what it did, without a logging stack of its own. Your application writes log entries and counters through the package's client. Turn Zero Cloud stores them and returns them when you read them. It also writes its own events about your application to the same store.
To start, take the package from the library (Use the library), and build new LogClient(transport) over a transport that adds the platform's origin and credential. Then call client.log.info(message, fields) or client.count(name) from your code, and your tool reads the records with read_logs and read_counters (Read logs and counters).
Use it when #
Use Logging when your application needs to record what happened, as a message with a level and a few named fields. Use it also to keep a count without scanning records to work it out: requests served, uploads completed, or sign-ins rejected.
Do not write end-user content into a record. A field contains only what your application deliberately puts there.
What it provides #
- Log entries and counters. An entry has a level, a message, and optional scalar fields. Counters are added up into hourly and daily totals. Write the request's
x-request-idheader in a field namedrequest_id, andread_logsreturns every entry of that request byfieldandvalue. - A client,
LogClient, with no dependencies, which sends records in batches without blocking the caller. The client describes it. - A stored stream per environment. Each batch specifies its environment. Under your application's platform credential, that environment must be the credential's environment. The one exception is
local, which only a development credential may write, so a local run writes to thelocalstream and never to production. Under a minted token the batch specifies the environment, and anx-turnzero-cloud-environmentheader, if sent, must specify the same one. - A source on every record, which tells your records from the platform's. Read logs and counters lists the sources and reads them.
- Reads and deletion through your tool. The actions
read_logsandread_countersread entries and counter totals. The actionpurge_logs, approved in the browser, removes the records for the application or one of its environments. The actiondelete_environmentremoves the development environment's records, anddelete_applicationremoves every environment's. - Size, rate, and retention limits. Each environment keeps up to 1 GB of your entries, or 5 GB with the 30-day option. The runtime harness has a separate limit of the same size. A batch past its limit is refused
log_capacity_reached, and the platform writes its own warning into the stream. Entries are kept seven days, or 30 with the paid option. Read logs and counters explains the option and its monthly charge. - A test double,
createLoggingDoubleat@turnzero/logging/testing, which keeps the entries and counters for a test to read back. Test with the double shows it in use.
Availability #
Version 0.20.2 is the package's current version, and this page describes it. Until the library's next publish, read_library can return an earlier version, the one you can take, so this page can describe a change that version lacks. The entry includes the testing subpath. The entry contains the compiled client, the package's product statement and detailed contract, and its integration guide. Turn Zero Cloud runs the logging service.
Related feature packages #
Database stores application records, and Storage stores files. Logging records application activity.
The client #
The package is published in the library as the npm package @turnzero/logging, not on the npm registry; Use the library installs it with one line.
Construction and options #
The constructor new LogClient(transport, options) builds the client. Your transport(path, init) function sends each path to TURNZERO_CLOUD_GATEWAY_URL, falling back to TURNZERO_CLOUD_API, with TURNZERO_CLOUD_APP_TOKEN as a bearer token. It sends the request and returns the response. Copy the transport from Call an AI model. The transport you write for the Storage or Egress client works with this client unchanged. Build one client per process and keep it.
| Option | Default | Meaning |
|---|---|---|
environment |
Required | The stream to write: APP_ENVIRONMENT when hosted, local for a local run. |
flushIntervalMs |
2,000 | Send the buffer this often while it contains records. |
flushAt |
200 | Send the buffer once it holds this many records. |
capacity |
2,000 | The most records the buffer holds; past it, the oldest are dropped. |
maxNames |
1,000 | The most counter names the client accepts in its lifetime. |
attemptTimeoutMs |
10,000 | The longest the client waits for one batch's response. |
backoffMaxMs |
60,000 | The longest wait between attempts while sends keep failing. |
fallback |
Standard output | A (line) => void that receives the lines of each record the client could not deliver. |
Methods and results #
- The methods
log.debug,log.info,log.warn, andlog.error(message, fields)buffer one entry and return at once. The argumentfieldsis a flat map of string, number, boolean, or null values; leave the object unchanged after the call, because the client keeps it until delivery. - The method
count(name, increment)adds to a counter,1by default. The increment is a positive whole number. - The method
flush()sends the buffer now and resolves when each batch has a response, has failed, or has timed out. It never rejects. - The method
close()flushes and stops the client. Call and await it before an orderly shutdown: a process that exits first can lose buffered records. Because each attempt is bounded,close()completes even when the connection stalls. - The method
stats()returnsbuffered,dropped,pendingDropped,delivered, andfailures, the client's own accounting.
Delivery and failures #
The client throws nothing and never fails the caller. Logs are not a guaranteed-delivery queue.
- A flush sends the buffer in batches of at most 500 records and 512 KiB of body. Each batch is tried once.
- A batch the service refuses ends the flush. That batch and the ones after it go to the client's fallback output, standard output by default, and are counted as dropped.
- An entry or counter containing a NUL character (U+0000) or an unpaired UTF-16 surrogate in any name or value is counted as refused and not stored, while the rest of its batch is stored. The client writes no fallback line for it and adds nothing to
logging.dropped. The service counts it in the stream's counterlogging.refused_records. - An unpaired surrogate usually comes from text cut by UTF-16 code unit through an emoji, as
sliceandsubstringcut it. Cut on code points instead, or callString.prototype.toWellFormed()on the text before you log it. - A batch with no response within
attemptTimeoutMsis treated as refused. Its summary line reads, for example,timed out after 10000 ms, and a later response is ignored, though the service may have stored the batch. - The fallback writes one summary line, then one line per record. The summary identifies a failed send by the error's message, with the cause's code and message beside it. So a failed lookup or a refused connection shows as itself, not as an outage.
- A failure of the fallback itself is ignored, so it never fails the caller.
- While sends keep failing, each failed attempt writes one JSON line with the event
logging.backoff. The line gives the count of failed attempts in a row, the records that attempt lost, and the wait in milliseconds before the next attempt. - Each failed send doubles the wait before the next, up to
backoffMaxMs. The client reports what it dropped through the counterlogging.droppedon a later successful send. - When the bounded buffer overflows, the client drops the oldest records without writing them to the fallback.
The fallback's summary line gives a refused batch's status, as refused 503. The logging service's refusals are application_credential_required, environment_mismatch, environment_undeclared, invalid_batch, batch_too_large, rate_capped, log_capacity_reached, credential_lookup_busy, ingest_failed, authentication_required, invalid_token, insufficient_scope, end_user_credential_not_admitted, account_suspended, no_such_application, secret_grant_not_admitted, egress_key_not_admitted, misdirected_origin, retired_host, and store_unavailable. The refusals page gives each one's cause and remedy.
Test with the double #
The test double, createLoggingDouble() from @turnzero/logging/testing, stores entries and counters in memory behind the real client. Build the client over double.transport with environment: 'local', the stream a local run writes, which the double accepts by default. Call await client.flush() before reading the double, and await client.close() at the end. The functions entries() and total() read back what the double stored, and batches records each call. A test runs this sample over the double.
import assert from 'node:assert/strict';
import { LogClient } from '@turnzero/logging';
import { createLoggingDouble } from '@turnzero/logging/testing';
const logging = createLoggingDouble();
const client = new LogClient(logging.transport, { environment: 'local' });
client.log.info('note added', { note_id: 1 });
client.count('notes.added');
await client.flush();
assert.equal(logging.entries({ contains: 'note added' }).length, 1);
assert.equal(logging.total('notes.added'), 1);
await client.close();
The client throws nothing, as Delivery and failures says. It writes a batch the double refused, or could not deliver, to its fallback output and counts it in client.stats().dropped. So read a refusal on the batch the double recorded, in its status and answer.error. The function refuseNext(503) scripts a refusal for the next call, and failNext() an unreachable service.
| Refusal | Cause | Remedy |
|---|---|---|
invalid_batch, environment_undeclared, environment_mismatch, batch_too_large |
A malformed record, an environment the double does not accept, or a batch past its limits. | Use environment: 'local', and keep each record a level, a message, and flat fields. |
rate_capped, log_capacity_reached |
The batch passed batchesPerMinute or retainedCapacityBytes. |
Raise the limit. |
is not a refusal the logging service returns |
refuseNext named a refusal the service does not have. |
Use a refusal the service has, or a bare status. |