Use WebSockets

Prompt:

Show new chat messages to everyone in the room as soon as they are sent.

Also works:

• "Add live updates to the dashboard over a WebSocket."
• "Why does my socket keep closing on the hosted app?"

What your tool does

• Adds the websockets member to the manifest, listing any other browser origin whose pages open a connection, and submits the whole manifest with submit_manifest.
• Accepts the upgrade in the application's one server with a WebSocket library such as ws or Socket.IO, on a path of the application's own.
• Connects from a page on the application's own hostname, or from a native app with its bearer token.
• Has the server send a ping at least every 60 seconds, and has the client reconnect after a close, waiting longer after each failed attempt.
• Deploys, and promotes where the application has a development environment.
• Tests the connection on your machine, then on a hostname that serves upgrades, and reads a refused upgrade's name and detail.
• Asks for no browser approval: submit_manifest, deploy, and promote are reversible-tier actions.

What you need

• An application that deploys (Deploy an application).
• A server built on Node.js's http module, directly or through a framework such as Express, so the WebSocket library can take the upgrade from it.

Before your AI starts

This section is for your AI tool: what it checks and gathers before it begins. You don't need to do these steps yourself.

• Your tool connected and signed in (Connect your tool).
• list_applications returns the application's identifier and each environment's grain. An environment's grain is how its compute runs: container, compute of its own, as production always is, or pod, one of many development environments sharing a cluster. A development environment on the pod grain does not serve upgrades yet (step 7).
• The application's current manifest, because a submission replaces the whole document.
• ws or socket.io among the application's dependencies, installed before the build.

Steps

1. Declare WebSockets in the manifest

An application accepts WebSocket upgrades only where its manifest declares the websockets member. Without it, every upgrade is refused 400 websockets_not_declared before it reaches the application. The member is an object holding at most origins. The empty object admits a page served from the application's own hostname, and a client outside a browser, such as a native app or a script. No test checks this sample.

"websockets": {}

List an origin only where a page on another address opens the connection, such as a web client hosted elsewhere. Only https:// origins can be listed. A page served from http://localhost cannot connect to a hosted hostname, so run the server on your machine too (step 7).

origins holds at most ten. Each is exact: https://, a lowercase hostname, and a port only where it is not 443, with no path, no wildcard, and no repeat. Write an origin on port 443 without a port, since :443 is refused, and keep each dot-separated part of its hostname to 63 characters and the whole hostname to 253. A test checks this sample against the platform's rule for origins.

"websockets": { "origins": ["https://app.example.com", "https://staging.example.com:8443"] }

The router checks the handshake's origin before it reads the declaration. A page on another site sees 403 origin_not_admitted before websockets_not_declared, so a 403 does not mean the declaration is in place.

A submission whose list breaks a rule is refused 400 manifest_invalid. Each line of its violations names the entry's path and the rule, such as /websockets/origins/1: must omit the default port.

The declaration takes effect when submit_manifest records it, in both environments, with no deploy: the serving router reads it within about 30 seconds. Removing the member, or an origin, closes the open connections it admitted with code 1008 within about the same time. The manifest lists every member.

2. Accept the upgrade in the server

The upgrade reaches the application's own server, on the port its requests use. Hand that server to the WebSocket library, so one process serves both. With ws, a socket server on the path /live reads as below, where app is the application's request handler, such as an Express app. No test checks this sample.

import http from 'node:http';
import { WebSocketServer } from 'ws';

const server = http.createServer(app);
const sockets = new WebSocketServer({ server, path: '/live' });

sockets.on('connection', (socket) => {
  socket.on('message', (data) => {
    for (const other of sockets.clients) other.send(data.toString());
  });
});

server.listen(process.env.PORT ?? 8080);

Socket.IO attaches to the same http server in the same way, and serves its own path, /socket.io/. Socket.IO sends its own heartbeat as a data message every 25 seconds, and each reply counts one backend action. Its client also starts on HTTP long-polling, and each poll is a request. So prefer ws. With Socket.IO, set transports: ['websocket'] on the client and leave out step 4's ping.

Use a path of the application's own. An upgrade under /__account/, /__router/, or /.well-known/ is refused 400 upgrade_not_served, because the platform answers those paths itself. An upgrade on / reaches the application, even where the platform serves the web client's page there for an ordinary request.

The upgrade request meets every check an ordinary request meets, the per-address limits and the sign-in gate among them (Egress firewall and request limits). A handshake carries no body: one with a body is refused 400 invalid_request. The connection opens only where the server answers 101. Any other answer, such as the one the library writes for a path it does not serve, reaches the client as an answer, and the connection closes. The platform's refusals carry a JSON body with error; the library's own answer does not.

Once the server answers 101, the request deadline no longer applies, and the connection stays open within the limits in Limits and metering.

Each environment runs one copy of the server, so one process holds every connection to that environment. The exception is a deploy, promote, rollback, or restart, which starts a new copy and closes the old copy's connections (step 5).

3. Connect from the app's page, or list another origin

A page served from the application's hostname connects to the same hostname with wss://. No test checks this sample.

const socket = new WebSocket(`wss://${location.host}/live`);

Every browser sends an Origin header on the handshake, the address of the page that opens the connection. The router admits https:// with the hostname the upgrade arrived on, and each origin the declaration lists. Any other origin is refused 403 origin_not_admitted, before the router reads the visitor's session. So a page on another site cannot open a connection that carries your visitor's sign-in cookie.

A client that sends no Origin, such as a native app or a script, is admitted.

4. Keep the connection open

The router closes a connection after 120 seconds with no byte in either direction, with code 1001. Have the server send a WebSocket ping at least every 60 seconds; the sample below sends one every 30. Every browser and WebSocket client answers a ping with a pong by itself, and neither frame counts as a backend action. A message the page sends to keep the connection open would count as one. No test checks this sample.

const heartbeat = setInterval(() => {
  for (const socket of sockets.clients) socket.ping();
}, 30_000);
server.on('close', () => clearInterval(heartbeat));

A connection also closes two hours after it opened, with code 1001, whatever it carries. The client reconnects, as step 5 describes.

5. Reconnect after a close

The router closes a connection with a code and a short reason, and the client reconnects. The code tells the client how soon:

The reasons a 1008 close carries:

A connection can also end without a close frame, which a browser reports as code 1006. A network fault does this, and so does the server ending the connection itself. So does a close the router could not place between two frames within one second. The client reconnects with a growing wait.

For a growing wait, wait about one second before the first attempt, double the wait after each failed attempt up to about a minute, and add a random part to each wait. Stop reconnecting on account suspended and usage over quota, which last until the account or the month changes, and tell the user. No test checks this sample.

const STOP = new Set(['account suspended', 'usage over quota']);
let wait = 1_000;
function connect() {
  const socket = new WebSocket(`wss://${location.host}/live`);
  socket.onopen = () => { wait = 1_000; };
  socket.onclose = (event) => {
    if (event.code === 1008 && STOP.has(event.reason)) return;
    const delay = event.code === 1001 ? Math.random() * 1_000 : wait + Math.random() * 1_000;
    if (event.code !== 1001) wait = Math.min(wait * 2, 60_000);
    setTimeout(connect, delay);
  };
}
connect();

6. Sign in on the connection

A page on the application's hostname sends the session cookie with the handshake, as it does with a request. The router verifies the session at the handshake and sets the header x-turnzero-cloud-session on the upgrade request. So the server reads the signed-in user there as it does for a request (Verifying an end user). An expired cookie token is issued again on the 101, as on any response.

A native app sends its bearer token in the handshake's Authorization header, where its WebSocket client lets it set headers. A browser's WebSocket cannot, so a page relies on the cookie. An expired or revoked bearer token is refused 401 authentication_required on every audience, so refresh it before reconnecting.

The router reads the session once, at the handshake. Revoking the session, the user, the realm's sessions, or the key that signed the session closes the connection with code 1008 within about 60 seconds. The token's expiry alone does not close an open connection, and the two-hour limit still applies.

Under an invited or workforce audience, an upgrade without a valid session is refused 401 authentication_required, never sent to the sign-in page, because a handshake is no page visit. A path listed in session_free_paths is admitted without one.

7. Deploy and test

Test on your machine first. Run the server as Run your application on your machine describes, and connect to ws://localhost:<port>/live, on the port your server listens on. The platform's checks and limits do not apply there. Your server gets no x-turnzero-cloud-session header there, so every local connection is signed out.

Then deploy, and promote where the application has a development environment. The production hostname serves upgrades once production runs a version. A development environment on the container grain serves them too. One on the pod grain refuses each upgrade 400 websockets_not_yet_served, and its detail names the production hostname. There, test on the production hostname once production is deployed. list_applications and read_status show each environment's grain (Grain and development group).

A refused upgrade answers with a status and a JSON body whose error names the refusal and whose detail says why, and the connection then closes. A browser's WebSocket does not show the answer, and reports a close with code 1006. Read the answer with a client outside the browser, such as curl, naming the application's hostname and the path. No test checks this sample.

curl --http1.1 --include -H "Connection: Upgrade" -H "Upgrade: websocket" -H "Sec-WebSocket-Version: 13" -H "Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==" https://<label>.ai.host/live

A refusal prints its status line and its JSON body. A served upgrade prints HTTP/1.1 101 Switching Protocols and then waits on the open connection; stop it with Ctrl+C.

Limits and metering

The router bounds each connection and counts connections on each of its replicas. A router replica is one running copy of the router; each holds its own counts, and a client cannot choose which copy it reaches. The router runs at least three replicas for applications at https://turnzero.ai, so a production environment can hold at least three times its plan's figure. Turn Zero's own development platform at https://dev.turnzero.ai, which is not your application's development environment, runs one router replica, so there an environment's figure is its whole bound.

Half the environment's figure is 5 on Free, 100 on Standard, and 250 on Pro, so on Free one client address holds at most 5 connections to the application on each replica. Where a count is reached, the detail of socket_limit_reached names the limit and says that it is per replica. Where the router replica is stopping, the detail says so, and the answer carries Retry-After: 1, so the next attempt reaches another replica.

Users behind one network address share these figures. On Free, they share five connections on each router replica. On every plan, their connections without a session share 600 data messages a minute to the application on each router replica. Each signed-in user among them has 600 a minute of their own. The address's connections with sessions share 60,000 a minute, so a hundred signed-in users behind it each send their full 600. An office or a mobile carrier's gateway is one such address.

On the Free plan, an application's connections may stay open for 300 connection-hours in a UTC month, summed over its environments. Once the month reaches it, a new upgrade is refused 429 socket_hours_capped until the next UTC month, which its Retry-After header and its detail give. A connection already open runs to its own limits. Standard and Pro have no cap. Plan and usage lists the cap with the other plan quantities.

No read shows the month's connection time. One tab left open all month holds 720 connection-hours, because the pings keep it open. On Free, close the socket when the page is hidden, on the visibilitychange event, and open it again when the page is shown.

The upgrade counts one backend action, as the request it is. Each data message the client sends counts one more, at its last frame. Ping, pong, and close frames count nothing, and neither do the messages your server sends. The bytes the router sends to the client count as data transfer, added each minute while the connection is open, so they count in the month they cross.

An upgrade refused before it reaches the application counts nothing. An upgrade that reaches the application and is not switched, because the application answers otherwise, counts one backend action. The bytes of the answer the router writes count as data transfer. An upgrade the router forwards that fails before any answer, refused 502 upstream_failed, also counts one backend action, as a request that fails this way does.

While the application is over its plan's backend actions or data transfer for the month, its open connections close with code 1008 and new upgrades are refused 429 usage_over_quota, as its requests are.

Expected result

The client's handshake is answered 101, and the connection opens. Messages pass both ways until the client or the server closes the connection, or the router closes it with a code from step 5. read_logs with the source router shows one sockets_ended record each minute for each reason connections ended, with its reason, its code, and its count. It also gives the total and longest duration in milliseconds, duration_ms_sum and duration_ms_max (Read logs and counters). read_usage counts the upgrade and each message the client sent in backend_actions.

Refusals

A refused upgrade is answered on the connection with the status and a JSON body naming the refusal, and the connection then closes. An upgrade also meets every refusal an ordinary request on the hostname meets, such as 429 rate_capped, 429 usage_over_quota, and 401 authentication_required (Egress firewall and request limits; Plan and usage). Refusals lists every refusal the platform returns.

Related

• The manifest lists the websockets member beside the others.
• Egress firewall and request limits gives the connection limits beside the request limits.
• Plan and usage covers the plans, the Free connection-hours cap, and the usage the connections count toward.
• Verifying an end user explains the session header the upgrade request carries.
• Run your application on your machine runs the server for a local test.