Call an AI model #

View this page as Markdown

On this page

Prompt:

Have the backend write a one-line summary of each new note with AI. I don't have a key for any AI service.

Also works:

  • "Use the platform's AI key."
  • "Suggest a title for each upload."

Generate text from your backend using the platform's AI key, with calls charged to the account's balance. Use the AI Allowance package and choose a fallback for an unavailable platform key or a provider capacity failure.

To keep conversation history, add database-backed routes with Build a database-backed service and store the messages and replies there.

At a glance:

  • Package: the AI Allowance package, taken from the library (step 1).
  • Settings your server reads: TURNZERO_CLOUD_GATEWAY_URL, or else TURNZERO_CLOUD_API, and TURNZERO_CLOUD_APP_TOKEN.
  • Import: import { generateText } from '@turnzero/ai_allowance';
  • The transport: a function that sends each path to that origin with authorization: Bearer <TURNZERO_CLOUD_APP_TOKEN>.
  • The call: generateText(transport, { system, messages, maxOutputTokens }) returns the answer’s text and token usage. Its allowance members are null for a customer application. Step 2 has the whole server.
  • Account pause: if generateText throws AllowanceRefusal with refusal: 'account_paused', follow step 5.

What your AI tool does #

  • Reads the use-ai-allowance skill (read_context, id skill:use-ai-allowance) and follows it.
  • Writes the transport, the small function that sends each call, from the settings a deployed copy receives. Step 2's sample shows one.
  • Adds the package and its version to the manifest's packages list at the next submit_manifest.
  • Asks you how the application should respond to allowance_unavailable or a provider capacity failure: answer without AI, or use your own declared upstream and key. It asks nothing else.

What you need #

  • An application. Size and usage explains charges to its account.
  • No account and no key with an AI provider.

Before your AI starts #

For the AI tool: Check the following before starting.

  • A signed-in connection (Connect your AI assistant).
  • The application's identifier, from list_applications.
  • The current AI charge on the pricing page, or from read_rate_card.
  • The backend's folder, the one that contains package.json.

Steps #

1. Take the package #

Call read_library with contains: "ai_allowance" to find the ai_allowance library entry, then with entry: "ai_allowance" to read it. Add the package as Use the library describes. The package goes in app/lib/ai_allowance/, and package.json declares it as "@turnzero/ai_allowance": "file:lib/ai_allowance".

Each call goes through the platform's own route and uses a key the platform keeps. So the application stores no key for it. It also declares no egress entry and no upstream (outside service) for it.

2. Call generateText from a route #

generateText(transport, request, options) sends one request and returns the answer. The transport is a function you write. It puts the platform's origin, its base address, in front of each path. It also sends the application's credential as a bearer token. A deployed copy reads both values from its settings. A local run receives the development credential, so the same transport works there too.

This server answers POST /summaries with a one-line summary of the request’s note. It returns null on account_paused, allowance_unavailable, or a provider capacity failure. Its health path is /health. A test runs the sample over the package’s double and checks the summary, request and fallback.

// src/server.mjs: POST /summaries returns a one-line summary of the note in the body.
import { createServer } from 'node:http';
import { AllowanceRefusal, AllowanceUpstreamError, generateText } from '@turnzero/ai_allowance';

// The platform's origin and this copy's credential, from the settings it receives.
const origin = process.env.TURNZERO_CLOUD_GATEWAY_URL ?? process.env.TURNZERO_CLOUD_API;
const transport = (path, init = {}) =>
  fetch(origin + path, { ...init, headers: { ...init.headers, authorization: `Bearer ${process.env.TURNZERO_CLOUD_APP_TOKEN}` } });

async function summarize(note) {
  try {
    const answer = await generateText(transport, {
      system: 'Summarize the note in one line.',
      messages: [{ role: 'user', content: note }],
      maxOutputTokens: 200,
    });
    return answer.text;
  } catch (error) {
    // Stop this work on an account pause; handle key and capacity failures too.
    const unavailable = error instanceof AllowanceRefusal && ['account_paused', 'allowance_unavailable'].includes(error.refusal);
    const providerFull = error instanceof AllowanceUpstreamError && error.quota;
    if (unavailable || providerFull) return null; // Return without another outbound call.
    throw error;
  }
}

const port = Number(process.env.PORT ?? 8080);
createServer(async (req, res) => {
  if (req.method === 'GET' && req.url === '/health') return res.writeHead(200).end('ok');
  if (req.method !== 'POST' || req.url !== '/summaries') return res.writeHead(404).end();
  req.setEncoding('utf8');
  let note = '';
  for await (const chunk of req) note += chunk;
  try {
    const summary = await summarize(note);
    res.writeHead(200, { 'content-type': 'application/json' }).end(JSON.stringify({ summary }));
  } catch (error) {
    console.error(error);
    res.writeHead(502).end();
  }
}).listen(port, () => console.log(`listening on ${port}`));

The client on the AI Allowance package page explains every field of the request. Among them are system, the roles in messages, responseSchema, maxOutputTokens, and thinkingBudget. Without the package, send a POST to /egress/v0/ai-allowance/v1beta/models/<model>:generateContent over the same transport, as that section describes.

A direct call without the package needs no ai_allowance entry in the manifest's packages list. Keep the entries for any other library packages the app holds.

3. Stream or hold a conversation #

To stream an answer, pass onText in the options. Each piece of text arrives in order, and the returned text holds the whole answer. To stream it on to a browser, write each piece to your response as it arrives. Once your response headers are sent, the serving router passes the response on. It cuts the response off when the request's execution window, its time limit, ends.

Pass the platform's request.signal beside onText in the third argument: { onText, signal: request.signal }. It fires when the request's execution window ends, not when the client leaves, as Node.js Runtime explains. These are the only options; there is no timeout or deadline option, and cancellation rejects the call.

In your local run, use a signal that fires when the response closes unfinished, rather than Node's request.signal, which some Node 24 releases abort at a normal request end. Pass that signal as signal beside onText; the ai-chat example shows abortSignalOf for this.

For a chat, send the conversation's turns as messages; keep them in your own store if they must survive the page closing. Add each answer that has text as an assistant turn, and leave out an empty answer. Every turn you send counts as input again on each call. So send only the recent turns, starting on a user turn.

Without a store, the page can hold its turns in memory and post them with each message, while the server sends the recent turns and keeps nothing, as the ai-chat example does.

4. Read AI spend and balance #

Call tzcloud_read_usage for the application’s AI spend this UTC month. Its ai_allowance_usd_micros measure gives the month’s total in micro-dollars, with no allowance quantity. For the account’s balance, warning record, pause state and monthly usage lines, call read_balance.

Each AI call draws the provider’s published price plus the pricing page’s markup from the account’s balance. Production, development and local calls all draw that balance. Read the current rates on the pricing page. AI Allowance explains how tokens are charged.

An AI answer does not report that balance. Its allowance object has used, quota and resetsAt set to null for customer applications. Do not treat those nulls as zero or calculate an amount remaining from them.

5. Handle an account pause #

When the account pauses, the router answers every request on the application’s hostnames with 429 account_paused, including the web client’s shell and sign-in routes. Scheduled runs and storage calls are refused too, and a background pass puts the compute to sleep. When the account pauses explains the account-wide stop.

Application code can receive account_paused from the gateway only in work already running when the pause begins. End that work without another outbound call or retry. Calls on your own declared upstream and key are refused too. The pause ends at the next UTC month; the HTTP response’s Retry-After gives the wait.

For allowance_unavailable or a provider capacity failure, the application can keep serving. Choose its response: answer without AI, or use your own declared upstream and key.

To use your own key, declare an upstream as Call an external API with an API key describes. Call it in the catch for that failure, while the account is not paused. These calls have no markup and draw nothing from the balance. The boundary and the fallback explains the other failures.

6. Test over the double #

The package includes a test double at @turnzero/ai_allowance/testing. It stands in for the allowance route, and generateText runs unchanged. So a test needs no gateway, no key, and no provider. You prime the double with the answer it should give. A test runs this sample and checks that the primed answer comes back.

import { generateText } from '@turnzero/ai_allowance';
import { createAiAllowanceDouble } from '@turnzero/ai_allowance/testing';

const double = createAiAllowanceDouble();
double.prime({ text: 'A note about lunch.', usage: { input: 40, output: 12 } });
const answer = await generateText(double.transport, { prompt: 'Summarize this note in one line: lunch at noon' });

To run step 2's own transport, replace the global fetch with double.fetch in the test. Test your application locally shows the double in a test suite, including its refusals.

7. Deploy #

Deploy as Deploy from your computer describes. The deploy gives the application the platform credential that the transport sends. That credential is bound to its environment, so the call needs no environment header.

Expected result #

POST /summaries returns the model’s one-line summary. tzcloud_read_usage reports its AI spend this month under ai_allowance_usd_micros; read_balance reports the account’s balance and monthly lines. If the gateway refuses its AI call with account_paused, the sample returns a null summary.

Refusals #

See the refusals reference for other platform errors.

Refusal Status Cause Remedy
account_paused 429 The account is paused at a zero balance until the next UTC month. Follow Handle an account pause. Do not retry the call.
allowance_exhausted 429 One of Turn Zero's own unlimited applications reached its allowance quantity. Wait for resets_at, or use your own upstream and key. This boundary does not apply to customer applications.
allowance_unavailable 503 The platform has no key or no price in effect for the AI route. Call your own declared upstream. Report it if it persists.
allowance_application_required 403 The call used an account-wide credential, such as a session or an account-scoped token. Call with the application's platform credential, or with a token for the application and the environment header.
allowance_busy 429 The application already has as many allowance calls running as the platform allows at once. Retry after one of them ends.
allowance_path_refused 403 The request is not a POST to a generate method of a model whose name opens with gemini-. Call through the AI Allowance package's generateText function. Leave out model, or name a Gemini model, any whose name opens with gemini-.
plan_quantity_unset 409 An unlimited application has no gemini-allowance quantity set. Report it to platform staff, or use your own upstream and key.