Call an AI model #
On this page
Prompt:
Have the backend write a one-line summary of each new note with AI. I don't have a key for any AI service.
Also works:
- "Use the platform's AI key."
- "Suggest a title for each upload."
Generate text from your backend using the platform's AI key, with calls charged to the account's balance. Use the AI Allowance package and choose a fallback for an unavailable platform key or a provider capacity failure.
To keep conversation history, add database-backed routes with Build a database-backed service and store the messages and replies there.
At a glance:
- Package: the AI Allowance package, taken from the library (step 1).
- Settings your server reads:
TURNZERO_CLOUD_GATEWAY_URL, or elseTURNZERO_CLOUD_API, andTURNZERO_CLOUD_APP_TOKEN. - Import:
import { generateText } from '@turnzero/ai_allowance'; - The transport: a function that sends each path to that origin with
authorization: Bearer <TURNZERO_CLOUD_APP_TOKEN>. - The call:
generateText(transport, { system, messages, maxOutputTokens })returns the answer’stextand token usage. Itsallowancemembers are null for a customer application. Step 2 has the whole server. - Account pause: if
generateTextthrowsAllowanceRefusalwithrefusal: 'account_paused', follow step 5.
What your AI tool does #
- Reads the
use-ai-allowanceskill (read_context, idskill:use-ai-allowance) and follows it. - Writes the transport, the small function that sends each call, from the settings a deployed copy receives. Step 2's sample shows one.
- Adds the package and its version to the manifest's
packageslist at the nextsubmit_manifest. - Asks you how the application should respond to
allowance_unavailableor a provider capacity failure: answer without AI, or use your own declared upstream and key. It asks nothing else.
What you need #
- An application. Size and usage explains charges to its account.
- No account and no key with an AI provider.
Before your AI starts #
For the AI tool: Check the following before starting.
- A signed-in connection (Connect your AI assistant).
- The application's identifier, from
list_applications. - The current AI charge on the pricing page, or from
read_rate_card. - The backend's folder, the one that contains
package.json.
Steps #
1. Take the package #
Call read_library with contains: "ai_allowance" to find the ai_allowance library entry, then with entry: "ai_allowance" to read it. Add the package as Use the library describes. The package goes in app/lib/ai_allowance/, and package.json declares it as "@turnzero/ai_allowance": "file:lib/ai_allowance".
Each call goes through the platform's own route and uses a key the platform keeps. So the application stores no key for it. It also declares no egress entry and no upstream (outside service) for it.
2. Call generateText from a route #
generateText(transport, request, options) sends one request and returns the answer. The transport is a function you write. It puts the platform's origin, its base address, in front of each path. It also sends the application's credential as a bearer token. A deployed copy reads both values from its settings. A local run receives the development credential, so the same transport works there too.
This server answers POST /summaries with a one-line summary of the request’s note. It returns null on account_paused, allowance_unavailable, or a provider capacity failure. Its health path is /health. A test runs the sample over the package’s double and checks the summary, request and fallback.
// src/server.mjs: POST /summaries returns a one-line summary of the note in the body.
import { createServer } from 'node:http';
import { AllowanceRefusal, AllowanceUpstreamError, generateText } from '@turnzero/ai_allowance';
// The platform's origin and this copy's credential, from the settings it receives.
const origin = process.env.TURNZERO_CLOUD_GATEWAY_URL ?? process.env.TURNZERO_CLOUD_API;
const transport = (path, init = {}) =>
fetch(origin + path, { ...init, headers: { ...init.headers, authorization: `Bearer ${process.env.TURNZERO_CLOUD_APP_TOKEN}` } });
async function summarize(note) {
try {
const answer = await generateText(transport, {
system: 'Summarize the note in one line.',
messages: [{ role: 'user', content: note }],
maxOutputTokens: 200,
});
return answer.text;
} catch (error) {
// Stop this work on an account pause; handle key and capacity failures too.
const unavailable = error instanceof AllowanceRefusal && ['account_paused', 'allowance_unavailable'].includes(error.refusal);
const providerFull = error instanceof AllowanceUpstreamError && error.quota;
if (unavailable || providerFull) return null; // Return without another outbound call.
throw error;
}
}
const port = Number(process.env.PORT ?? 8080);
createServer(async (req, res) => {
if (req.method === 'GET' && req.url === '/health') return res.writeHead(200).end('ok');
if (req.method !== 'POST' || req.url !== '/summaries') return res.writeHead(404).end();
req.setEncoding('utf8');
let note = '';
for await (const chunk of req) note += chunk;
try {
const summary = await summarize(note);
res.writeHead(200, { 'content-type': 'application/json' }).end(JSON.stringify({ summary }));
} catch (error) {
console.error(error);
res.writeHead(502).end();
}
}).listen(port, () => console.log(`listening on ${port}`));
The client on the AI Allowance package page explains every field of the request. Among them are system, the roles in messages, responseSchema, maxOutputTokens, and thinkingBudget. Without the package, send a POST to /egress/v0/ai-allowance/v1beta/models/<model>:generateContent over the same transport, as that section describes.
A direct call without the package needs no ai_allowance entry in the manifest's packages list. Keep the entries for any other library packages the app holds.
3. Stream or hold a conversation #
To stream an answer, pass onText in the options. Each piece of text arrives in order, and the returned text holds the whole answer. To stream it on to a browser, write each piece to your response as it arrives. Once your response headers are sent, the serving router passes the response on. It cuts the response off when the request's execution window, its time limit, ends.
Pass the platform's request.signal beside onText in the third argument: { onText, signal: request.signal }. It fires when the request's execution window ends, not when the client leaves, as Node.js Runtime explains. These are the only options; there is no timeout or deadline option, and cancellation rejects the call.
In your local run, use a signal that fires when the response closes unfinished, rather than Node's request.signal, which some Node 24 releases abort at a normal request end. Pass that signal as signal beside onText; the ai-chat example shows abortSignalOf for this.
For a chat, send the conversation's turns as messages; keep them in your own store if they must survive the page closing. Add each answer that has text as an assistant turn, and leave out an empty answer. Every turn you send counts as input again on each call. So send only the recent turns, starting on a user turn.
Without a store, the page can hold its turns in memory and post them with each message, while the server sends the recent turns and keeps nothing, as the ai-chat example does.
4. Read AI spend and balance #
Call tzcloud_read_usage for the application’s AI spend this UTC month. Its ai_allowance_usd_micros measure gives the month’s total in micro-dollars, with no allowance quantity. For the account’s balance, warning record, pause state and monthly usage lines, call read_balance.
Each AI call draws the provider’s published price plus the pricing page’s markup from the account’s balance. Production, development and local calls all draw that balance. Read the current rates on the pricing page. AI Allowance explains how tokens are charged.
An AI answer does not report that balance. Its allowance object has used, quota and resetsAt set to null for customer applications. Do not treat those nulls as zero or calculate an amount remaining from them.
5. Handle an account pause #
When the account pauses, the router answers every request on the application’s hostnames with 429 account_paused, including the web client’s shell and sign-in routes. Scheduled runs and storage calls are refused too, and a background pass puts the compute to sleep. When the account pauses explains the account-wide stop.
Application code can receive account_paused from the gateway only in work already running when the pause begins. End that work without another outbound call or retry. Calls on your own declared upstream and key are refused too. The pause ends at the next UTC month; the HTTP response’s Retry-After gives the wait.
For allowance_unavailable or a provider capacity failure, the application can keep serving. Choose its response: answer without AI, or use your own declared upstream and key.
To use your own key, declare an upstream as Call an external API with an API key describes. Call it in the catch for that failure, while the account is not paused. These calls have no markup and draw nothing from the balance. The boundary and the fallback explains the other failures.
6. Test over the double #
The package includes a test double at @turnzero/ai_allowance/testing. It stands in for the allowance route, and generateText runs unchanged. So a test needs no gateway, no key, and no provider. You prime the double with the answer it should give. A test runs this sample and checks that the primed answer comes back.
import { generateText } from '@turnzero/ai_allowance';
import { createAiAllowanceDouble } from '@turnzero/ai_allowance/testing';
const double = createAiAllowanceDouble();
double.prime({ text: 'A note about lunch.', usage: { input: 40, output: 12 } });
const answer = await generateText(double.transport, { prompt: 'Summarize this note in one line: lunch at noon' });
To run step 2's own transport, replace the global fetch with double.fetch in the test. Test your application locally shows the double in a test suite, including its refusals.
7. Deploy #
Deploy as Deploy from your computer describes. The deploy gives the application the platform credential that the transport sends. That credential is bound to its environment, so the call needs no environment header.
Expected result #
POST /summaries returns the model’s one-line summary. tzcloud_read_usage reports its AI spend this month under ai_allowance_usd_micros; read_balance reports the account’s balance and monthly lines. If the gateway refuses its AI call with account_paused, the sample returns a null summary.
Refusals #
See the refusals reference for other platform errors.
| Refusal | Status | Cause | Remedy |
|---|---|---|---|
account_paused |
429 | The account is paused at a zero balance until the next UTC month. | Follow Handle an account pause. Do not retry the call. |
allowance_exhausted |
429 | One of Turn Zero's own unlimited applications reached its allowance quantity. | Wait for resets_at, or use your own upstream and key. This boundary does not apply to customer applications. |
allowance_unavailable |
503 | The platform has no key or no price in effect for the AI route. | Call your own declared upstream. Report it if it persists. |
allowance_application_required |
403 | The call used an account-wide credential, such as a session or an account-scoped token. | Call with the application's platform credential, or with a token for the application and the environment header. |
allowance_busy |
429 | The application already has as many allowance calls running as the platform allows at once. | Retry after one of them ends. |
allowance_path_refused |
403 | The request is not a POST to a generate method of a model whose name opens with gemini-. |
Call through the AI Allowance package's generateText function. Leave out model, or name a Gemini model, any whose name opens with gemini-. |
plan_quantity_unset |
409 | An unlimited application has no gemini-allowance quantity set. |
Report it to platform staff, or use your own upstream and key. |
Related #
- Build an app with sign-in and AI shows AI calls behind a verified user's session in one complete app.
- AI Allowance: every request and answer field, the count in dollars, and the fallback rule.
- Generate an image with AI: a picture on the same route, and what each one costs.
- Call an external API with an API key: call an AI service with your own key while the account is not paused.
- Use the library: taking the package into the project.
- Test your application locally: the double in a test suite.
- Size and usage: application limits and charges to the account.
- A deployed app's settings: the origin and the credential the transport reads.
- The turnzero command's ai-chat template: a new app with sign-in that calls
generateTextfrom its chat route, sending recent conversation turns through the platform’s AI route.