---
title: "Test in development, then promote"
description: Add a development copy of your app to test each version before your users see it, then promote a tested version to production.
slug: /guides/turn-on-development-and-promote
order: 24
visibility: public
redirects_from: [/guides/turn-on-a-development-environment-and-promote, /guides/test-in-development-then-promote]
---

# Test in development, then promote

Prompt:

> Add a development environment, so I can test before production.

Also works:

- "Let me try each version somewhere before my users see it."
- "The new version works. Put it live."

One call, `tzcloud_create_environment`, turns development on ([Turn the development environment on](#turn-the-development-environment-on)). After that, each deploy goes to development first. Once you say yes, `tzcloud_promote` puts the tested version on production ([Promote to production](#promote-to-production)). Without development, an application has production alone: its deploy puts the version live, and that deploy is the whole of taking it to production, with no promote.

[How the hostnames are formed](/cloud/concepts/applications-and-environments/#the-development-hostname) explains the application's label, the unique suffix added at creation, and both hostname forms.

## What your AI tool does

- Reads the `develop-then-promote` and `release` [skills](/cloud/concepts/glossary/#skill) (`read_context`, ids `skill:develop-then-promote` and `skill:release`) and follows them.
- Calls `tzcloud_create_environment` once, when you ask for a development environment.
- Before promoting, requests each route the change touched on the development hostname and checks `tzcloud_read_logs` for errors.
- Once development is `deployed`, promotes if you already asked for the promote; otherwise, asks you and waits for your yes. With no prior approval and nobody to ask, stops and reports.
- Asks for no browser approval: `tzcloud_create_environment` and `tzcloud_promote` run without one.

## What you need

- An application created as [Deploy from your computer](/cloud/getting-started/deploy-an-application/) describes.
- Every application can have a development environment. Its server is charged at the Shared 0.5 GB rate and always sleeps when idle. [Size and usage](/cloud/guides/plan-and-usage/) gives the limits and explains charges; the [pricing page](https://turnzero.ai/pricing) gives the current rates.

## Before your AI starts

**For the AI tool:** Check the following before starting.

- Your AI tool connected and signed in ([Connect your AI assistant](/cloud/getting-started/connect-your-tool/)).
- The application identifier, which `tzcloud_list_applications` returns.

## Steps

You turn development on once. After that, you deploy each version to development and promote it to production.

### Turn the development environment on

A new application has one environment, production. Its deploys go straight to production, and `tzcloud_roll_back` puts an earlier version back. A second environment, development, is a hosted copy of your app. There you test a version before your users see it.

Your AI tool calls `tzcloud_create_environment` with the application identifier and `environment: "development"`. The work is done when the call returns, and it needs no browser approval.

```json
{ "application": "<application id>", "environment": "development" }
```

The call returns `created: true` and the development hostname. If the application already has development, the call returns `created: false`. It then changes nothing, except to finish any step an earlier call left undone. The hosted development copy of your server needs its first deploy before it can serve application routes.

The answer also lists what development does not hold yet: each sign-in or push credential, and each secret a manifest setting refers to. Store each one with `tzcloud_store_secret`, naming development. A missing secret bound to a setting refuses the deploy. Missing sign-in or push credentials do not: the deploy can succeed while those providers remain unavailable. Submit the manifest again after storing a sign-in credential so the realm receives it. [Check the deploy](/cloud/getting-started/deploy-an-application/#checking-a-deploy-before-you-make-it) to see both kinds of missing credential before deploying.

You can call it before or after `tzcloud_submit_manifest`: either order ends in the same state. `tzcloud_submit_manifest` creates development's credentials on every application. Where the manifest declares a database, it also creates development's database. This holds whichever call comes first. `tzcloud_create_environment` adds the hosted development copy and creates no database. Each call reports the application's state as it read it, so two calls run at once can describe two different moments, and both are true.

From then on:

- a deploy goes to development, at `<label>-dev.ai.host`, and `tzcloud_promote` moves a version to production, at `<label>.ai.host`, where `<label>` is the application's current `label` ([The development hostname](/cloud/concepts/applications-and-environments/#the-development-hostname));
- where the manifest declares the accounts service, development has its own [realm](/cloud/concepts/glossary/#realm), which is its own list of end users, and local runs sign testers in there;
- the manifest's `realm` member and `push` providers apply to development as well as production;
- development's schedules run from its first deploy;
- test end users can be created on development too. Turning development on removes none from production, and your AI tool's session still creates them there ([Test sign-in with test end users](/cloud/guides/test-sign-in-with-test-end-users/#1-choose-the-environment)).

Local data that names production's users matches no one on development, because development's realm has its own users.

### Deploy to development

With development on, a deploy goes to development. Your AI tool calls `tzcloud_deploy_command` and runs the command line it returns, as steps 3 to 5 of [Deploy from your computer](/cloud/getting-started/deploy-an-application/#3-build-and-upload-the-artifact) describe. If production already serves a version, it keeps serving it while development deploys.

### Check development before promoting

**For the AI tool:** Check the changed behavior before promoting. A successful deploy proves the application starts and passes its health checks; it does not prove the changed routes work.

If you can run commands, write or update a verification script in the project. Give it the base URL so the same checks can run locally and against development. Check the expected responses and saved data, and make a failed check end the script with a nonzero status. [Build a public upload gallery](/cloud/guides/build-a-public-upload-gallery/) includes a tested upload service with such a script. Local tests with [test doubles](/cloud/guides/test-your-application-locally/) check application logic without calling the platform; also check the deployed service.

Without a shell, check the changed routes with `tzcloud_request_route`, or `tzcloud_write_route` for a route that writes, naming `environment: "development"`. Check saved data with `tzcloud_query_database` where needed. For routes that require sign-in, use `test_end_user` as [Test the signed-in paths](/cloud/guides/test-sign-in-with-test-end-users/#8-test-the-signed-in-paths) describes. Read development's logs with `tzcloud_read_logs` and investigate errors before promoting. A failed `tzcloud_request_route` answer's `detail` gives the reads for that request: one `tzcloud_read_logs` read for its log entries, then a console search.

To check other routes before a deploy or promote switches traffic, list read-only `GET` routes in the manifest's `smoke_paths` ([Listing smoke paths](/cloud/guides/author-the-manifest/#listing-smoke-paths)). A promote requests them from the new production copy, using production's data and secrets. Promoting the version production already serves skips these checks.

### Promote to production

Once development's deploy is `deployed` and you agree, or already asked for the promote, your AI coding tool calls `tzcloud_promote` with the application identifier and `wait_seconds`. Without a `version`, it takes the version development is serving.

```json
{ "application": "<application id>", "wait_seconds": 45 }
```

The promote takes the same `wait_seconds` as a deploy and responds the same way. Without it, the promote returns at once with status 202 and `state: "deploying"`. With it, the response waits for the promote to end, up to that many seconds. The response's `environment` is always `production`. Where something other than the platform responds, call `tzcloud_read_status` first, and promote again only where it shows no promote started after your call.

Besides `wait_seconds`, `tzcloud_promote` takes two optional members:

- `version`, the number of a deployed row in the [version history](/cloud/guides/manage-versions-and-environments/#the-version-history). If the call gives no version and development runs none, the call is refused 409 `nothing_to_promote`. A number that is not a deployed row of the application is refused 404 `no_such_version`.
- `rotate_database_credential`, which also sets a new password on the production database role, as [What happens during a promote](#what-happens-during-a-promote) describes.

A promote builds nothing. It reuses the image the deploy built, so production runs the same code development ran. The image also holds the [runtime harness](/cloud/concepts/glossary/#runtime-harness) as it was at that deploy. So a promote keeps that harness, and only a new deploy takes the current one. On an application with one environment, `tzcloud_promote` is refused 409 `environment_not_created`, since the deploy already reaches production.

#### What a promote does to production's database

Where the manifest declares the database kind and no production database exists yet, the first promote creates it on a production server. The platform creates no tables and runs no migrations. Your application's own [migration walk](/cloud/concepts/glossary/#migration-walk) does that. It applies your migration files to production's database when the new production container starts, before its server listens. It does the same in development. A promote takes no snapshot of the data first.

So a new migration changes production's data as soon as the new container starts, while the old version is still serving. A failed promote does not undo the change. A later rollback moves the code back but leaves the data as the migration left it. [Add a database](/cloud/guides/add-a-database/) says how to write a migration that the running version keeps working through.

#### What happens during a promote

On Shared 0.5 GB, production usually runs as a pod on shared machines. The steps below are the same there. So where `tzcloud_read_status` shows `grain` as `pod`, you see the same new copy, health check, and switch.

The promote works in this order:

1. It creates a new production platform credential and gives it to the new container. The current value keeps working until the switch.
2. With `rotate_database_credential: true`, it sets a new password on the production database role before the new container starts.
3. It starts the new version's production container next to the one serving now. The new container gets production's settings and credentials, including `TURNZERO_CLOUD_REALM_KEYS` where production has a realm.
4. It polls the health path on the new container alone, until it returns 200 three times in a row, three seconds apart. A 503 or a connection error between them starts the count again, inside the check's 180 seconds. Then any [smoke paths](/cloud/concepts/the-deploy-health-check/#checking-your-other-routes) are requested, unless production already serves this version. Once both pass, it switches the production hostname to the new container. A first promote records `deployed` at once. Where production already serves a version, the switch waits until the promote is 10 seconds old, and `deployed` comes 3 seconds after it, once every router has the switch.
5. It removes the previous container about 15 seconds after the switch. That is the routers' lookup interval for the serving container, 10 seconds by default, plus a 5-second grace period.

After a first promote records `deployed`, a request may still be answered `not_deployed` for up to about 2 seconds while the routers pick up the new container.

A failed promote ends `failed` and names the outcome. Its new container is removed. The serving container is untouched, and the current platform credential stays in place. The previously serving container keeps its platform credential until the platform removes that container, so a request that still reaches it is served.

A database password rotation is the exception. From the rotation until the switch, the previous container cannot open new database connections, though its open connections keep working. That is why the rotation runs only when you ask for it, not at every promote.

At the switch, production's schedules restart from the promote. A scheduled run still in progress on the previous container can fail, and the schedule runs again at its next due time.

With `wait_seconds`, the promote's response includes the outcome where the promote ends within the wait. Where it ended `deployed`, the response also carries `server_errors_since`: production's server errors (5xx answers) counted since that version began serving ([Check the release](/cloud/getting-started/deploy-an-application/#check-the-release)). The count covers only the routes requested since then, so request the others you rely on. Routes listed as `smoke_paths` in the manifest are checked before each deploy and promote switches. Otherwise make the `next` call the response gives, `tzcloud_read_status` with `wait_seconds`, or check the version's row in `tzcloud_list_versions`.

A promote interrupted by a platform restart before its switch ends `failed` with the outcome `interrupted` within about fifteen minutes. One interrupted after its switch ends `deployed`, because the new version is already running.

You can promote the version production already runs. The promote creates a new production platform credential and applies the settings again. With `rotate_database_credential: true`, it also sets a new database password.

## Expected result

With development turned on, the deploy is served at `<label>-dev.ai.host`, and the promote adds a `"promote"` row for production to the version history.

- After `tzcloud_create_environment`, `tzcloud_read_status` lists development beside production, development as `never_deployed` until its first deploy.
- After the promote, `tzcloud_read_status` shows production's `deploy.state` as `deployed` and its `version` as the version development served.

## Refusals

See the [refusals reference](/cloud/reference/refusals/) for other platform errors.

A promote can also meet the refusals it shares with a deploy, such as `deploy_in_flight` and `setting_secret_missing`. The [Refusals](/cloud/getting-started/deploy-an-application/#refusals) section of Deploy from your computer lists them.

After development is turned on, `tzcloud_create_environment` can also be refused `name_bound_to_setting`, `custody_entry_missing`, `custody_entry_unmovable`, or `realm_not_declared`. This happens while it copies the manifest's sign-in or push settings to development. Development stays on. The refusal's `detail` says what happened to the setting, and whether to call `tzcloud_create_environment` again or submit the manifest. [Author the manifest](/cloud/guides/author-the-manifest/#refusals) explains each.

| Refusal | Status | Cause | Remedy |
| --- | --- | --- | --- |
| `nothing_to_promote` | 409 | `tzcloud_promote` gave no version, and the development environment runs none. | Deploy to development first, or pass a deployed version. |
| `no_such_version` | 404 | `tzcloud_promote`: the version is not a deployed row of the application, or its environment was deleted. | Name a row `tzcloud_list_versions` returns with `state: "deployed"`. |
| `version_image_pruned` | 409 | `tzcloud_promote`: the platform has deleted the named version's image. | Choose a version `tzcloud_list_versions` marks `promotable`. Or deploy that version's artifact again as a new version, then promote it. |
| `environment_not_created` | 409 | `tzcloud_promote` on an application with one environment, whose deploy already reaches production. | Deploy instead, or turn development on with `tzcloud_create_environment` first. |
| `target_environment_halted` | 409 | `tzcloud_promote` while production is halted. | Call `tzcloud_halt_environment` on production with `resume: true`, then promote again. |
| `deletion_in_progress` | 409 | `tzcloud_promote` while a deletion of the application or of either environment runs. | Wait for the deletion to finish; `read_approval_status` shows whether it completed. If it failed, request the same deletion again to complete it. |
| `invalid_request` | 400 | `tzcloud_create_environment` named `production`, which always exists. | Name `development`, or leave `environment` out. |

## Related

- [Deploy from your computer](/cloud/getting-started/deploy-an-application/) builds and deploys the version you promote.
- [Manage versions and environments](/cloud/guides/manage-versions-and-environments/) rolls production back, reads the version history, and deletes the development environment.
- [Applications and environments](/cloud/concepts/applications-and-environments/#environments) describes the two environments.
- [What happens without asking](/cloud/concepts/what-happens-without-asking/) lists what the platform does on its own, such as halting an idle development environment.
- [Add sign-in to your app](/cloud/guides/add-sign-in-to-your-app/) describes the realm a development environment gets.
- [tzcloud_promote](/cloud/reference/actions/promote/) and [tzcloud_create_environment](/cloud/reference/actions/create-environment/) are the actions' generated reference entries.
