# nginx/nginx.conf

http {
  map $http_upgrade $connection_upgrade {
    default upgrade;
    '' close;
  }

  server {
    listen 80;
    server_name retool.app.localhost;
    error_log /var/log/nginx/retool80.error_log debug;

    location /api/multiplayer/ {
      proxy_pass http://multiplayer:3009/api/multiplayer/;

      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection $connection_upgrade;
      proxy_set_header Host $host;
    }

    # Existing configuration to route traffic to Retool's API.
    # This config must come after the /api/multiplayer config above since
    # it is less specific.
    location / {
      proxy_pass http://api:3000/;

      resolver 127.0.0.11 valid=30s;
      add_header X-XSS-Protection "1; mode=block";
      add_header X-Content-Type-Options nosniff;
    }
  }
}
```

## Verify functionality

To verify that the feature is working, open an app in the Retool editor. If the multiplayer feature is enabled, you will see your avatar on the right side of the status bar.

The **Network** tab should show a WebSocket connection. The **Status** column should say `101` and the **Time** column should say `Pending` because the connection is left open.

WebSocket connection

The following screenshot shows example connection details. The expected response **Status Code** is `101 Switching Protocols`.

Example connection details

## Troubleshooting

If multiplayer functionality isn’t working, or you see many repeated connection attempts in the **Network** tab, the proxy likely isn’t configured correctly. To confirm traffic is reaching the multiplayer service, check the service’s logs for messages such as `“Handle updated request for ” and “Handle new connection for ...”`. 

If you don't see any logs from the multiplayer service, that can be an indicator that the routing is configured incorrectly, and that the traffic was directed to the backend service instead. Double check that your [web proxy](#configure-web-proxy) is configured correctly.

If you have any issues, please reach out to Retool Support and send your multiplayer service logs.

---

## Configure same-origin and sandbox for iframes

import EnterpriseRequirement from "../_partials/_enterprise-plan-requirement.mdx";

By default, self-hosted deployments enforce the [same-origin policy](https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy) for iframes and custom components embedded in Retool apps. All embedded content is considered to be from a separate origin and fails the same-origin policy. 

This isolates embedded content (e.g., custom components) for security purposes but it can restrict functionality, such as:

- Accesing cookies.
- Accessing certain JavaScript APIs.
- Making any HTTP requests.

If necessary, you can set the `ALLOW_SAME_ORIGIN_OPTION` environment variable to use the `allow-same-origin` attribute and bypass these restrictions.

## Update environment variables

You can update your environment variable configuration to use `allow-same-origin` for iframe content by setting the `ALLOW_SAME_ORIGIN_OPTION` environment variable to `true`. 

:::danger Set `SANDBOX_DOMAIN` to avoid security risks

If you need iframes to use `allow-same-origin`, you should also set the `SANDBOX_DOMAIN` environment variable.

:::

## Configure sandbox domain

By default, your Retool instance will make use of a single domain, and all access to your Retool backend will arrive via that domain. However, having only a single domain can cause some security issues when `ALLOW_SAME_ORIGIN_OPTION` is set to `true`. 

Setting up an additonal domain to point to your Retool backend, and then setting the value `SANDBOX_DOMAIN` to this second domain, can protect you from these security issues. 

Once `SANDBOX_DOMAIN` is set, it provides an alternative origin for the browser to use when executing builder-written JavaScript. This isolates builder-written code from interacting with the the rest of the page and it's contents, including authentication cookies for your Retool backend. 

If `SANDBOX_DOMAIN` is not set, but `ALLOW_SAME_ORIGIN_OPTION` is set, all builder-written JavaScript code runs on the same origin as your Retool deployment instance, and has access to everything on the page, including all cookies. A malicious builder on your Retool instance could take advantage of this to run their javascript code in other builder's browsers, and take actions on their behalf. 

The value you use for `SANDBOX_DOMAIN` should be a fully functional domain that routes HTTP requests to your Retool instance. In most cases, this requires going through the process of registering a new domain, and configuring it to point to your Retool backend.

For example, if you currently host your Retool backend on `mydomain.com`, Retool recommends you registering a new domain, such as `mydomain-sandbox.com`, and set `mydomain-sandbox.com` to point to your Retool backend (identically to how you setup `mydomain.com`). Then, set the value of the `SANDBOX_DOMAIN` environment variable to be `https://mydomain-sandbox.com`. Note that you must include the protocol in the value, which is usually `https`.

---

## Prevent users from creating or using specific resource types

import EnterpriseRequirement from "../_partials/_enterprise-plan-requirement.mdx";

Self-hosted organizations can specify optional restrictions using [environment variables](../reference/environment-variables/index.mdx) that prevent users from creating or using certain resource types. The restrictions you set depend upon your use case. Configuring these restrictions does not remove or modify the configuration of an existing resource.

Any restrictions you set can be reverted at any time. This enables you to temporarily disable resources (e.g., if your security team needs to review an integration before it can be used).

## Prevent users from creating certain resource types

Use the `RESOURCE_TYPES_CREATION_DENY_LIST` environment variable to specify a comma-separated list of resource integration types that cannot be created. Users can still interact with any existing resources of these types but they will not be able to create new ones.

```title="Example"
RESOURCE_TYPES_CREATION_DENY_LIST=graphql,twilio,postgresql
```

## Prevent users from creating or using certain resource types

In some cases, you may need to block all use of a certain resource type. You can use the `RESOURCE_TYPES_DENY_LIST` environment variable to provide a list of all resource integration types that are to be blocked. This effectively disables the resource type on the deployment instance, preventing users from creating and querying resources of these types.

```title="Example"
RESOURCE_TYPES_DENY_LIST=graphql,twilio,postgresql
```

Once enabled, any queries for restricted resources will not run and return a query error.

:::note

The error message displayed in self-hosted Retool 3.300 and later also explains that that query failed due to the resource being restricted. Prior releases only return a query error.

:::

## Specify the environment variable values

Both environment variables can accept a comma-separated list containing any of the following resource type values. Refer to the [self-hosted deployment tutorials](../tutorials.mdx) to learn more about configuring environment variables for your instance.

| Resource               | Type                 |
| ---------------------- | -------------------- |
| PostgreSQL             | `postgresql`         |
| MySQL                  | `mysql`              |
| MSSQL                  | `mssql`              |
| OracleDB               | `oracledb`           |
| Redshift               | `redshift`           |
| MCP                    | `mcp`                |
| MongoDB                | `mongodb`            |
| Google Sheets          | `googlesheets`       |
| Elasticsearch          | `elasticsearch`      |
| Cassandra              | `cassandra`          |
| CosmosDB               | `cosmosdb`           |
| CouchDB                | `couchdb`            |
| RethinkDB              | `rethinkdb`          |
| REST API               | `restapi`            |
| GraphQL                | `graphql`            |
| BigQuery               | `bigquery`           |
| S3                     | `s3`                 |
| GCS                    | `gcs`                |
| Slack                  | `slackopenapi`       |
| Salesforce             | `salesforce`         |
| Athena                 | `athena`             |
| GitHub                 | `github`             |
| Stripe                 | `stripe`             |
| Twilio                 | `twilio`             |
| SendGrid               | `sendgrid`           |
| Firebase               | `firebase`           |
| DynamoDB               | `dynamodb`           |
| Basecamp               | `basecamp`           |
| Close.io               | `closeio`            |
| Snowflake              | `snowflake`          |
| Redis                  | `redis`              |
| Vertica                | `vertica`            |
| Presto                 | `presto`             |
| SAP Hana               | `saphana`            |
| Lambda                 | `lambda`             |
| OpenAPI                | `openapi`            |
| Google Cloud Datastore | `datastore`          |
| gRPC                   | `grpc`               |
| SMTP                   | `smtp`               |
| Jira                   | `jira`               |
| BigID                  | `bigid`              |
| AlloyDB                | `alloydb`            |
| Databricks             | `databricks`         |
| Databricks Lakebase    | `databricksLakebase` |
| JDBC                   | `jdbc`               |
| Kafka                  | `kafka`              |
| SQS                    | `sqs`                |
| SNS                    | `sns`                |
| Tavily                 | `tavily`             |

---

## Set up Retool Database on Self-hosted deployments

import EnterpriseRequirement from "../_partials/_enterprise-plan-requirement.mdx";

[Retool Database](../../data-sources/quickstarts/retool-database.mdx) is a PostgreSQL resource provided by Retool. You can query it like any other resource, but it also includes a spreadsheet-like interface for interacting with data.

On self-hosted deployments, you can use your own PostgreSQL database to host Retool Database. Your database automatically connects to the main Retool backend service for your deployment. This allows you to use Retool Database's interface for managing data. 

Retool Database is included in all pricing plans. Because you control your own PostgreSQL database on self-hosted deployments, there are no data limits or overages for Retool Database on self-hosted deployments.

:::info

Retool Database is separate from the [storage database](../concepts/architecture.mdx#containers-and-services), which is required in self-hosted Retool deployments to store user information and logs.

:::

## Requirements

To use Retool Database on self-hosted deployments, you need:

- A PostgreSQL database. On self-hosted Retool versions 3.14 or later, you need a database running PostgreSQL version 14.3 or later.
- Credentials for a database user with permissions to create databases and roles. These credentials are used once to create the database and generate a generic user, who only has access to the created database. Further interactions, such as querying the database, use the generic user and its permissions.

:::note

Organizations with self-hosted deployments using [Kubernetes](../tutorials/kubernetes/manifests.mdx), [Helm](../tutorials/kubernetes/helm.mdx), or [ECS Fargate](../tutorials/ecs-fargate/index.mdx) must specify a PostgreSQL database to use for the backend. 

:::

## 1. Add database configuration

Navigate to `{your-domain}/resources?setupRetoolDB=1` on Retool and fill out the form with the following fields.

| **Setting**       | **Description**                                   |
| ----------------- | ------------------------------------------------- |
| Database host     | The URL or IP address.                            |
| Database name     | The name of the database.                         |
| Database username | The username.                                     |
| Database password | The password.                                     |
| Database port     | The port number.                                  |
| Use SSL/TLS       | Connect with [SSL/TLS](../../data-sources/guides/connections/ssl.mdx). |

Select **Test your connection** to confirm you can connect to Retool Database.  

:::warning
Using Docker to host Retool Database is not recommended for storing production data, and Retool recommends [externalizing the database](./retool-database.mdx#specify-an-external-postgresql-database)

:::

A Retool Database PostgreSQL container is automatically created and provisioned on new deployments that use [Docker Compose](../tutorials.mdx). Refer to the [retool-onpremise](https://github.com/tryretool/retool-onpremise) repository for more information on adding a Retool Database PostgreSQL container to existing deployments.

## 2. Save your resource

Click **Save and initialize** to save your Retool Database settings. You can now use with the [Database editor UI](../../data-sources/quickstarts/retool-database.mdx#interact-with-data-using-the-database-editor-ui) to configure and interact with your Retool Database tables.

## 3. Import existing data

Retool always creates a new database for each of your environments. To import existing data to Retool Database, use [pg_dump and psql](https://www.postgresql.org/docs/14/backup-dump.html) commands. 

You can use the [connection strings](../../data-sources/quickstarts/retool-database.mdx#connection-strings) dropdown to find the database host, name, user, password, and port for the current environment. 

:::note

Changing the connection string does not switch the underlying data store, and the only way to migrate existing data from one database to another is through `pg_dump` and `psql`. You can also connect an existing PostgresSQL database using the [native integration](../../data-sources/guides/integrations/database/postgresql.mdx).

:::

Use `pg_dump` and `psql` commands to migrate data between databases. Your commands might use the following structure:

```title="Export SQL"
pg_dump EXISTING_DATABASE_CONNECTION_STRING > pg_dump.sql
```

```title="Import SQL"
psql RETOOL_CONNECTION_STRING < pgdump.sql
```

---

## Set up Retool Storage on Self-hosted deployments

import EnterpriseRequirement from "../_partials/_enterprise-plan-requirement.mdx";

[Retool Storage](../../data-sources/quickstarts/retool-storage.mdx) is a file storage resource provided by Retool. You can use it like any other file storage resource, with the addition of a Retool-provided file manager interface.

On self-hosted deployments, you can use your own storage bucket to host Retool Storage. This allows you to use Retool Storage's interface for managing files.

:::info 

Retool supports the following storage providers:

* [Amazon Simple Storage Service (Amazon S3)](../../data-sources/guides/integrations/object-file-store/amazon-s3.mdx)
* [Google Cloud Storage (GCS)](../../data-sources/guides/integrations/google/google-cloud-storage.mdx)
* S3-compatible storage providers like [Digital Ocean Spaces](https://docs.digitalocean.com/reference/api/spaces-api/) or [Wasabi](https://wasabi.com/downloads/)

:::

Retool Storage is included in all [pricing plans](https://retool.com/pricing). Because you control your own file storage bucket on self-hosted deployments, there are no data limits or overages for Retool Storage on self-hosted deployments.

### Requirements

To use Retool Storage on self-hosted deployments, you need credentials for a user who has permissions to modify and read files within your file storage provider. Retool uses these credentials to upload, download, and delete files.

### 1. Add storage configuration

Navigate to `{your-domain}/resources` on Retool and next to Retool Storage click **Settings**

Set up the resource per your storage provider.

Click **Test your connection** to confirm that you can connect to your storage provider.  

### 2. Save your resource

Click **Save** to save your Retool Storage settings. You can now use with the [Retool Storage interface](../../data-sources/quickstarts/retool-storage.mdx) to preview, upload, and download your Retool Storage files.

:::warning 

### Changing storage provider will break all existing files

In order to migrate to another provider, all files need to be manually copied from the prior storage bucket to the new one with identical folder structure.

:::

---

## Rotate your encryption key

import EnterpriseRequirement from "../_partials/_enterprise-plan-requirement.mdx";

Retool supports encryption key rotation for organizations with self-hosted deployments. This encryption key is used to encrypt sensitive values in Retool's backend database. Encryption key rotation is a *zero-downtime* operation. 

Organizations should rotate their encryption key if they suspect that their key was leaked or exposed to an unauthorized individual. Use the procedures below to add an encryption key to a deployment that previously did not have one. When using a deployment previously missing an encryption key, Retool generally does *not* encrypt values found to be in plaintext. 

The minimum version required to complete an encryption key rotation is 3.148. 

### 1. Take a snapshot of your deployment's database
Before you begin, Retool recommends that you take a snapshot of your instance's database and store it somewhere safely with limited permissions. Retool's key rotation procedure is intended to have no downtime or data loss, but its status as an public beta feature means there is a higher risk of data loss. 

### 2. Update your deployment's environment variables
Start with generating a new encryption key. Retool recommends your keys to be at least 64 characters in length. Set the following environment variables:

```
USE_ENCRYPTION_V2=true
ENCRYPTION_KEY_OLD={old encryption key}
ENCRYPTION_KEY={new encryption key}

ENABLE_LOGS_FOR_DECRYPTION_WITH_OLD_KEY=true # optional, used for observability (more below)
```

After updating these environment variable:
* Your database reads will be able to decrypt values encrypted with either the old or the new encryption key. 
* Your database writes will be encrypted with the new encryption key. 
* After adding a new key, it won't be safe to remove it until another database rotation is completed since it could be in use

If you are setting the encryption key for the first time, and therefore do not have an old encryption key, you can leave the `ENCRYPTION_KEY_OLD` environment variable blank in the encryption key rotation script. 

### 3. Run the database rotation script
To complete your encryption key rotation, encrypt all the values in your database to be encrypted with the new key using the following script:

```shell
./retool_backend --backfill-encrypt-secrets

Optional args:
--batch-size 
    > the number of rows to operate on in each batch (default 1000)
--resume-run-uuid 
    > the uuid of the backfill/key rotation run to resume, in case it previously failed/exited/etc

```

### 4. Monitor your deployment for completion
Monitor the output of the above script for completion of the database rotation. After completion, you can monitor your instance for any usage of the old encryption key. With the `ENABLE_LOGS_FOR_DECRYPTION_WITH_OLD_KEY` environment variable, any usage of the old key for decryption will emit one of the following logs:

```shell
[decrypt] Decrypting a secret using the old encryption key AND old encryption schema!
[decrypt] Decrypting a secret using the old encryption key!
[decryptBuffer] Decrypting a secret using the old encryption key!
```

If you see these logs after completing the encryption key rotation script, please contact us immediately at support@retool.com. 

### 5. Remove previous encryption key
If you are rotating your encryption key because you suspect a data breach, Retool recommends that you remove all references to the previous key as soon as you are confident the encryption key rotation script is complete. To do so, you can remove the `ENCRYPTION_KEY_OLD` environment variable from your instance.

---

## Rotate SSH keys

import EnterpriseRequirement from "../_partials/_enterprise-plan-requirement.mdx";

Organizations with Self-hosted Retool deployments can regenerate SSH keys used for SSH tunneling. This allows you to follow your organization's credential rotation policy or update keys if your encryption key has changed.

## Invalidate existing SSH keys

If you've rotated your encryption key—for example, while upgrading from Retool Cloud to Self-hosted—you may need to invalidate existing keys. These steps require access to the [Retool PostgreSQL database](storage-database.mdx).

### 1. Check the `ssh_keys` table

This SQL query should only return one record.

```sql
select * from ssh_keys
```

### 2. Remove the existing SSH key

```sql
truncate ssh_keys
```

## Regenerate and download the new SSH key

When you download Retool's public key, Retool creates a new SSH key if it doesn't already exist.

If your key is for SSH tunneling, use the **Download Retool's public key** link on the resource configuration page. Open the `retool.pub` file and copy the entire key to your clipboard. Clicking the link generates a new record in the `ssh_key` table and downloads the public key to your local machine.

---

## Scale your self-hosted deployment infrastructure

import ReleaseVersions from '@site/static/data/releases/stable.json'
import EnterpriseRequirement from "../_partials/_enterprise-plan-requirement.mdx";

Retool's self-hosted Docker image consists of several containers required to run your deployment. Before you continue, you should review [Retool's self-hosted architecture](../concepts/architecture.mdx) to understand which services need scaling as usage increases.

## Scale your Retool deployment

It's recommend to self-host Retool on a container orchestration service, such as Kubernetes. Retool provides [Helm charts](../tutorials/kubernetes/helm.mdx) and [Terraform modules](../tutorials/ecs-fargate/terraform.mdx) for you to easily get started. See the [deployment overview](../tutorials.mdx) for a full list of supported deployment options.

Retool is packaged as a single stateless Docker container. The only dependency it has is a PostgreSQL database, which is used to store data such as user information, organization settings, audit logs, and applications.

To scale a Retool instance, you follow these high-level steps:

1. Host the [PostgreSQL database](storage-database.mdx) on an external system, e.g., AWS RDS. The separation of database and application allows you to independently scale and manage each service.
2. Start multiple Retool containers that use the same Postgres database. The number of containers you deploy depends on your traffic and resource requirements. We recommend scaling the `api` container. You can do this by updating the replica count in your container orchestration service of choice.
3. Use a load balancer to route traffic between the Retool server containers to ensure high availability.

:::note

You should only run one replica of the `jobs-runner` in each Retool environment because it runs database migrations and other background tasks that should only operate as a singleton. You can scale up as many replicas as necessary of the containers/pods that run the other service types.

:::

## Near-zero downtime upgrades

All `MAJOR.MINOR` release upgrades, such upgrading from {Object.keys(ReleaseVersions)[1]} to {Object.keys(ReleaseVersions)[0]}, perform database migrations. Patch release upgrades within the same `MAJOR.MINOR` release, such as {Object.keys(ReleaseVersions)[0]}.1 to {Object.keys(ReleaseVersions)[0]}.2, generally do not.

### General upgrade strategy

Each container or pod attempts to perform database migrations at startup after an upgrade. In multi-container setups, upgrades must be coordinated carefully to prevent container deadlocks and minimize downtime.

:::tip
Database migrations are backward-compatible so older containers or pods can continue serving traffic while these migrations run.
:::

1. Stop all containers except `jobs-runner`.
1. Start a new `jobs-runner` container using the latest `MAJOR.MINOR.PATCH` image tag, such as {Object.keys(ReleaseVersions)[0]}.1-stable.
1. Wait for the database migrations to complete.
1. Stop the old `jobs-runner` container.
1. Start the remaining containers using the new image.
1. Verify all containers are healthy before routing traffic.

Kubernetes + Helm deployments support rolling upgrades that simplify the process. First, use `helm upgrade` with the latest release tag. For example:

```
helm upgrade retool retool/retool \
  --namespace retool \
  --set image.tag=3.196.1-stable
```

During the upgrade process:

- A new `jobs-runner` pod is created and starts running migrations.
- All other new pods start but wait for migrations to complete.
- Once all new pods are verified as healthy, the old pods are terminated automatically.

:::tip

You can monitor the progress of database migrations for the `job-runner` pod using `kubectl`:

```
kubectl logs deployment/jobs-runner -n retool
```
:::

For Kubernetes deployments without Helm, perform a manual rolling update.

1. Patch the `jobs-runner` deployment to use the new image tag.
1. Wait for the new pod to start and complete database migrations.
1. Patch the remaining deployments to use the new image.
1. Monitor rollout status and ensure all pods are healthy:

:::tip

You can monitor the progress of database migrations for the `job-runner` pod using `kubectl`:

```
kubectl rollout status deployment/ -n retool
```
:::

---

## _config Check

Retool checks the validity of the configuration to ensure that secrets are available. If Retool cannot access the secrets manager you've configured, an error message appears in the Secrets Manager settings.

Validate secrets configuration.

---

## Retrieve secrets from AWS Secrets Manager

import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

Configuring Resources in Retool can require handling sensitive values, e.g. database passwords or API keys. Retool is [SOC 2 Type 2 compliant](https://docs.retool.com/legal/security), and most customers store these values with Retool. However, depending on your security posture, you may want to store secret values externally, rather than encrypted in Retool’s database.

To support this, Retool supports retrieving secrets from AWS Secrets Manager for customers running self-hosted Retool.

## Requirements

To use the AWS Secrets Manager with Retool, you need:

- A self-hosted Retool deployment.
- Have [admin permissions](../../../permissions/guides.mdx) for your Retool organization.
- Already have AWS Secrets Manager configured to store your secrets.

## 1. Create IAM policy for Retool

In AWS, [create an IAM policy](https://docs.aws.amazon.com/secretsmanager/latest/userguide/auth-and-access_examples.html) to grant Retool access to your secrets. Retool recommends you store secrets under a `retool/` namespace for easier access control. The following is an example policy you might use.

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue",
        "secretsmanager:DescribeSecret"
      ],
      "Resource": ["arn:aws:secretsmanager:Region:AccountId:secret:retool/*"]
    },
    {
      "Effect": "Allow",
      "Action": ["secretsmanager:ListSecrets"],
      "Resource": "*"
    }
  ]
}
```

## 2. Attach IAM policy to your principal

Next, attach the policy to the principal associated with your Retool workload. For example, if you deploy Retool on EC2, attach this policy to the instance profile for the EC2 instance.

## 3. Configure Secrets Manager in Retool
Secrets management must be configured for the primary organization, and for any spaces within an organization that need to use secrets.

import MultiInstance from './_multi-instance.mdx';

For each [Retool Space](../../../org-users/tutorials/spaces), visit **Settings** > **Secrets Manager** and enter the **Region** associated with your AWS Secrets Manager instance and an optional **Namespace** (details below). Click **Test connection** to confirm Retool can connect to your AWS Secrets Manager, then click **Save**.

### Namespace

The **Namespace** field is a value that will be prepended to any secret names Retool looks up. For example, if all your Retool-facing secrets in AWS Secrets Manager start with `retool/`, you might set the namespace to be `retool/` as well. If you have multiple Retool deployment instances, you can use this to resolve the same secret name in a Resource configuration to different secrets in your AWS account. For this reason, setting the namespace is recommended if you use [Protected Resources](../../../source-control/guides/protect/resources.mdx).

For example, if you have "development" and "production" deployment instances, you might use the following naming convention for secrets in your AWS account:

```
// Secrets for development
retool/dev/db_user
retool/dev/db_password

// Secrets for production
retool/prod/db_user
retool/prod/db_password
```

On your Retool development instance, you would then set the namespace to `retool/dev`, and on your production instance to `retool/prod`. You can then reference secrets for either deployment instance using `{{ secrets.db_user }}` and `{{ secrets.db_password }}`, and Retool will fetch the appropriate secret at runtime.

import ConfigCheck from './_config-check.mdx';

## 4. Use secrets in resources

After you save your Secrets Manager configuration in Retool, if you granted Retool the `ListSecrets` permission, available secrets appear on **Settings** > **Secrets Manager**.

You can reference them in Resource configurations using template syntax, e.g. `{{ secrets.name }}` or `{{ secrets['name'] }}`, either by copying values from the **Reference** column, or with autocomplete on the Resource configuration page.

You can access JSON keys with `{{ secrets.name.key }}` or `{{ secrets['name']['key'] }}`. You can access arbitrary levels of nested keys in your secrets. Note that keys will not autocomplete.

:::info

If your secret values are not quoted, you may need to wrap the template string in quotes in resource configurations.

For example, given an unquoted secret (`secret-value`) for the key `key_name`, you'd use `"{{ secrets.key_name.key }}"`. Given a quoted secret (`"secret-value"`), you'd use `{{ secrets.key_name.key }}`.

:::

When you rotate secrets in AWS, Retool automatically reads updated values from your secrets manager. This means you don't need to restart your Retool instance or update any configuration when rotating secrets.

## Time to live (TTL) setting

Retool caches secrets it fetches with a configurable time to live. This reduces API calls to your secrets manager, which keeps queries fast and reduces costs.
However, this means secrets may become temporarily "stale" when they are rotated. You can update **Cache TTL (ms)** in **Settings** > **Secrets Manager** to minimize failed queries in the event that a secret is rotated.

## Security considerations

Any user that can configure resources can use secrets in resources. This could lead to inappropriate uses of secrets. To mitigate risk, you can make use of the `secretsmanager:GetSecretValue` and `secretsmanager:DescribeSecret` actions via IAM policies.

---

## Manage secrets with environment variables

import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

You can use environment variables to manage secrets like database passwords and bearer tokens. This allows you to automatically rotate secret credentials, and help protect sensitive information.

If your use case requires higher level of security, you may want to use [AWS Secrets Manager](./aws.mdx) or [HashiCorp Vault](./hashicorp-vault.mdx) to externally manage secrets. These integrations allow you to rotate secrets without redeploying your Retool instance.

To manage secret and non-secret environment variables through Retool's web interface, consider using [configuration variables](../../../org-users/guides/configuration/config-vars.mdx).

## 1. Define the RETOOL_EXPOSED_DB_PASSWORD variable

Set the `RETOOL_EXPOSED_DB_PASSWORD` variable to your database connection string. Depending on how you set up Retool, you might need to restart the Docker container.

:::note

To avoid leaking potentially sensitive environment variables, Retool only allows users to read environment variables with the `RETOOL_EXPOSED_` prefix.

:::

## 2. Create a new Postgres connection

Add a new Postgres connection and use the connection string format. Make sure to replace the database password with the `%RETOOL_EXPOSED_DB_PASSWORD%` variable. It should look something like this.

You can use environment variables like this for any field that you define. For example, you could use environment variables to configure the headers you send in API requests.

## 3. Save and restart the container

Click **Save** and restart the container.

---

## Manage secrets with the file system

import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

Some deployment systems, like Docker swarm and Docker secrets, require secrets values to be read from the file system instead of being set through environment variables. For instance, instead of setting a `POSTGRES_PASSWORD` in your environment, you point `POSTGRES_PASSWORD_FILE` to a text file that contains the password.

## 1. Set the `RETOOL_LOAD_FILE_SECRETS` environment variable

:::caution

You must have the ability to set non-secret environment variables.

:::

At startup, Retool looks for the configured secret files and sets them as environment variables for running the container, not the entire system. Make sure to set the `RETOOL_LOAD_FILE_SECRETS` environment variable to `true`. Without this, Retool doesn't load secrets from the file system.

## 2. Configure environment variables

Instead of adding secrets to the environment directly, add the path to where your secrets are stored on the file system and append `_FILE` to the end of each environment variable name. For example, if you want to set the `ENCRYPTION_KEY`, you’d set `ENCRYPTION_KEY_FILE` to the path on the file system where the file exists.

```text
RETOOL_LOAD_FILE_SECRETS=true
ENCRYPTION_KEY_FILE=/path/to/key
```

When starting Retool, you can check the logs to verify the environment variables are set:

```
RETOOL-CONFIG: RETOOL_LOAD_FILE_SECRETS is true, reading the following secrets from the filesystem
RETOOL-CONFIG: Setting ENCRYPTION_KEY via /path/to/key
```

If you don't see messages similar to this, see the [troubleshooting](#troubleshooting) section.

## Manage your own secrets

Retool supports managing your own secrets using environment variables prefixed with `RETOOL_EXPOSED`. You can use the file system to manage these secrets too.

Instead of using the `RETOOL_EXPOSED` prefix, use `RETOOL_FILE_EXPOSED`. For example, if you want multiple resources to use your database password, set `RETOOL_FILE_EXPOSED_DB_PASSWORD` to the path on the file system.

```text
RETOOL_LOAD_FILE_SECRETS=true
RETOOL_FILE_EXPOSED_DB_PASSWORD=/path/to/db/password
```

## Troubleshooting

| **Error**                                                                                  | **Resolution**                                                                                                                                                                                                                                |
| ------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `RETOOL-CONFIG: Error setting SECRET via SECRET_FILE: /path/to/secret_file does not exist` | This error means the path in the `SECRET_FILE` environment variable doesn't exist. Double check the path is accessible to the container running Retool at build time.                                                                         |
| `RETOOL-CONFIG: Error setting SECRET via SECRET_FILE: /path/to/secret_file is a directory` | This means the path provided in the `SECRET_FILE` environment variable exists, but isn't a file and can't be used to set the secret. Ensure the path provided in the `SECRET_FILE` environment variable points to a file and not a directory. |

---

## Retrieve secrets from GCP Secrets Manager

import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

Configuring Resources in Retool can require handling sensitive values, e.g., database passwords or API keys. Retool is [SOC 2 Type 2 compliant](https://docs.retool.com/page/security), and most customers store these values with Retool. However, depending on your security posture, you may want to store secret values externally, rather than encrypted in Retool’s database. To support this, you can store and retrieve secrets from GCP Secrets Manager.

## Requirements

To use the GCP Secrets Manager with Retool, you must:

- Run self-hosted Retool on versions 3.20 or later on the Enterprise plan.
- Have [admin permissions](../../../permissions/guides.mdx) for your Retool organization.
- Already have GCP Secrets Manager configured to store your secrets.

## 1. Set up Application Default Credentials

In GCP, [set up application default credentials](https://cloud.google.com/docs/authentication/provide-credentials-adc?hl=en) to grant Retool access to your secrets.

### Workload Identity on Google Kubernetes Engine (GKE)

If you are using Google Kubernetes Engine to host Retool, you can use [workload identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity) to set up application default credentials. Retool already creates a service account called `retool` in the GKE cluster. To complete the setup, add your IAM service account annotation to this pre-existing service account.

Alternatively, to automate this process, you can inject the annotations via `values.yaml` in the Retool Helm chart:

```yaml
serviceAccount:
  create: true
  name:
  annotations:
    iam.gke.io/gcp-service-account: ${gke_service_account}
```

## 2. Configure Secrets Manager in Retool
Secrets management must be configured for the primary organization, and for any spaces within an organization that need to use secrets.

import MultiInstance from './_multi-instance.mdx';

For each [Retool Space](../../../org-users/tutorials/spaces), navigate to **Settings** > **Secrets Manager** and enter the [Project ID and Project Number](https://cloud.google.com/resource-manager/docs/creating-managing-projects#identifying_projects) associated with your GCP Secrets Manager instance, and an optional **Namespace**. Click **Test connection** to confirm Retool can connect to your GCP Secrets Manager, then click **Save**.

### Namespace

The **Namespace** field is a value that's prepended to any secret names Retool looks up. For example, if all your Retool-facing secrets in GCP Secrets Manager start with `retool-`, you might set the namespace to be `retool-` as well. If you have multiple Retool deployment instances, you can use this to resolve the same secret name in a resource configuration to different secrets in your GCP account. For this reason, setting the namespace is recommended if you use [Protected Resources](../../../source-control/guides/protect/resources.mdx).

For example, if you have "development" and "production" deployment instances, you might use the following naming convention for secrets in your GCP account:

```
// Secrets for development
retool-dev-db_user
retool-dev-db_password

// Secrets for production

retool-prod-db_user
retool-prod-db_password
```

On your Retool development instance, you would then set the namespace to `retool/dev`, and on your production instance to `retool/prod`. You can then reference secrets for either deployment instance using `{{ secrets.db_user }}` and `{{ secrets.db_password }}`, and Retool will fetch the appropriate secret at runtime.

import ConfigCheck from './_config-check.mdx';

## 3. Use secrets in resources

After you save your Secrets Manager configuration in Retool, if you granted Retool the Secret Manager Admin or Secret Manager Viewer IAM roles, available secrets appear on **Settings** > **Secrets Manager**.

For users to access secret values, the Secret Manager Admin or Secret Manager Secret Accessor IAM roles must be used. You can reference them in resource configurations using template syntax, e.g., `{{ secrets.name }}` or `{{ secrets['name'] }}`, either by copying values from the **Reference** column, or with autocomplete on the resource configuration page.

You can access JSON keys with `{{ secrets.name.key }}` or `{{ secrets['name']['key'] }}`. You can access arbitrary levels of nested keys in your secrets. Note that keys will not autocomplete.

:::info

If your secret values are not quoted, you may need to wrap the template string in quotes in resource configurations.

For example, given an unquoted secret (`secret-value`) for the key `key_name`, you'd use `"{{ secrets.key_name.key }}"`. Given a quoted secret (`"secret-value"`), you'd use `{{ secrets.key_name.key }}`.

:::

When you rotate secrets in GCP, Retool automatically reads updated values from your secrets manager. This means you don't need to restart your Retool instance or update any configuration when rotating secrets.

## Time to live (TTL) setting

Retool caches secrets it fetches with a configurable time to live. This reduces API calls to your secrets manager, which keeps queries fast and reduces costs.
However, this means secrets may become temporarily stale when they are rotated. You can update **Cache TTL (ms)** in **Settings** > **Secrets Manager** to minimize failed queries in the event that a secret is rotated.

## Security considerations

Any user that can configure resources can use secrets in resources. This could lead to inappropriate uses of secrets. You can restrict the Secret Manager Admin, Secret Manager Secret Accessor, and Secret Manager Viewer IAM roles to authorized users only in GCP to mitigate the risk.

---

## Retrieve secrets from HashiCorp Vault

import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

Configuring Resources in Retool can require handling sensitive values, e.g. database passwords or API keys. Retool is [SOC 2 Type 2 compliant](https://docs.retool.com/page/security), and most customers store these values with Retool. However, depending on your security posture, you may want to store secret values externally, rather than encrypted in Retool’s database.

To support this, Retool supports retrieving secrets from HashiCorp Vault for customers running self-hosted Retool.

## Requirements

:::note

Retool only supports versioned KV (v2) engines.

:::

To use HashiCorp Vault with Retool, you need:

- A self-hosted Retool deployment
- A Retool organization on the Enterprise plan.
- [Admin permissions](../../../permissions/guides.mdx) for your Retool organization.
- Have HashiCorp Vault configured to store your secrets in a [versioned KV (v2)](https://developer.hashicorp.com/vault/docs/secrets/kv/kv-v2) engine.

## 1. Enable AppRole Auth Method

Follow the first two steps in the [Vault tutorial](https://developer.hashicorp.com/vault/tutorials/auth-methods/approle) to create a role for Retool with a policy to grant access to your secrets. The following is an example policy you might use.

```hcl
path "secret-mount/data/*" {
  capabilities = [ "read" ]
}

path "secret-mount/metadata/*" {
  capabilities = [ "list" ]
}
```

Note: If your Vault instance supports namespaces, your `AppRole` policy must exist in the same namespace as your KV engine.

## 2. Get a RoleId and SecretId

Next, follow the third step from the [Vault tutorial](https://developer.hashicorp.com/vault/tutorials/auth-methods/approle) to get a `RoleId` and `SecretId` for the role created above.

## 3. Configure Secrets Manager in Retool
Secrets management must be configured for the primary organization, and for any spaces within an organization that need to use secrets.

import MultiInstance from './_multi-instance.mdx';

For each [Retool Space](../../../org-users/tutorials/spaces), visit **Settings** > **Secrets Manager** and enter the following settings from your Vault deployment.

| Setting           | Description                                                                    | Example                                |
| ----------------- | ------------------------------------------------------------------------------ | -------------------------------------- |
| Vault URL & Port  | URL and port of your Vault server.                                             | `https://vault.mycorp.com:8200`        |
| Path to KV Engine | The name of your KV engine.                                                    | `secrets`                              |
| Role ID           | `RoleId` from Vault.                                                           | `40481de9-8281-cf4d-2056-abc5c738e392` |
| Secret ID         | `SecretId` from Vault.                                                         | `12345d5a-ead6-5f9f-c683-a571ad716668` |
| AppRole Mount     | Path at which the AppRole auth method is mounted. This defaults to `approle/`. | `auth/approle`                         |
| Vault Namespace   | Full namespace path to the desired KV engine                                   | `ns1/ns2`                              |

Click **Test connection** to confirm Retool can connect to Vault, then click **Save**.

### Retool Namespace

The **Retool Namespace** field is a value that will be prepended to any secret names Retool looks up. For example, if all your Retool-facing secrets in Vault start with `retool/`, you might set the namespace to be `retool/` as well. If you have multiple Retool deployment instances, you can use this to resolve the same secret name in a Resource configuration to different secrets in Vault. For this reason, setting the namespace is recommended if you use [Protected Resources](../../../source-control/guides/protect/resources.mdx).

For example, if you have "development" and "production" deployment instances, you might use the following naming convention for secrets in Vault:

```
// Secrets for development
retool/dev/db_user
retool/dev/db_password

// Secrets for production
retool/prod/db_user
retool/prod/db_password
```

On your Retool development instance, you would then set the namespace to `retool/dev`, and on your production instance to `retool/prod`. You can then reference secrets for either deployment instance using `{{ secrets.db_user }}` and `{{ secrets.db_password }}`, and Retool will fetch the appropriate secret at runtime.

import ConfigCheck from './_config-check.mdx';

## 4. Use secrets in resources

After you save your Secrets Manager configuration in Retool, if you granted Retool the `read` capability for the metadata, available secrets appear on **Settings** > **Secrets Manager**.

You can reference them in Resource configurations using template syntax, e.g. `{{ secrets.name.key }}` or `{{ secrets['name'].key }}`, either by copying values from the **Reference** column, or with autocomplete on the Resource configuration page.

You can access JSON keys with `{{ secrets.name.key }}` or `{{ secrets['name']['key'] }}`. You can access arbitrary levels of nested keys in your secrets. Note that keys will not autocomplete.

:::info

If your secret values are not quoted, you may need to wrap the template string in quotes in resource configurations.

For example, given an unquoted secret (`secret-value`) for the key `key_name`, you'd use `"{{ secrets.key_name.key }}"`. Given a quoted secret (`"secret-value"`), you'd use `{{ secrets.key_name.key }}`.

:::

When you rotate secrets in Vault, Retool automatically reads updated values from your secrets manager. This means you don't need to restart your Retool instance or update any configuration when rotating secrets.

## Time to live (TTL) setting

Retool caches secrets it fetches with a configurable time to live. This reduces API calls to your secrets manager, which keeps queries fast and reduces costs.
However, this means secrets may become temporarily "stale" when they are rotated. You can update **Cache TTL (ms)** in **Settings** > **Secrets Manager** to minimize failed queries in the event that a secret is rotated.

## Security considerations

Any user that can configure resources can use secrets in resources. This could lead to inappropriate uses of secrets. You can restrict the `list` capability in Vault to mitigate the risk.

---

## Self-hosted Retool secrets management guides

import DocCardList from "@theme/DocCardList";

---

## Configure multiple Secrets Manager configurations

import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

You can create multiple [Secrets Manager](./index.mdx) configurations if you need to retrieve secrets from different locations.

## Enable the multiple configurations beta

Multiple configurations for Secrets Manager is currently in public beta. Admins can enable this functionality from the **Beta** page in your Retool organization's settings.

## Create additional configurations

You configure additional secrets managers using the same options as a single configuration. You can add multiple configurations for any supported secrets manager provider.

Click **Add Secrets Manager** and then select the configuration to create. You then provide the required settings based on whether you are configuring [AWS](./aws.mdx), [HashiCorp Vault](./hashicorp-vault.mdx), or [Google Cloud Platform](./gcp.mdx).

Each configuration requires a unique string identifier (e.g., `vault_2`) so that it can be referenced across Retool.

Multiple configurations.

## Reference secrets from different configurations

How you reference secrets with multiple configurations is slightly different once you start using multiple configurations, as each configuration's identifier must now also be included in the `{{ }}` expression.

```text title="Reference multiple configurations"
{{ secrets['CONFIGURATION_NAME']['KEY_NAME']}}

{{ secrets['gcp_1']['secret3'] }}
{{ secrets['aws_1']['secret3'] }}
{{ secrets['vault_1']['secret3'] }}
```

Reference secrets from different configurations.

## Select a different default configuration

Once you add a configuration, the previous secrets manager you configured is automatically set as the default. Any existing secrets references that don't include the configuration name are parsed as aliases that point to the default configuration.

For example, if the default configuration is named `gcp_1`, you can reference a secret named `secret_1` using either:

- `{{ secrets['secret_1'] }}`
- `{{ secrets['gcp_1']['secret_1'] }}`

If necessary, you can select a different configuration to operate as the default from the **Secrets Manager** settings page. Click  to view options for the configuration and set it to the default.

If you change the default configuration to `aws_1` but currently use `{{ secrets['secret_1'] }}`, this would then point to  `{{ secrets['aws_1']['secret_1'] }}`.

:::warning

Changing the default configuration will cause any existing alias references—without the configuration identifier—to point to the newly selected default, and potentially break existing usage. Before you make changes, ensure you verify all existing references to secrets so they include the configuration name.
:::

---

## Configure and migrate to an external database

import EnterpriseRequirement from "../_partials/_enterprise-plan-requirement.mdx";

For non-production and development purposes, the default Docker Compose configuration for Self-hosted Retool includes a PostgreSQL container and does not set up SSL. This is not suitable for production use cases and you must host the Retool Storage database on an external, managed PostgreSQL database. Managed databases are more maintainable, scalable, and reliable than containerized PostgreSQL instances.

If you originally deployed an instance using the default PostgreSQL container, you can migrate its data to an external PostgreSQL database before using the instance in production.

## Prerequisites

Before migrating, ensure that your architecture meets the following recommendations and requirements:

- The minimum recommended version for the PostgreSQL database is version 13. Your PostgreSQL database must also enable the [uuid-ossp](https://www.postgresql.org/docs/current/uuid-ossp.html) module and use the Read Committed [isolation level](https://www.postgresql.org/docs/current/transaction-iso.html).
- Retool recommends allocating at least 60GB of [storage](../reference/requirements.mdx) when you set up a new Retool instance. If you're migrating an existing instance, you might need more space.
- The Retool PostgreSQL database user must have superuser privileges on the `hammerhead_production` database. This is necessary to perform essential tasks, such as installing updates.

import Tables from '../_partials/_postgres-tables.mdx';

## 1. Export data from Retool's Docker container

To export data from Retool's PostgreSQL container, run the following command on the virtual machine hosting the Retool containers, in the Retool directory.

```shell
docker compose exec postgres \
 pg_dump hammerhead_production --no-acl --no-owner --clean \
 -U  retool_internal_user -f retool_db_dump.sql
```

This dumps the data into a file named `retool_db_dump.sql` in the root of the volume used by the Retool PostgreSQL container.

To export data to your host machine, run the following command.

```shell
docker compose exec postgres \
  pg_dump hammerhead_production --no-acl --no-owner --clean \
  -U  retool_internal_user > retool_db_dump.sql
```

## 2. Migrate the data to an external hosted database

:::tip Recommended PostgreSQL version

Retool recommends using [PostgreSQL 13.7](https://www.postgresql.org/docs/release/13.7/) for the external database.

:::

To migrate the data to an external database, run the following command. Replace `$DB_CONNECTION_URI` with the PostgreSQL connection string that connects to your externally hosted database. Make sure to [correctly format the URI](https://www.postgresql.org/docs/9.3/libpq-connect.html#AEN39692).

```shell
docker compose exec postgres \
 psql $DB_CONNECTION_URI -f retool_db_dump.sql
```

## 3. Create a user with superuser privileges

Some databases grant superuser privileges to the first user added to the database, but you should check your database's documentation to learn more. Depending on your database, you might need to create a user. Some examples include the:

- `rds_superuser` for Amazon RDS.
- `cloudsqlsuperuser` for Google Cloud SQL.
- `azure_pg_admin` Azure Database for PostgreSQL.

## 4. Configure Retool to use the external database

The `docker.env` files defines several Retool options, including which database to connect to. The lines you need to edit follow this pattern:

```shell
POSTGRES_DB=hammerhead_production
POSTGRES_USER=retool_internal_user
POSTGRES_HOST=postgres
POSTGRES_PORT=5432
POSTGRES_PASSWORD={a random string}
```

Edit those variables to correspond with the database you created. For example:

```shell
POSTGRES_DB=retooldb
POSTGRES_USER=retool_user
POSTGRES_HOST=retool.xxxxxxxxxx.us-east-2.rds.amazonaws.com
POSTGRES_PORT=5432
POSTGRES_PASSWORD=xyzabc
```

Alternatively, you can delete the five environment variables and instead specify a database connection string:

```shell
DATABASE_URL=postgres://retool_user:xyzabc@retool.xxxxxxxxxx.us-east-2.rds.amazonaws.com:5432/retooldb
```

## 5. Restart the Retool server

After updating the Retool configuration, run this command from the `retool-onpremise` directory to restart the server.

```shell
sudo docker compose up -d
```

---

## Collect self-hosted telemetry data

import EnterpriseRequirement from "../_partials/_enterprise-plan-requirement.mdx";

:::warning Requirements and limitations

This feature is in development. It is currently available for self-hosted instances deployed using [retool-helm](https://github.com/tryretool/retool-helm) 6.2.0 or above, and running the most recent [Edge](../../releases/edge/index.mdx) or [Stable](../../releases/stable/index.mdx) release.

:::

Organizations with Self-hosted deployment instances can collect telemetry data using either:

- A [Retool-provided](#configure-retool-telemetry-collector) observability agent.
- A [self-managed](#configure-self-managed-observability-agents) observability agent.

Retool's observability agent can forward data to both Retool and custom destinations. Retool also supports using your own observability agent if you prefer to have complete control over telemetry data collection.

Telemetry data collection is not enabled by default. You must configure your deployment instance to start collecting and forwarding telemetry data.

## Configure Retool telemetry collector

:::tip Improve support with health monitoring

When telemetry data is forwarded to Retool, your deployment's health is continually monitored. This allows Retool to have more insight into potential issues and improves the level of support when diagnosing issues.

:::

Use the Helm CLI or update your Helm configuration file to enable telemetry collection. This will send data to Retool by default. Set `sendToRetool` to `false` to disable this if you do not want to send data to Retool.

```shell
helm upgrade --set telemetry.enabled=true telemetry.sendToRetool.enabled=false...
```

```yaml
...
telemetry:
  enabled: true
  sendToRetool:
    enabled: false
...
```

The telemetry image uses the same release version as the main backend by default. If necessary, you can specify a version tag to use using the `image.tag` option:

```yaml
telemetry:
  image:
    tag: 3.52.0-stable
```

If set, the telemetry image is fixed to the specified tag. Retool does not recommend including a tag unless you have a specific use case.

### Collection and forwarding

The **telemetry collector** container contains two services: [grafana-agent](https://grafana.com/docs/agent) and [vector](https://vector.dev/docs). You can configure `vector` to send data to either Retool or to custom destinations.

**telemetry collector** uses a secure TLS connection with short-lived client certificates when sending data to Retool. Data is securely stored on Amazon S3 buckets in `us-west-2` and not shared with any other third-parties or subprocessors.

### Types of telemetry data

When enabled, your deployment produces the following types telemetry data:

- Container Metrics (CPU, memory, network usage)
- Retool Runtime Metrics (frontend performance, backend request counts and latency)
- Container Logs (request logs, error logs, info logs)

| Source name          |       Sent to Retool        | Description                                                                                                                                 |
| -------------------- | :-------------------------: | ------------------------------------------------------------------------------------------------------------------------------------------- |
| `metrics_statsd`     |   | Retool internal metrics. This includes frontend performance, backend request count, latency, etc.                                           |
| `metrics_statsd_raw` |   | Same as `metrics_statsd`, but without any identifying tags added by the telemetry collector.                                                |
| `metrics`            |   | All collected metrics. This includes container health metrics and all metrics from `metrics_statsd`.                                        |
| `container_logs`     |   | All logs from the containers in the Retool deployment, except `audit_logs` and `debug_logs` excluded and deployment identifying tags added. |
| `container_logs_raw` |   | All logs from the containers in the Retool deployment, without any exclusion, tagging, or other processing done.                            |
| `audit_logs`         |  | Retool audit logs which are printed to container stdout, if any. Requires the relevant config to enable that feature.                       |
| `debug_logs`         |  | Debug level logs, if any. These are separated so as to avoid accidentally forwarding high volumes of debug logs to destinations.            |

### Send telemetry data to custom destinations

Retool supports sending to any custom destination supported by Vector. Refer to the [Vector sinks reference documentation](https://vector.dev/docs/reference/configuration/sinks/) for a complete list of supported sink types and configuration.

You specify custom destinations using the `customVectorConfig` variable with `sink` configurations. Each sink must include a list of [telemetry sources](#types-of-telemetry-data) for which it forwards.

```yaml
...
telemetry:
  customVectorConfig:
    sinks: ...
...
```

### Example configuration for Datadog

The following example illustrates a telemetry configuration where data is forward to Retool and [Datadog](https://datadog.com).

```yaml
...
telemetry:
  extraEnv:
    - name: DD_AGENT_HOST
      valueFrom:
        fieldRef:
          fieldPath: status.hostIP
  customVectorConfig:
    sinks:
      # forward statsd metrics to datadog-agent port 8125
      metrics_datadog:
        address: ${DD_AGENT_HOST}:8125
        buffer:
          when_full: drop_newest
        inputs:
          - metrics_statsd_raw
        mode: udp
        type: statsd
  enabled: true
...
```

### Example configuration for Prometheus Remote Write

The following example illustrates a telemetry configuration where data is forward to Retool and a [Prometheus Remote Write](https://vector.dev/docs/reference/configuration/sinks/prometheus_remote_write/) destination.

```yaml
...
telemetry:
  customVectorConfig:
    sinks:
      metrics_prometheus:
        type: prometheus_remote_write
        endpoint: https://prometheus:8087/api/v1/write
        inputs:
          - metrics
        buffer:
          when_full: drop_newest
  enabled: true
...
```

## Configure self-managed observability agents

Configuring a self-managed agent depends on a number of factors. Use the following information to configure your agent for telemetry data collection.

### Logs

Your agent should collect logs in the same way as any other container.

#### Format

Most Retool container logs are JSON-formatted. Retool recommends configuring your log collector to parse logs as JSON first, but falling back to a simple string message format upon failure.

#### Level

Set the [`LOG_LEVEL`](../reference/environment-variables/index.mdx#log_level) environment variable if you need to adjust log volume and verbosity.

#### Debugging

If you need to troubleshoot log collection, enable debug logs with the [`DEBUG`](../reference/environment-variables/index.mdx#debug) environment variable. This will result in a very large number of logs. You should only use it for troubleshooting purposes.

### Metrics

Not all emitted metrics are currently documented while telemetry data is in development as they may change in future versions. Current metrics cover various internal Retool runtime health metrics, such as frontend performance timings, resource query timings and error rates, internal cache sizes, workflow execution rates and timings, etc.

For more information about telemetry metrics, refer to the [Temporal documentation](https://docs.temporal.io/develop/worker-performance).

#### main-backend container

Retool **backend** containers are instrumented to emit metrics in the [DogStatsD](https://docs.datadoghq.com/developers/dogstatsd/datagram_shell?tab=metrics) format. To collect these metrics, you must configure the agent with a `statsd` UDP listener which is specifically DogStatsD-aware.

Set the `STATSD_HOST` environment variable to the IP or DNS name of your agent. If your agent is using a port other than `8125` (the default for most agents), set the `STATSD_PORT` environment variable to the correct port number.

#### workflow-worker container

In addition to `statsd` metrics , the **workflow-worker** container is also instrumented to also emit Temporal SDK metrics to an OpenTelemetry (OTLP) gRPC collector. Temporal SDK metrics can help you scale your deployment to keep up with Workflows traffic by tracking metrics such as queue latency.

If necessary, set the `WORKFLOW_TEMPORAL_OPENTELEMETRY_COLLECTOR` environment variable to the address of your OTLP gRPC endpoint. For example:

```shell
WORKFLOW_TEMPORAL_OPENTELEMETRY_COLLECTOR=http://localhost:4317
```

### Tracing

Retool supports Datadog for collecting backend traces. If you use the Datadog agent, set the `DD_TRACE_ENABLED` environment variable to `true`.

In some cases, you may need to configure additional trace collection options, such as:

- Setting the trace agent hostname with the `DD_TRACE_AGENT_HOSTNAME` environment variable.
- Adjusting the sample rate with the `DD_TRACE_SAMPLE_RATE` environment variable.

Retool uses the [dd-trace](https://www.npmjs.com/package/dd-trace) library and supports all available configuration parameters.

---

## Enable Retool-managed Vectors on self-hosted deployments

import EnterpriseRequirement from "../_partials/_enterprise-plan-requirement.mdx";

Organizations with self-hosted Retool deployments must configure and enable Retool Vectors before use.

## Install dependencies

To use Retool Vectors for self-hosted deployments:

- Configure [Retool Database](retool-database.mdx) on your self-hosted deployment.
- Install [pgvector](https://github.com/pgvector/pgvector), an open-source PostgreSQL extension for similarity search, in the same database cluster as Retool Database. Ensure your `pgvector` version is compatible with your PostgreSQL version.
- If you're using Amazon RDS, you also need to run [PostgreSQL 15.2 or later](https://aws.amazon.com/about-aws/whats-new/2023/05/amazon-rds-postgresql-pgvector-ml-model-integration/).

:::warning

Make sure to use Retool Database (often the `retooldb-postgres` container), not the Retool Storage database (often `postgres`). The storage database tracks app info, users, audit logs, etc. Retool Database is a PostgreSQL database you can use in apps and workflows, and has a UI for creating and editing tables.

:::

## Verify installation

Once you have enabled Retool Database and `pgvector`, you can start using Vectors. 

To verify that `pgvector` is installed correctly and the role you are using for Retool Database can create tables, run the following SQL statement in a [query](../../data-sources/quickstarts/retool-database.mdx#query-your-data):

```sql
CREATE TABLE sample_vector_table (id serial PRIMARY KEY, embedding vector(1536));
```

## Configure environment variables

:::important

The following section is required only if your deployment is Self-hosted *and* you are using a Retool-managed AI model. If you are using a self-managed model, you can skip this section.

:::

import Partial from '../../_shared/_retool-managed-self-hosted.mdx';

## Enable Vectors

import Enable from '/docs/_shared/_enable-vectors.mdx';

## Error handling
If you are experiencing issues creating vectors, connect to Retool Database using the available [connection strings](../../data-sources/guides/retool-database/external-access). 

1. Click the  next to **Database**. 
2. Click **Connection** and copy the connection string you want to use (PSQL or Postgres).
3. Use the connection string to connect to the database created.
4. Retry creating the Vector resource in Retool.

---

## Upgrade a legacy Azure Virtual Machines deployment to support Retool Workflows

import Legacy from "../../_partials/_workflows-legacy.mdx"
import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

## Requirements

Your self-hosted deployment must meet the following requirements for Retool Workflows.

### VM configuration

Retool Workflows requires a Linux-based virtual machine that meets the following system requirements:

import VmConfiguration from "../../_partials/_vm-configuration.mdx";

### Retool version

import Version from "../../_partials/_workflows-legacy-version.mdx"

import Temporal from "../../_partials/_workflows-legacy-temporal.mdx"

## System architecture changes

import Architecture from "../../_partials/_architecture-diagram.mdx"

## 1. Upgrade Retool

import DockerUpdate from "../../_partials/_update.mdx";

## 2. Configure Temporal

import Docker from "../../_partials/_workflows-legacy-docker.mdx"

---

## Upgrade a legacy Docker deployment to support Retool Workflows

import Legacy from "../../_partials/_workflows-legacy.mdx"
import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

## Requirements

Your self-hosted deployment must meet the following requirements for Retool Workflows.

### VM configuration

Retool Workflows requires a Linux-based virtual machine that meets the following system requirements:

import VmConfiguration from "../../_partials/_vm-configuration.mdx";

### Retool version

import Version from "../../_partials/_workflows-legacy-version.mdx"

import Temporal from "../../_partials/_workflows-legacy-temporal.mdx"

## System architecture changes

import Architecture from "../../_partials/_architecture-diagram.mdx"

## 1. Upgrade Retool

import DockerUpdate from "../../_partials/_update.mdx";

## 2. Configure Temporal

import Docker from "../../_partials/_workflows-legacy-docker.mdx"

---

## Upgrade a legacy Amazon EC2 deployment to support Retool Workflows

import Legacy from "../../_partials/_workflows-legacy.mdx"
import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

## Requirements

Your self-hosted deployment must meet the following requirements for Retool Workflows.

### VM configuration

Retool Workflows requires a Linux-based virtual machine that meets the following system requirements:

import VmConfiguration from "../../_partials/_vm-configuration.mdx";

### Retool version

import Version from "../../_partials/_workflows-legacy-version.mdx"

import Temporal from "../../_partials/_workflows-legacy-temporal.mdx"

## System architecture changes

import Architecture from "../../_partials/_architecture-diagram.mdx"

## 1. Upgrade Retool

import DockerUpdate from "../../_partials/_update.mdx";

## 2. Configure Temporal

import Docker from "../../_partials/_workflows-legacy-docker.mdx"

---

## Upgrade a legacy Google Compute Engine deployment to support Retool Workflows

import Legacy from "../../_partials/_workflows-legacy.mdx"
import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

## Requirements

Your self-hosted deployment must meet the following requirements for Retool Workflows.

### VM configuration

Retool Workflows requires a Linux-based virtual machine that meets the following system requirements:

import VmConfiguration from "../../_partials/_vm-configuration.mdx";

### Retool version

import Version from "../../_partials/_workflows-legacy-version.mdx"

import Temporal from "../../_partials/_workflows-legacy-temporal.mdx"

## System architecture changes

import Architecture from "../../_partials/_architecture-diagram.mdx"

## 1. Upgrade Retool

import DockerUpdate from "../../_partials/_update.mdx";

## 2. Configure Temporal

import Docker from "../../_partials/_workflows-legacy-docker.mdx"

---

## Add Retool Workflows to a legacy Kubernetes with Helm deployment

import Legacy from "../../_partials/_workflows-legacy.mdx"
import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

import WorkflowsHelmRetoolManagedTemporal from "../../_partials/_workflows-helm-retool-managed-temporal.mdx"
import WorkflowsHelmSelfHostedTemporal from "../../_partials/_workflows-helm-self-hosted-temporal.mdx"
import WorkflowsHelmLocalTemporal from "../../_partials/_workflows-helm-local-temporal.mdx"

## Requirements

Your self-hosted deployment must meet the following requirements for Retool Workflows.

### VM configuration

Retool Workflows requires a Linux-based virtual machine that meets the following system requirements:

import VmConfiguration from "../../_partials/_vm-configuration.mdx";

### Retool version

import Version from "../../_partials/_workflows-legacy-version.mdx"

import Temporal from "../../_partials/_workflows-legacy-temporal.mdx"

## System architecture changes

import Architecture from "../../_partials/_architecture-diagram.mdx"

## 1. Upgrade your deployment

import Update from "../../_partials/_update-helm-retool.mdx"

## 2. Configure deployment

---

## Upgrade and configure a legacy deployment with Retool Workflows

import Legacy from "../../_partials/_workflows-legacy.mdx"

Use one of the following guides to upgrade a legacy deployment of self-hosted Retool and add support for Retool Workflows.

import DocCardList from "@theme/DocCardList";

---

## Upgrade a legacy deployment to support Retool Workflows

import Legacy from "../../_partials/_workflows-legacy.mdx"
import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

## Requirements

Your self-hosted deployment must meet the following requirements for Retool Workflows.

### VM configuration

Retool Workflows requires a Linux-based virtual machine that meets the following system requirements:

import VmConfiguration from "../../_partials/_vm-configuration.mdx";

### Retool version

import Version from "../../_partials/_workflows-legacy-version.mdx"

import Temporal from "../../_partials/_workflows-legacy-temporal.mdx"

## System architecture changes

import Architecture from "../../_partials/_architecture-diagram.mdx"

## 1. Upgrade your deployment 

Follow these instructions to update your Retool instance to a newer release version. 

:::caution Back up the instance before performing an update

Retool strongly recommends that you {props.url ? back up the VM : "back up the VM" } before performing an update. If you cannot complete a full backup, you should at least:

- Create a snapshot of your PostgreSQL database.
- Copy the environment variables in `docker.env` to a secure location outside of Retool.

:::

import SelfhostedLatest from "@site/src/components/SelfhostedLatest";

To update your deployment to a newer version of Self-hosted Retool:

1. Update the `Dockerfile` with the newer version number. For example: tryretool/backend:
2. If your deployment is using `CodeExecutor.Dockerfile`, update it with the same version.
3. Run `./upgrade.sh` to perform the update.

Apply the updates and restart the deployment instance. The Retool instance is temporarily stopped while the update takes place and restarts automatically. Retool recommends performing the upgrade during off-peak hours to minimize downtime for users.

## 2. Configure new and existing services

Complete the following steps to configure Temporal.

  

:::caution Egress and ingress

[code-executor](../../concepts/architecture.mdx#code-executor) must have network access to [workflows-backend](../../concepts/architecture.mdx#workflows-backend) and **Temporal**.

:::

Provision the [code-executor](../../concepts/architecture.mdx#code-executor) container using the `tryretool/code-executor-service` image. You must use the same version as `tryretool/backend`. For example:

- tryretool/backend:
- tryretool/code-executor-service:

:::caution Egress and ingress

[workflows-worker](../../concepts/architecture.mdx#workflows-worker) must have network access to [code-executor](../../concepts/architecture.mdx#code-executor), **Temporal**, [postgres](../../concepts/architecture.mdx#postgres), and [workflows-backend](../../concepts/architecture.mdx#workflows-backend).

:::

Provision the [workflows-worker](../../concepts/architecture.mdx#workflows-worker) container. This container runs `tryretool/backend` with the following environment variables:

| Variable                       | Description                                                                                                                                                                                                |
| ------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `SERVICE_TYPE`                 | Set to `WORKFLOW_TEMPORAL_WORKER`.                                                                                                                                                                         |
| `ENCRYPTION_KEY`               | Required for encrypting traffic to Retool-managed Temporal Cluster. If you've already set this, there is no need to change or update. Should be set to the same value across all services that require it. |
| `WORKFLOW_BACKEND_HOST`        | Url for endpoint of [workflows-backend](../../concepts/architecture.mdx#workflows-backend). Must include protocol (e.g., `https://workflows-backend.example.com`)                             |
| `CODE_EXECUTOR_INGRESS_DOMAIN` | Url for endpoint of `code-executor`service. Must include protocol `https://code-executor.example.com:3004`)                                                                                                |
| `DISABLE_DATABASE_MIGRATIONS`  | Set to `true`                                                                                                                                                                                              |

:::caution Egress and ingress

[workflows-backend](../../concepts/architecture.mdx#workflows-backend) must have network access to [postgres](../../concepts/architecture.mdx#postgres), **Temporal**, and [workflows-worker](../../concepts/architecture.mdx#workflows-worker).

:::

Provision the [workflows-backend](../../concepts/architecture.mdx#workflows-worker) container. This would be a container running `tryretool/backend` image with the following environment variables:

| Variable                       | Description                                                                                                                                                                                                |
| ------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `SERVICE_TYPE`                 | Set to `WORKFLOW_BACKEND`                                                                                                                                                                                  |
| `ENCRYPTION_KEY`               | Required for encrypting traffic to Retool-managed Temporal Cluster. If you've already set this, there is no need to change or update. Should be set to the same value across all services that require it. |
| `WORKFLOW_BACKEND_HOST`        | Url for endpoint of [workflows-backend](../../concepts/architecture.mdx#workflows-backend). Must include protocol (`http` or `https`)                                                                                                      |
| `CODE_EXECUTOR_INGRESS_DOMAIN` | Url for endpoint of `code-executor`service. Must include protocol (`http` or `https`)                                                                                                          |
| `DISABLE_DATABASE_MIGRATIONS`  | Set to `true`                                                                                                                                                                                              |

:::caution Egress and ingress

[api](../../concepts/architecture.mdx#api) must have network access to [code-executor](../../concepts/architecture.mdx#code-executor) and **Temporal**.

:::

Update the [api](../../concepts/architecture.mdx#api) container to include the following environment variables:

| Variable                       | Description                                                                                                                                                                                                |
| ------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `ENCRYPTION_KEY`               | Required for encrypting traffic to Retool-managed Temporal Cluster. If you've already set this, there is no need to change or update. Should be set to the same value across all services that require it. |
| `WORKFLOW_BACKEND_HOST`        | The [workflows-backend](../../concepts/architecture.mdx#workflows-backend) endpoint URL. Must include protocol (`http` or `https`)                                                            |
| `CODE_EXECUTOR_INGRESS_DOMAIN` | The [code-executor](../../concepts/architecture.mdx#code-executor) endpoint URL. Must include protocol (`http` or `https`)                                                                    |

  

If you haven't already, either sign up for [Temporal Cloud](https://cloud.temporal.io/) (if you haven't already) or spin up a [self-hosted Temporal](https://docs.temporal.io/production-deployment) cluster.

:::caution Egress and ingress

[code-executor](../../concepts/architecture.mdx#code-executor) must have network access to [api](../../concepts/architecture.mdx#api) and **Temporal**.

:::

Provision the [code-executor](../../concepts/architecture.mdx#code-executor) container using the `tryretool/code-executor-service` image. You must use the same version as `tryretool/backend`. For example:

- tryretool/backend:3.33.10-stable
- tryretool/code-executor-service:3.33.10-stable

Specify the following environment variables:

| Variable       | Description                                                                                              |
| -------------- | -------------------------------------------------------------------------------------------------------- |
| `NODE_ENV`     | `production`                                                                                             |
| `NODE_OPTIONS` | Used to specify the maximum heap size for the JavaScript v8 engine. Set to `--max_old_space_size=1024`. |

:::caution Egress

[workflows-worker](../../concepts/architecture.mdx#workflows-worker) must have network egress to [code-executor](../../concepts/architecture.mdx#code-executor), **Temporal**, and [workflows-backend](../../concepts/architecture.mdx#workflows-backend).

:::

Provision the [workflows-worker](../../concepts/architecture.mdx#workflows-worker) service. This would be a container running `tryretool/backend` w/ `SERVICE_TYPE=WORKFLOW_TEMPORAL_WORKER` and the following environment variables:

| Variable                                  | Description                                                                                                                                                                                                |
| ----------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `ENCRYPTION_KEY`                          | Required for encrypting traffic to Retool-managed Temporal Cluster. If you've already set this, there is no need to change or update. Should be set to the same value across all services that require it. |
| `WORKFLOW_BACKEND_HOST`                   | Url for endpoint of [workflows-backend](../../concepts/architecture.mdx#workflows-backend). Must include protocol (`http` or `https`)                                                                                                      |
| `CODE_EXECUTOR_INGRESS_DOMAIN`            | Url for endpoint of `code-executor`service. Must include protocol (`http` or `https`)                                                                                                          |
| `DISABLE_DATABASE_MIGRATIONS`             | Set to `true`                                                                                                                                                                                              |
| `WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_HOST` | The frontend host of the Temporal Cluster.                                                                                                                                                                 |
| `WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_PORT` | The port with which to connect to the Temporal Cluster. Defaults to `7233`.                                                                                                                                |
| `WORKFLOW_TEMPORAL_TLS_ENABLED`           | (Optional) Whether to enable mTLS. Required if using Temporal Cloud.                                                                                                                                       |
| `WORKFLOW_TEMPORAL_TLS_CRT`               | (Optional) The base64-encoded mTLS certificate. Required if using Temporal Cloud.                                                                                                                          |
| `WORKFLOW_TEMPORAL_TLS_KEY`               | (Optional) The base64-encoded mTLS key. Required if using Temporal Cloud.                                                                                                                                  |

:::caution Egress and ingress

[workflows-backend](../../concepts/architecture.mdx#workflows-backend) must have:

- Ingress from [code-executor](../../concepts/architecture.mdx#code-executor).
- Egress to [code-executor](../../concepts/architecture.mdx#code-executor), [postgres](../../concepts/architecture.mdx#postgres), [workflows-worker](../../concepts/architecture.mdx#workflows-worker).
:::

Provision a `workflows-backend` service. This would be a container running `tryretool/backend` w/ `SERVICE_TYPE=WORKFLOW_BACKEND` and the following environment variables:

| Variable                                  | Description                                                                                                                                                                                                |
| ----------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `ENCRYPTION_KEY`                          | Required for encrypting traffic to Retool-managed Temporal Cluster. If you've already set this, there is no need to change or update. Should be set to the same value across all services that require it. |
| `WORKFLOW_BACKEND_HOST`                   | Url for endpoint of [workflows-backend](../../concepts/architecture.mdx#workflows-backend). Must include protocol (`http` or `https`)                                                                                                      |
| `CODE_EXECUTOR_INGRESS_DOMAIN`            | Url for endpoint of `code-executor`service. Must include protocol (`http` or `https`)                                                                                                          |
| `DISABLE_DATABASE_MIGRATIONS`             | Set to `true`                                                                                                                                                                                              |
| `WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_HOST` | The frontend host of the Temporal Cluster.                                                                                                                                                                 |
| `WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_PORT` | The port with which to connect to the Temporal Cluster. Defaults to `7233`.                                                                                                                                |
| `WORKFLOW_TEMPORAL_TLS_ENABLED`           | (Optional) Whether to enable mTLS. Required if using Temporal Cloud.                                                                                                                                       |
| `WORKFLOW_TEMPORAL_TLS_CRT`               | (Optional) The base64-encoded mTLS certificate. Required if using Temporal Cloud.                                                                                                                          |
| `WORKFLOW_TEMPORAL_TLS_KEY`               | (Optional) The base64-encoded mTLS key. Required if using Temporal Cloud.                                                                                                                                  |

Update the existing [api](../../concepts/architecture.mdx#api) container (`SERVICE_TYPE=MAIN_BACKEND`) with the following environment variables:

| Variable                                  | Description                                                                                                                                                                                                |
| ----------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `ENCRYPTION_KEY`                          | Required for encrypting traffic to Retool-managed Temporal Cluster. If you've already set this, there is no need to change or update. Should be set to the same value across all services that require it. |
| `WORKFLOW_BACKEND_HOST`                   | [workflows-backend](../../concepts/architecture.mdx#workflows-backend) endpoint URL. Must include `http` or `https`.                                                                          |
| `CODE_EXECUTOR_INGRESS_DOMAIN`            | `code-executor` endpoint URL. Must include `http` or `https`.                                                                                                                                              |
| `WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_HOST` | The frontend host of the Temporal Cluster.                                                                                                                                                                 |
| `WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_PORT` | The port with which to connect to the Temporal Cluster. Defaults to `7233`.                                                                                                                                |
| `WORKFLOW_TEMPORAL_TLS_ENABLED`           | (Optional) Whether to enable mTLS. Required if using Temporal Cloud.                                                                                                                                       |
| `WORKFLOW_TEMPORAL_TLS_CRT`               | (Optional) The base64-encoded mTLS certificate. Required if using Temporal Cloud.                                                                                                                          |
| `WORKFLOW_TEMPORAL_TLS_KEY`               | (Optional) The base64-encoded mTLS key. Required if using Temporal Cloud.                                                                                                                                  |

:::caution

Retool does not recommend deploying a local Temporal Cluster alongside your Retool deployment instance. 

:::

---

## Self-hosted Retool how-to guides

import Howto from '/docs/_partials/_doctypes/_howto.mdx';
import EnterpriseRequirement from "./_partials/_enterprise-plan-requirement.mdx";

:::tip Get started with self-hosted Retool

Follow a [tutorial](./tutorial/) to deploy self-hosted Retool on your infrastructure.

:::

---

## Self-hosted Retool documentation

import EnterpriseRequirement from "./_partials/_enterprise-plan-requirement.mdx";

Organizations can deploy a self-hosted Retool instance on their own cloud infrastructure.

---

## Self-hosted Retool quickstart

import SelfhostedLatest from "@site/src/components/SelfhostedLatest";
import EnterpriseRequirement from "./_partials/_enterprise-plan-requirement.mdx";

This guide serves as an introduction to [self-hosted Retool](https://retool.com/self-hosted). It covers many of the concepts and terminology you would come across when deploying and operating a self-hosted instance of Retool. After reading this page, you should have a good understanding of self-hosted Retool.

import DocCardList from "@theme/DocCardList";

Each instance uses a distributed set of containers with services for different functions. These work together to securely run the platform within your VPN or VPC.

:::tip Learn more about self-hosted architecture

Read the [system architecture](concepts/architecture.mdx) guide to learn more about the containers, services, and dependencies for self-hosted deployment instances.

:::

import Requirements from "./_partials/_deployment-requirements.mdx";

## Requirements

### Network

Deployments must be allowed access to Retool's IP addresses or domains. If you make use of outbound firewall rules, include the following IP addresses or domains in its allowlist. These allow your deployment to connect to Retool's license check, user authentication, and usage reporting services.

import Ips from "./_partials/_ips.mdx";

import Domains from "./_partials/_domains.mdx";

Each data source you use with Retool is represented by a resource, such as APIs and external databases. Your deployment instance must have access to any of its resources to interact with data.

Refer to the [requirements](./reference/requirements.mdx) documentation for more details on network requirements.

## Architecture

Each instance uses a distributed set of containers with services for different functions. These work together to securely run the platform within your VPN or VPC.

### Containers and services

A standard deployment instance uses the following containers and services that interact with each other. The `SERVICE_TYPE` [environment variable](./reference/environment-variables/index.mdx#service_type) values determine what services a container runs.

| Container                               | Image                   | Repository                                                                                  | Services                                                       |
| --------------------------------------- | ----------------------- | ------------------------------------------------------------------------------------------- | -------------------------------------------------------------- |
| [api](#api)                             | `backend`               | [tryretool/backend](https://hub.docker.com/r/tryretool/backend)                             | `MAIN_BACKEND``DB_CONNECTOR``DB_SSH_CONNECTOR`     |
| [jobs-runner](#jobs-runner)             | `backend`               | [tryretool/backend](https://hub.docker.com/r/tryretool/backend)                             | `JOBS_RUNNER`                                                  |
| [workflows-worker](#workflows-worker)   | `backend`               | [tryretool/backend](https://hub.docker.com/r/tryretool/backend)                             | `WORKFLOW_TEMPORAL_WORKER`                                     |
| [workflows-backend](#workflows-backend) | `backend`               | [tryretool/backend](https://hub.docker.com/r/tryretool/backend)                             | `WORKFLOW_BACKEND``DB_CONNECTOR``DB_SSH_CONNECTOR` |
| [code-executor](#code-executor)         | `code-executor-service` | [tryretool/code-executor-service](https://hub.docker.com/r/tryretool/code-executor-service) | No service type.                                               |

#### api

The **api** container manages the core functionality for a Retool instance, such as:

- Frontend interactions (e.g., building apps and workflows).
- Hosting apps and workflows.
- Managing users and permissions.

import MainBackend from "./_partials/_service-main-backend.mdx";
import DbConnector from "./_partials/_service-db-connector-services.mdx";

**api** requires ingress from client interactions, such as loading an app in a browser. These are typically handled by a load balancer (e.g., nginx) which proxies the requests.

| Container                   |       Network ingress        |       Network egress        |
| :-------------------------- | :--------------------------: | :-------------------------: |
| **code-executor**           |  |  |
| **Externalized Storage DB** |  |  |
| **retooldb-postgres**       |  |  |
| **Temporal**                |  |  |
| **Resources**               |  |  |

You can replicate **api** as you scale to manage higher volumes of workflow traffic.

#### jobs-runner

The **jobs-runner** container manages background tasks, runs database migrations for Retool version upgrades, and [Source Control](../source-control/index.mdx).

import JobsRunner from "./_partials/_service-jobs-runner.mdx";

| Container                   |       Network ingress        |       Network egress        |
| :-------------------------- | :--------------------------: | :-------------------------: |
| **Externalized Storage DB** |  |  |

You must not replicate **jobs-runner**. It performs tasks and migrations that must be performed as a single container.

#### workflows-worker

The **workflows-worker** container continuously polls the Temporal Cluster for tasks required to either start or execute blocks within a workflow. It makes requests to **code-executor** to execute blocks and process results, then reports back to Temporal to continue or complete workflow execution.

import WorkflowTemporalWorker from "./_partials/_service-workflow-temporal-worker.mdx";

| Container                   |       Network ingress        |       Network egress        |
| :-------------------------- | :--------------------------: | :-------------------------: |
| **code-executor**           |  |  |
| **Externalized Storage DB** |  |  |
| **retooldb-postgres**       |  |  |
| **Temporal**                |  |  |
| **Resources**               |  |  |

You can replicate **workflows-worker** as you scale to manage higher volumes of workflow traffic.

#### workflows-backend

The **workflows-backend** container receives and processes requests from **code-executor** to process workflows. It also handles the upload of workflows logs, block results, and run status updates.

import WorkflowBackend from "./_partials/_service-workflow-backend.mdx";

| Container                   |       Network ingress        |       Network egress        |
| :-------------------------- | :--------------------------: | :-------------------------: |
| **code-executor**           |   |  |
| **Externalized Storage DB** |  |  |
| **retooldb-postgres**       |  |  |
| **Temporal**                |  |  |
| **Resources**               |  |  |

You can replicate **workflows-backend** as you scale to manage higher volumes of workflow traffic.

#### code-executor

The **code-executor** container executes single block runs from the Workflows IDE, multiple blocks as part of a workflow execution, and any arbitrary user code written in a workflow. It executes JavaScript and Python code directly, or makes requests to **workflows-backend** to run resource queries against databases, APIs, etc.

By default, the **code-executor** uses [nsjail](https://github.com/google/nsjail) to sandbox code execution. [nsjail](https://github.com/google/nsjail) requires [privileged](https://docs.docker.com/reference/cli/docker/container/run/#privileged) container access. If your deployment does not support privileged access, the code executor automatically detects this and runs code without sandboxing.

| Container             |       Network ingress        |        Network egress        |
| :-------------------- | :--------------------------: | :--------------------------: |
| **api**               |   |  |
| **workflows-backend** |  |   |
| **workflows-worker**  |   |  |

You can replicate **code-executor** as you scale to manage higher volumes of workflow traffic.

#### retooldb-postgres

You can optionally [configure your deployment to use Retool Database](guides/retool-database.mdx) with a separate PostgreSQL database. You can use `retooldb-postgres` to get started but Retool recommends you migrate to an externally hosted database for use in production.

### Other dependencies

Self-hosted deployment instances have additional dependencies.

#### Externalized storage database

Your deployment must be able to connect to the externalized PostgreSQL database and have access to it.

#### Resources

Each data source you use with Retool is represented by a resource, such as APIs and external databases. Your deployment instance must have access to any of its resources to interact with data.

import Temporal from "./_partials/_temporal-info.mdx";

### Temporal

import TemporalInfo from "./_partials/_temporal-info.mdx";

import TemporalCompare from "./_partials/_temporal-compare.mdx";

### Docker images

Retool uses Docker images that configure the containers and their respective services. Each container runs one or more services to distribute functionality. This approach allows for efficient scaling of a Retool deployment instance as needs grow.

Retool maintains two image repositories on [Docker Hub](https://hub.docker.com/u/tryretool):

- The [backend](https://hub.docker.com/r/tryretool/backend) image is used for most containers and services.
- The [code-executor-service](https://hub.docker.com/r/tryretool/code-executor-service) image is used to execute blocks of code in Retool Workflows.

Each Docker Hub repository tag corresponds to a particular version of Retool. You specify the version to use when deploying or upgrading a release. For instance:

- 
    tryretool/backend:
    
  
- 
    tryretool/code-executor-service:
    
  

## Releases

Retool maintains two release channels for self-hosted Retool: [Stable](../releases/stable/index.mdx) and [Edge](../releases/edge/index.mdx).

import Stable from "../releases/_partials/_stable.mdx";
import Edge from "../releases/_partials/_edge.mdx";

---

## Authentication environment variables

import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

Authentication environment variables available for use with [Self-hosted Retool](../../index.mdx) deployments.

import Disclaimer from "./_disclaimer.mdx";

---

## Code executor environment variables

import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

**code-executor** environment variables available for use with [Self-hosted Retool](../../index.mdx) deployments. You should only set these environment variables on containers running [`tryretool/code-executor-service`](https://hub.docker.com/r/tryretool/code-executor-service/tags) images.

import Disclaimer from "./_disclaimer.mdx";

---

## Cookies environment variables

import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

Cookies environment variables available for use with [Self-hosted Retool](../../index.mdx) deployments.

import Disclaimer from "./_disclaimer.mdx";

---

## General environment variables

import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

General environment variables available for use with [Self-hosted Retool](../../index.mdx) deployments.

import Disclaimer from "./_disclaimer.mdx";

---

## Environment variables reference

Environment variables control or override certain functions and characteristics of [Self-hosted Retool](../../index.mdx) instances. Some Retool features require you to set environment variables, such as [SSO](../../../sso/index.mdx) or [Source Control](../../../source-control/index.mdx).

import Disclaimer from "./_disclaimer.mdx";

---

## Logging environment variables

import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

Logging-related environment variables available for use with [Self-hosted Retool](../../index.mdx) deployments.

import Disclaimer from "./_disclaimer.mdx";

---

## Mobile environment variables

import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

Environment variables related to Retool Mobile.

import Disclaimer from "./_disclaimer.mdx";

---

## Queries environment variables

import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

Queries environment variables available for use with [Self-hosted Retool](../../index.mdx) deployments.

import Disclaimer from "./_disclaimer.mdx";

---

## Redis environment variables

import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

Redis environment variables available for use with [Self-hosted Retool](../../index.mdx) deployments.

import Disclaimer from "./_disclaimer.mdx";

---

## Resources environment variables

import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

Resources environment variables available for use with [Self-hosted Retool](../../index.mdx) deployments.

import Disclaimer from "./_disclaimer.mdx";

---

## Source Control environment variables

import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

Source Control environment variables available for use with [Self-hosted Retool](../../index.mdx) deployments. You can also set these variables from the [Source Control settings page](https://login.retool.com/auth/login?redirectOnLogin=settings/source-control) or with the [Retool API](https://docs.retool.com/api).

import Disclaimer from "./_disclaimer.mdx";

---

## Storage database environment variables

import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

Storage database environment variables available for use with [Self-hosted Retool](../../index.mdx) deployments.

import Disclaimer from "./_disclaimer.mdx";

---

## Workflows environment variables

import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

Workflow environment variables available for use with [Self-hosted Retool](../../index.mdx) deployments. These environment variables specifically can be applied to the `api`, `workflows-worker`, and `workflows-backend` [services](../../concepts/architecture.mdx#containers-and-services).

import Disclaimer from "./_disclaimer.mdx";

---

## Self-hosted Retool glossary

import SharedGlossary from '../../_partials/_glossary.mdx';
import EnterpriseRequirement from "../_partials/_enterprise-plan-requirement.mdx";

---

## Self-hosted Retool requirements

import EnterpriseRequirement from "../_partials/_enterprise-plan-requirement.mdx";

Deploying [Self-hosted Retool](https://retool.com/self-hosted/) on your own infrastructure lets you build applications with data in your virtual private cloud (VPC) or behind your virtual private network (VPN). Businesses in the healthcare and finance industries often deploy Retool to remain compliant.

import Requirements from '../_partials/_deployment-requirements.mdx';

## Network requirements

[Retool Self-hosted](../index.mdx) organizations must ensure that their deployments allow access to Retool's IP addresses or domains. If you make use of outbound firewall rules, include the following IP addresses or domains in its allowlist. These allow your deployment to connect to Retool's license check, user authentication, and usage reporting services.

import Ips from '../_partials/_ips.mdx';

import Domains from '../_partials/_domains.mdx';

:::info
Egress is required when using Retool-managed Temporal. Refer to the [Egress](../concepts/temporal#egress) section of the [Temporal](../concepts/temporal) conceptual guide for more information. 
::: 

### HTTP proxy connections

Retool supports connections to the internet through a HTTP proxy. Add `HTTP_PROXY=http://example.com:8080` to your deployment's `docker.env` file with the required URL and port number.

### License checks

Retool uses HTTP to connect to `licensing.tryretool.com` on port `443` to verify your license. License checks are made at least once a day.

### Inviting users

Retool connects to `invites.tryretool.com` and `email-service.retool.com` on port `443` when inviting users. Retool verifies the users are authorized under your current billing plan, and then sends an invite to their email address.

### Usage reporting

Retool sends application usage information to `p.tryretool.com` on port `443`, which is used to inform product decisions.

#### Usage categories

The categories of usage information sent to Retool includes, but is not limited to, the following:

- Page views, along with the page URL.
- Query saves, including the query name and type.
- Component creation and the component type.
- Query preview, including the query name and type.
- Adding a resource, including the resource name and type.

Events are also sent with the hostname, public IP address, browser user-agent string, and the user's email address.

## Retool AI

Self-hosted customers can deploy and embed Retool AI within apps and workflows. When a user performs any action with Retool AI, the input is shared with the applicable third-party LLM provider, listed in our [Subprocessors](https://docs.retool.com/legal/subprocessors) page. Inputs are deleted within 30 days.

You can configure a direct connection to a [supported AI platform](../../data-sources/concepts/models.mdx) by providing an API key. You also have the option to use a Retool-managed OpenAI connection. If enabled, AI requests are proxied through Retool The Retool-managed OpenAI connection is disabled by default. Contact your Retool account representative or our [support team](mailto:support@retool.com) to gain access.

For production use cases, we recommend you provide an API key and directly connect to an AI platform.

When Retool AI is used to build or configure a query, application, or workflow (e.g., prompting [Ask AI](../../queries/guides/ask.mdx) to help write queries), any inputs or outputs that correspond to the categories above may continue to be used to inform product improvements. Retool does not use any inputs submitted to, or outputs generated from, Retool AI that is embedded within a deployed application or workflow (e.g., text stored in [Retool Vectors](../../data-sources/quickstarts/retool-vectors.mdx)). These inputs and outputs are treated as "Customer Data" in according with the [Customer Terms of Service](https://docs.retool.com/legal/customer-terms-of-service), [Security Practices](https://docs.retool.com/legal/security), and [Data Processing Addendum](https://docs.retool.com/legal/dpa).

---

## Temporal environment variables for local clusters

import EnterpriseRequirement from "../_partials/_enterprise-plan-requirement.mdx";

Self-hosted organizations can configure the following Temporal environment variables if they want to use a local cluster. These environment variables are relevant only if all the following are true for your organization:

- Does not use Kubernetes for self-hosted deployments.
- Did not use Retool's Helm charts when deploying a self-hosted instance.
- Uses Retool's Temporal image.
- Will use a PostgreSQL database as the backing store for Temporal.

:::warning

Temporal environment variables are only used to configure a local Temporal Cluster. These variables are managed by Temporal and function separately from Retool's environment variables.

:::

---

## Self-hosted Retool reference

import Reference from '/docs/_partials/_doctypes/_reference.mdx';
import EnterpriseRequirement from "./_partials/_enterprise-plan-requirement.mdx";

:::tip Get started with a self-hosted Retool deployment

If you are unfamiliar with deploying self-hosted Retool want to get started, read the [quickstart](./quickstart) to learn about the fundamental concepts.

:::

---

## _callout

:::info
Retool-managed, self-hosted deployments are available for invoiced customers. Contact your Retool account manager to learn more.
:::

---

## Retool-managed deployment architecture

import Callout from "../_callout.mdx";
import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

A self-hosted and Retool-managed _deployment_ is a customer-owned VPC that's hosted on [Amazon Web Services](https://aws.amazon.com) (AWS) and contains a [self-hosted](../../index.mdx) instance. You retain full ownership of, and control over, your data, encryption keys, access, and network infrastructure.

## Infrastructure layers

A Retool-managed, self-hosted deployment consists of two _infrastructure layers_ that operate on a dedicated AWS account.

- The [support](#support-layer) layer: The foundation of a Retool-managed, self-hosted deployment.
- The [services](#services-layer) layer: The resources needed by the self-hosted instance.

You have full ownership and control over every aspect of the deployment. Retool only has access to what's required for managing the self-hosted instance.

### Support layer

The support layer represents the dedicated VPC, resources, and configuration necessary for the deployment. It includes:

| Resource                              | Description                                                                                                                    |
| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
| VPC                                   | The environment that contains the Runner VM and Retool-managed self-hosted instance.                                           |
| AWS Secrets Manager                   | The resource in which secrets and environment variables are stored.                                                            |
| Runner VM                             | The service that allows Retool to manage the self-hosted instance without having access to your larger infrastructure or data. |
| IAM roles and policies                | The resources that govern the scope of access granted to the Runner VM for Retool.                                             |
| DNS and private network configuration | Resources related to routing user traffic to the VPC.                                                                          |

Retool cannot directly access nor manage any resources within the support layer, and works with you to create them. To ensure that the resources are configured exactly as intended, Retool uses an Infrastructure as Code approach; a CloudFormation template is provided to your AWS admin that automatically provisions and configures the VPC, Runner VM, and IAM resources.

#### Secrets and environment variables

[AWS Secrets Manager](https://aws.amazon.com/secrets-manager/) is used to securely store secrets and [environment variables](../../reference/environment-variables/index.mdx) needed for the deployment. Resources needing to use these values retrieves them directly from Secrets Manager, to which Retool has no access.

For instance, your AWS admin adds the following secrets to Secrets Manager which are required by the CloudFormation template:

- Retool Encryption Key
- Retool JWT Secret

:::tip

Unless you are migrating from an Retool Cloud to a BYOC instance, the Retool encryption key and JWT secret values can be randomly generated. Retool recommends using `openssl rand -base64 32`. Customers who are migrating from Retool Cloud are provided values to use instead.

:::

Using Secrets Manager for environment variables also allows customers to make changes to update their instance configuration without directly modifying it.

#### Runner VM

The _Runner VM_ is a dedicated virtual server that operates within the VPC and functions as an agent for Retool's management services. It allows Retool to manage the instance without the need for broader access to your cloud environment. The Runner VM receives instructions from Retool—such as performing an update—and then executes them on the Retool-managed, self-hosted instance.

Any temporary disruption to the Runner VM does not impact the instance itself. If the Runner VM is unavailable, Retool loses access, and cannot remotely monitor or manage the instance.

#### IAM roles and policies

The Runner VM requires access that's managed using IAM roles and policies. These are used to securely manage Retool's access, prevents any escalation of privileges, and any configuration changes that might compromise the scope of access.

The Runner VM depends on three IAM roles to function correctly and have sufficient access to Retool-managed, self-hosted resources.

| Role        | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Provision   | Used at initial setup time, primarily to create new resources. The customer disables this role after initial setup is complete for maximum data security.                                                                                                                                                                                                                                                                                                     |
| Maintenance | Used after setup for the lifetime of the Retool-managed, self-hosted instance. This role is typically used to perform `GET` or `describe-action` to monitor the health of the instance. The Runner VM may also need to create or delete resources as part of ongoing maintenance. This role is subject to tighter restrictions that prevent access to customer data. This role is subject to additional restrictions that prevent access to customer data and credentials. |
| Deprovision | Used to delete all cloud resources if the customer has decommissioned their instance. This role can remain disabled and only enabled if needed.                                                                                                                                                                                                                                                                                                          |

#### DNS and private network configuration

Once your instance is ready, you configure DNS records for your users to have access. This is an action you perform with your DNS provider after both the CloudFormation stack and instance are set up.

Retool supports additional private network options, such as VPC PrivateLink and VPN user access. While supported, Retool can only provide guidance—customers are responsible for the configuration and maintenance of any custom network configuration.

### Services layer

The services layer contains required and optional resources for the self-hosted instance, most of which are managed by Retool. This is created and managed by Retool for you, via the Runner VM. Retool cannot directly access any resources within the services layer. The Runner VM is also used by Retool to interact with the resources of the service layer.

| Resource                        | Description                                               |
| ------------------------------- | --------------------------------------------------------- |
| EKS cluster                     | Kubernetes pods and services.                             |
| RDS PostgreSQL database (main)  | PostgreSQL database for the storage of metadata.          |
| Application Load Balancer (ALB) | Network traffic management and distribution.              |
| Route53 DNS zones and records   | DNS configuration for the instance (e.g., Retool Spaces). |

#### EKS cluster

The Retool-managed, self-hosted instance uses an [Amazon Elastic Kubernetes Service](https://aws.amazon.com/eks/) (Amazon EKS) cluster that operates within the dedicated VPC.

#### PostgreSQL database

The main database in which Retool stores metadata uses [Amazon RDS for PostgreSQL](https://aws.amazon.com/rds/postgresql/).

#### Route53 DNS zones

Retool uses an [Amazon Route53](https://aws.amazon.com/route53/) publicly hosted zone to manage DNS records within the deployment. This includes:

- TLS certificate validation records.
- Application Load Balanceer (ALB) alias records.
- Wildcard records for [Retool Spaces](../../../org-users/tutorials/spaces.mdx).

This hosted zone is scoped to the custom domain used for the instance.

## Configuration options

:::note
Retool only supports the available configuration options listed below and cannot substitute them with alternatives.
:::

Retool uses a standard deployment configuration by default that's suitable for most use-cases. You can also customize certain configuration options to best suit your needs.

- **Standard**: A configuration option that's used by default if you do not request an available custom configuration option.
- **Available**: A configuration option that's not used by default but can be enabled upon request.
- **Custom**: An alternative configuration option that's available for use upon request.

Although custom configuration options are available, Retool cannot support, access, or manage any external infrastructure they use. For instance, you can request that your deployment only be accessible by VPN but Retool cannot implement nor manage the VPN for you.

Some custom options can only be configured to deployment while others are available for use at any time. Unless otherwise stated, you must configure and manage any custom configuration options you request.

### DNS

Configuration options for managing DNS records of the deployment.

| Option                                                                                                                         | Description                                                                                                                                                                                                                                                                     |
| ------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| NS record delegation Standard                                              | DNS for the deployment domain is delegated using NS records. Supports automatic certificate renewal and subdomain support for Spaces.                                                                                                                                           |
| Direct Application Load Balancer and certificate validation records Custom | Individual DNS records are configured to provide more granular control, such as support for Web Application Firewall (WAF) policies. DNS records for certificate renewal are also configured. Retool can provide guidance but you must configure and manage this configuration. |

### Ingress

Configuration options for managing user access to the deployment.

| Option                                                                                                | Description                                                                                                                                                                                                                                |
| ----------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Retool-managed Application Load Balancer Standard | Publicly accessible ALB that handles internal and external traffic.                                                                                                                                                                        |
| Customer-managed Application Load Balancer Custom | ALB with custom configuration, such as Web Application Firewall (WAF) policies. Retool-managed, self-hosted deployments include a `byolb` ALB target group by default for which Retool can provide guidance but you must configure and manage this configuration. |
| Customer-managed VPN Custom                       | Accessible only through a VPN connection. Retool can provide guidance but you must configure and manage all aspects of the VPN connection between the user and the deployment.                                                        |

### Egress

Configuration options for managing how the deployment accesses external resources (e.g., cloud-hosted data sources).

| Option                                                                                             | Description                                                                                                                                                                                                                                 |
| -------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Public internet egress Standard                | Unrestricted internet access for connecting to any publicly reachable resource.                                                                                                                                                            |
| Static IP address pool for allowlists Standard | Retool connects to resources using a pool of [static IP addresses](../../../data-sources/reference/ip-allowlist-cloud-orgs.mdx) that can be added to allowlists.                                                                               |
| SSH tunnelling Standard                        | Connect to private resources using [SSH tunnels](../../../data-sources/guides/connections/ssh-tunnels.mdx).                                                                                                                             |
| VPN access Custom                              | Connect to private resources using a VPN. Retool can provide guidance but you must configure and manage all aspects of the VPN connection between the deployment and private resources.                                                       |
| VPC PrivateLink Custom                         | Private connection between the deployment's VPC and another VPC using [AWS PrivateLink](https://aws.amazon.com/privatelink/). Retool can provide guidance but you must configure and manage all aspects of the PrivateLink connection. |

### Storage database and fault tolerance

Configuration options that apply to the main PostgreSQL database of the Retool-managed, self-hosted instance. Any customization must be requested prior to setup.

| Option                                                                                                                                                     | Description                                                              |
| ---------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------ |
| Single-availability zone Standard                                                                      | A single, non-replicated RDS database used for metadata.                 |
| Multi-availability zone Available Configured pre-deployment | A fault-tolerant RDS database that's replicated across multiple regions. |

### Optional Retool features

Configuration options for features not enabled by default. Retool can enable, configure, and manage these features for you at any time. These features require additional resources which may incur additional AWS costs.

| Option                                                                                         | Description                                                                                                                                    |
| ---------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| Retool Database Available                     | Requires an additional [RDS database](https://aws.amazon.com/rds/) that you provision using Retool-provided information.                                                      |
| Retool Storage Available                      | Requires an [Amazon S3](https://aws.amazon.com/s3/) bucket. You can use an existing bucket or provision a new one using Retool-provided information.                         |
| Retool RPC Available                          | Requires an [Amazon ElastiCache](https://aws.amazon.com/elasticache/) for Redis resource that you provision using retool-provided information. |
| Collaborative editing (Multiplayer) Custom | No additional resources required and can be enabled upon request.                                                                              |

:::info
Retool always assumes that these services contain sensitive data. To maintain the security boundary that separates Retool from your wider infrastructure and data, Retool is prevented from accessing them.
:::

---

## Retool-managed deployment health monitoring and scaling

import Callout from '../_callout.mdx';
import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

Retool monitors the health of a Retool-managed, self-hosted instance 24/7 and is automatically alerted of any potential issues that may arise. This coverage includes:

- Containers (e.g., upgrade errors or misconfiguration)
- Error log spikes (e.g., functionality degredation, capacity limits)
- Databases (e.g., Amazon RDS storage or resource usage)

:::info
Health monitoring only applies to infrastructure of the Retool-managed, self-hosted instance. Refer to the [observability](../../../apps/guides/observability/index.mdx) documentation to learn how you can monitor object-level errors for apps, workflows, and more.
:::

## Alerts and response

Retool immediately reviews any alert that is triggered. If the alert requires remediation, a Retool engineer begins to diagnose the issue. If necessary, the engineer then takes any action to resolve the issue that triggered the alert.

Alerts can be triggered for any number of reasons so not every alert requires remediation. In the event that an alert did require a Retool engineer to take action, you're notified of what occurred within one business day.

:::note
Observability data is configured for use only by Retool and cannot be shared with customers.
:::

## Deployment scaling

Retool-managed, self-hosted deployments can scale as your needs grow. How it scales depends on its usage. You can contact Retool at any time to discuss scaling options. Retool will also proactively reach out if your deployment is reaching any limits.

### RDS database size

The RDS PostgreSQL database starts with 40GB of storage. It automatically increases in size as needed, up to the default limit of 200GB. Retool can raise the limit at any time upon request. There is no downtime whenever the database increases in size or the limit is raised.

The database only increases in size automatically. If the amount of data it contains is reduced, it does not scale down.

### RDS instance type

The RDS CPU and memory resources are determined by its instance type. The default type, `db.m6g.large`, uses 2x vCPUs and 8GiB memory. Unlike the RDS database size, CPU and memory resources do not scale automatically. Retool can upgrade—or downgrade—the instance type upon request. Retool also proactively monitors the instance and will recommend changes if they are deemed necessary.

:::warning
Changes to the RDS instance type require downtime. They should only be performed during a regularly scheduled maintenance period.
:::

### Backend containers and resources

Retool automatically scales the Retool-managed, self-hosted instance to ensure that it has sufficient capacity for the amount of traffic it needs. This includes making adjustments to the number and type of containers in use.

The default capacity for a Retool-managed, self-hosted instance is 20 queries per second (QPS). The resources required by your instance is dependent on your use case. If you require resources for a high volume of traffic, or if you expect an influx of traffic, contact your Retool account manager for guidance on scaling and preparation.

---

## Retool-managed deployment security

import Callout from "../_callout.mdx";
import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

Retool is only provided with sufficient access to a Retool-managed, self-hosted instance while being restricted from the customer's data and infrastructure. This limited access ensures that your data remains private while enabling Retool to successfully manage your instance.

## Remote access model

The [Runner VM](./architecture.mdx#runner-vm) is a _local agent_ that operates within the customer's AWS environment. Retool connects remotely to the Runner VM—with no access to any other customer infrastructure—for the sole purpose of managing the deployment.

## Access restrictions

Retool uses _deny-based_ access policies to restrict access only to the required infrastructure. To do this, the instance is deployed to a dedicated AWS account and Retool's access is restricted based on the following scopes.

| Scope                | Description                                                                                                                                                                    |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| User access          | No user access to the Retool application by default. Access can only be explicitly granted by the customer using a support user account.                                       |
| App-level secrets    | No access to encryption keys for sensitive database columns (e.g., resource credentials and SSO secrets), JWT signing keys, and customer-provided deployment secrets.          |
| Data storage access  | No access to customer data stores (e.g., Amazon RDS databases or Amazon S3 buckets).                                                                                                         |
| Privilege management | Identity and Access Management (IAM) policies prevent certain types of privilege escalation that could compromise Retool's deny-based access policies (e.g., creation of IAM roles with additional privileges). |

:::tip
In some cases, Retool may request temporary elevated privileges within the customer's environment for the purpose of issue resolution. This request is made only if absolutely necessary and there are no other options to use. The customer has full control over any request and the granting of elevated privileges.

:::

## Externally processed data

A Retool-managed, self-hosted deployment instance emits certain types of data outside the customer's environment for specific operational purposes.

| Data                  | Description                                                                                                                                                    |
| --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Usage and user events | Analytics about Retool feature usage and user events, such as signing in.                                                                                      |
| Pricing and metering  | Information related to your organization's pricing and metering, such as user count.                                                                           |
| Infrastructure health | Observability metrics about the health of the instance. This does not include any customer data or personally identifiable information (PII)              |
| Temporal              | Temporal metadata Encrypted metadata processed in Temporal Cloud for workflow coordination. This does not include the contents of code blocks and run outputs. |
| Alerts                | System alerts and notifications, such as administrative alerts and user invites.                                           |

---

## Retool-managed deployment concept guides

import Concept from '/docs/_partials/_doctypes/_concept.mdx';
import EnterpriseRequirement from "../_partials/_enterprise-plan-requirement.mdx";

---

## Configure Retool-managed deployment secrets and environment variables

import Callout from "../_callout.mdx";
import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

Depending on your use case, you may need to set some environment variables to [provide additional secrets](../../guides/secrets/index.mdx) or specify [environment variables](../../reference/environment-variables/index.mdx). Customers use AWS Secrets Manager to set environment variables and secrets for their Retool-managed, self-hosted instance.

## Configure in AWS Secrets Manager

You can manage environment variables and secrets in AWS Secrets Manager. This can be done using the Secrets Manager console or the [Secrets Manager API](https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/secretsmanager-examples-manage-secret.html).

1. Sign in to the [AWS Secrets Manager console](https://console.aws.amazon.com/secretsmanager).
1. Select the `deployment_secrets` secret.
1. Click **Retrieve secret value** and then **Edit**.

The `deployments_secrets` secret accepts key-value pairs, where each key corresponds to an environment variable. To add an environment variable, click **+Add row** in the **Key/value** tab.

Any changes you make are automatically applied and take effect after a few minutes.

:::tip
Deployment secrets use a consistent template for namespacing in case of multiple deployments per account.

```
${deployment_id}/${customer_name}_deployment_secrets
```
:::

---

## Retool-managed deployment how-to guides

import Howto from '/docs/_partials/_doctypes/_howto.mdx';
import EnterpriseRequirement from "../_partials/_enterprise-plan-requirement.mdx";

---

## Retool-managed deployment documentation

import Callout from './_callout.mdx';

Retool can deploy a [self-hosted](https://retool.com/self-hosted/) Retool instance on AWS and manage it for you. This provides your organization with a fully self-hosted deployment but without the overhead of setup and maintenance. You retain full ownership of, and control over, your data, encryption keys, access, and network infrastructure.

---

## Retool-managed deployment quickstart

import Callout from "./_callout.mdx";

A Retool-managed, self-hosted deployment is a customer-owned VPC hosted on [Amazon Web Services](https://aws.amazon.com) (AWS) that contains a Retool-managed [self-hosted](../index.mdx) instance. This provides your organization with a fully self-hosted deployment but without the overhead of setup and maintenance. You retain full ownership of, and control over, your data, encryption keys, access, and network infrastructure.

Retool-managed, self-hosted deployments are best suited for organizations that:

- Are unable to use Retool Cloud (e.g., compliance restrictions)
- Do not have the resources to maintain a self-hosted deployment (e.g., a limited DevOps team)

Retool uses an Infrastructure as Code approach to programmatically configure Retool-managed, self-hosted deployments. This ensures your deployment is created exactly as needed and with as little intervention as possible.

## Overview

A Retool-managed, self-hosted deployment consists of two _infrastructure layers_:

- The [support layer](#support-layer) represents the foundation of a deployment.
- The [services layer](#services-layer) represents the resources for the self-hosted instance.

You have full ownership and control over every aspect of a deployment. Retool only has access to what's required for managing the self-hosted instance.

### Support layer

The [support layer](./concepts/architecture.mdx#support-layer) represents the core resources needed for a deployment, such as:

- The VPC on which the instance runs.
- secrets used by the instance, such as encryption keys and environment variables.
- Access management configuration that governs what Retool has access to, such as IAM policies.
- External network configuration for users to access the instance, such as DNS and VPN networking.

This layer is configured in such a way that Retool can securely access the VPC to manage the self-hosted instance without access to the rest of your infrastructure or data.

Retool does not create or configure the support layer directly. This is by design to ensure that Retool has no access to your external infrastructure and data. Instead, Retool works with you to create this layer automatically with a 1-click CloudFormation template.

The VPC includes the Runner VM: a dedicated service that receives instructions from Retool—such as performing an update—and then executes them on the Retool-managed, self-hosted instance.

### Services layer

The [services layer](./concepts/architecture.mdx#services-layer) reresents the resources for the self-hosted instance, such as the Kubernetes cluster, databases, and internal network configuration. Retool manages most of these resources for you.

Retool can only interact with your instance via the Runner VM, which operates in the support layer.

## Deployment process

Retool works with you throughout the entire deployment process. In general, there are four steps to deploy the instance:

1. [Prepare for the deployment](#1-prepare-for-the-deployment). Retool works with you to prepare the right configuration for the deployment.
1. [Create the support layer](#2-create-the-support-layer). You set up the deployment's service layer on AWS using resources and guidance from Retool.
1. [Create the services layer](#3-deploy-the-services-layer). Retool creates the services layer and configures the instance.
1. [Enable access](#4-enable-access). You update the DNS configuration so network traffic can route to the instance, giving users access and completing the deployment.

A Retool-managed, self-hosted deployment can be completed within 24 hours if all information is provided and tasks completed as soon as they're needed.

### 1. Prepare for the deployment

At the beginning of the process, Retool asks you about your deployment preferences, such as:

- The AWS region in which to deploy.
- The instance domain.
- If you need to migrate from an existing instance.
- Whether you need to make any [custom configuration](./concepts/architecture.mdx#configuration-options) changes.

As part of the preparation, you create a new AWS account that's solely used for the deployment.

### 2. Create the support layer

Once preparation is complete, Retool produces a CloudFormation template for you. This is shared with your AWS admin as a 1-click install link that starts the CloudFormation stack setup on AWS.

Once your admin populates AWS Secrets Manager with encryption keys and environment variables, they use the template to perform the setup process. Once complete, the services layer can then be created.

### 3. Deploy the services layer

The Runner VM automatically creates the resources that make up the service layer, such as the EKS cluster and PostgreSQL databases. This process is fully automated using instructions provided by Retool.

### 4. Enable access

With the instance deployed and running, the final step is to configure the deployment's DNS records using the information provided by Retool so that it can be accessed by users. If you've requested additional network options for use with the instance, Retool can also provide guidance but cannot configure them directly.

## Patch updates and release upgrades

:::info
For deployments with multiple environments (e.g., staging), Retool may update these a few days prior to the production environment to avoid disruption.
:::

Retool manages all updates and upgrades for your deployment. Your instance runs the latest [stable release](../../releases/stable/index.mdx) by default and kept up-to-date with security patches.

All services used by the instance, such as Amazon RDS for PostgreSQL, follow the officially maintained or long-term support (LTS) release cycle and are kept up-to-date. This includes updating to a newer LTS release once the current one reaches its end-of-life.

## Migrate from an existing instance

Retool can help customers migrate from an existing Retool Cloud or self-hosted instance to a Retool-managed, self-hosted deployment.

Customers on the Enterprise plan with a Retool Cloud organization can request a migration to a Retool-managed, self-hosted deployment. To arrange a migration, contact your Retool account manager.

:::warning
As self-hosted migrations involve customer data, Retool can only provide guidance and cannot export or import instance data for you.

:::

Customers on the Enterprise plan with a self-hosted Retool instance can request a migration to a Retool-managed, self-hosted deployment. This is a three-part process:

1. Retool works with you to deploy a new Retool-managed, self-hosted instance but not configure DNS records.
2. You export your existing instance's storage database and restore it to the new instance.
3. Once restored, you then configure DNS records to enable access.

---

## Retool-managed deployment ownership and responsibilities

import Callout from "../_callout.mdx";
import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

Retool-managed, self-hosted deployments operate using a shared responsibility model. This governs whether Retool, the customer, or both are responsible for implementing or maintaining each part of the deployment.

Shared responsibility covers the _infrastructure_ and _management_ of the deployment:

- **Infrastructure**: Shared responibilities for each service of the deployment.
- **Management**: Shared responibilities for managing and maintaining each aspect of the deployment.

## Infrastructure responsibilities

| Resource                                                      | Owner    | Layer                                                   |
| ------------------------------------------------------------- | -------- | ------------------------------------------------------- |
| CloudFormation stack                                          | Customer | [Support](../concepts/architecture.mdx#support-layer)   |
| Secrets and environment variables                             | Customer | [Support](../concepts/architecture.mdx#support-layer)   |
| DNS and private network configuration                         | Customer | [Support](../concepts/architecture.mdx#support-layer)   |
| IAM roles and policies                                        | Retool   | [Support](../concepts/architecture.mdx#support-layer)   |
| VPC                                                           | Retool   | [Support](../concepts/architecture.mdx#support-layer)   |
| EKS cluster and pods                                          | Retool   | [Services](../concepts/architecture.mdx#services-layer) |
| RDS PostgreSQL database (main)                                | Retool   | [Services](../concepts/architecture.mdx#services-layer) |
| Application Load Balancer (ALB)                               | Retool   | [Services](../concepts/architecture.mdx#services-layer) |
| Certificate renewal with AWS Certificate Manager (ACM)        | Retool   | [Services](../concepts/architecture.mdx#services-layer) |
| Route53 DNS zones and records                                 | Retool   | [Services](../concepts/architecture.mdx#services-layer) |
| Amazon RDS PostgreSQL database for Retool Database (optional) | Retool   | [Services](../concepts/architecture.mdx#services-layer) |
| Amazon S3 bucket for Retool Storage (optional)                | Retool   | [Services](../concepts/architecture.mdx#services-layer) |
| ElastiCache Redis instance for Retool RPC (optional)          | Retool   | [Services](../concepts/architecture.mdx#services-layer) |

## Management responsibilities

| Responsibility                            | Owner    | Description                                                                                                                                                                                      |
| ----------------------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Deployment updates and upgrades           | Retool   | [Perform security updates and scheduled release upgrades](../quickstart.mdx#patch-updates-and-release-upgrades) of self-hosted Retool.                                                           |
| Deployment health                         | Retool   | [Monitor the health](../concepts/health-monitoring.mdx) of the instance.                                                                                                                         |
| Scaling                                   | Retool   | [Scale backend resources](../concepts/health-monitoring.mdx#deployment-scaling), such as CPU, memory and storage.                                                                                     |
| Enable optional features                  | Retool   | [Configure optional Retool features](../concepts/architecture.mdx#configuration-options), such as Retool Database.                                                                               |
| Supported configuration changes           | Both     | [Configure changes](../concepts/architecture.mdx#custom-configurations) and, if necessary, create necessary data stores.                                                                         |
| Migrations                                | Both     | [Migrate an existing Retool deployment](../quickstart.mdx#migrate-from-an-existing-instance). Migration options are dependent on each situation. Contact your Retool account manager to discuss. |
| Software observability                    | Customer | [Monitoring, testing, and maintenance](../../../apps/guides/observability/index.mdx) of user-built Retool software, such as apps and workflows.                                                  |
| AWS account management                    | Customer | Manage ownership and costs related to the [AWS account](../quickstart.mdx#1-prepare-for-the-deployment).                                                                                         |
| User management                           | Customer | [Manage Retool users](../../../org-users/index.mdx) who use and build software.                                                                                                                  |
| Secrets and environment variable handling | Customer | Securely handle and store [secrets and environment variables](../concepts/architecture.mdx#secrets-and-environment-variables), such as encryption keys.                                          |
| Data sources (resources)                  | Customer | Connect your [data sources](../../../data-sources/index.mdx) for use in Retool software.                                                                                                         |
| SSO                                       | Customer | Configure [single-sign on](../../../sso/index.mdx) to authenticate users in your organization.                                                                                                   |
| Restrict access to VPC resources          | Customer | Prevent changes being made to VPC resources, such as the Amazon EKS cluster, used for the Retool instance.                                                                                       |
| Custom configuration options              | Customer | Configure and maintain any [custom configuration options](../concepts/architecture.mdx#configuration-options), such as VPN access or PrivateLink.                                                |

---

## Retool-managed deployment reference

import Reference from '/docs/_partials/_doctypes/_reference.mdx';
import EnterpriseRequirement from "../_partials/_enterprise-plan-requirement.mdx";

---

## Deploy Self-hosted Retool on Azure Virtual Machines

import Requirements from "../_partials/_requirements.mdx";
import DockerDownload from "../_partials/_docker_download.mdx";
import DockerInstall from "../_partials/_docker_install.mdx";
import DockerKey from "../_partials/_docker_encryption_key.mdx";
import DockerConfigure from "../_partials/_docker_configure.mdx";
import DockerUpdate from "../_partials/_update.mdx";
import TemporalOptions from "../_partials/_temporal-options.mdx";
import Production from "../_partials/_additional-steps.mdx";
import DockerStart from "../_partials/_docker_start.mdx";
import SelfhostedLatest from "@site/src/components/SelfhostedLatest";
import EnterpriseRequirement from "../_partials/_enterprise-plan-requirement.mdx";

You can deploy an instance of Self-hosted Retool on [Azure Virtual Machines](https://azure.microsoft.com/en-us/products/virtual-machines/) with [Docker Compose](https://docs.docker.com/compose/).

## Requirements

## System architecture

The following diagram shows the resulting system architecture for your deployment.

## 1. Create a Linux VM

Create a new Linux VM in the [Azure Portal](https://portal.azure.com/) that meets the minimum requirements. Refer to the [Azure documentation](https://learn.microsoft.com/en-us/azure/virtual-machines/linux/quick-create-portal?tabs=ubuntu) to learn how to create a new virtual machine.

| VM Setting         | Description                                                                                                                                                                                                                                                                                                                       |
| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Authentication** | Use **SSH public key** authentication and provide a username. Generate a new key-pair and specify a name. Azure generates and stores the key in the Azure KeyVault to download later.                                                                                                                                             |
| **Network**        | Allow SSH (`22`), HTTP (`80`), and HTTPS (`443`) inbound public ports. The VM must have a public IP address set for external access, or a private IP which allows egress traffic from [Retool's IPs](../reference/requirements.mdx#network-requirements). Ensure **Delete public IP and NIC when VM is deleted** is enabled. |
| **Storage**        | Specify **Standard SSD** with a size that meets or exceeds the minimum requirements. Ensure **Delete with VM** is enabled for the newly added data disk.                                                                                                                                                                          |

Once the VM is initialized, download the private key when prompted. Update the permissions of the key-pair to ensure it has the required permissions for your SSH client.

```shell
chmod 400 
```

## 2. Download Self-hosted Retool

Azure automatically runs the new virtual machine once created. Obtain the public IP address from the Azure Portal and connect to it using SSH.

```shell
ssh -i keypair.pem @
```

## 3. Install dependencies

## 4. Back up encryption key

## 5. Configure the instance

## 6. Start the instance

" />

## Additional configuration

## Update your Azure VM instance

---

## Deploy Self-hosted Retool with Docker

import Requirements from "../_partials/_requirements.mdx";
import DockerDownload from "../_partials/_docker_download.mdx";
import DockerInstall from "../_partials/_docker_install.mdx";
import DockerKey from "../_partials/_docker_encryption_key.mdx";
import DockerConfigure from "../_partials/_docker_configure.mdx";
import DockerUpdate from "../_partials/_update.mdx";
import TemporalOptions from "../_partials/_temporal-options.mdx";
import DockerStart from "../_partials/_docker_start.mdx";
import Production from "../_partials/_additional-steps.mdx";
import SelfhostedLatest from "@site/src/components/SelfhostedLatest";
import EnterpriseRequirement from "../_partials/_enterprise-plan-requirement.mdx";

You can deploy a non-production instance of Self-hosted Retool for testing and development on a Linux-based VM using [Docker](https://www.docker.com/).

For non-production and development purposes, the default Docker Compose configuration includes a PostgreSQL container and does not set up SSL. This is not suitable for production use cases and you must host the Retool Storage database on an external, managed PostgreSQL database.

Managed databases are more maintainable, scalable, and reliable than containerized PostgreSQL instances. Follow the instructions in the [external storage database guide](../guides/storage-database.mdx#2-migrate-the-data-to-an-external-hosted-database) to configure your database.

Should you need to start using the deployment instance in production, [migrate to an externalized PostgreSQL database](../guides/storage-database.mdx) first.

:::danger

Retool recommends using a cloud computing provider, such as [AWS EC2](ec2.mdx), [Azure Virtual Machines](azure-vm.mdx), or [Google Compute Engine](gcp.mdx). Dedicated cloud services are more robust for use in production environments and enable you to scale as your needs grow.

:::

## Requirements

To deploy Retool for non-production or development purposes using Docker, you need:

- A Retool license key, which you can obtain from the [Retool Self-hosted Portal](https://my.retool.com/) or your Retool account manager.
- Familiarity with and installations of [Docker Engine](https://docs.docker.com/engine/install/) and [Docker Compose](https://docs.docker.com/compose/).
- A compatible Linux-based virtual machine.

{props.children}

### VM configuration

Self-hosted Retool instances require a Linux-based virtual machine that meets the following requirements:

import VmConfiguration from "../_partials/_vm-configuration.mdx";

## 1. Download Self-hosted Retool

## 2. Install dependencies

## 3. Back up encryption key

## 4. Configure the instance

## 5. Start the instance

## Additional configuration

## Update your Docker instance

---

## Deploy Self-hosted Retool on Amazon EC2

import Requirements from "../_partials/_requirements.mdx";
import DockerDownload from "../_partials/_docker_download.mdx";
import DockerInstall from "../_partials/_docker_install.mdx";
import DockerKey from "../_partials/_docker_encryption_key.mdx";
import DockerConfigure from "../_partials/_docker_configure.mdx";
import DockerUpdate from "../_partials/_update.mdx";
import TemporalOptions from "../_partials/_temporal-options.mdx";
import DockerStart from "../_partials/_docker_start.mdx";
import Production from "../_partials/_additional-steps.mdx";
import SelfhostedLatest from "@site/src/components/SelfhostedLatest";
import EnterpriseRequirement from "../_partials/_enterprise-plan-requirement.mdx";

You can deploy Self-hosted Retool on Amazon EC2 with [Docker Compose](https://docs.docker.com/compose/).

## Requirements

In addition, Retool recommends you:

- Follow this guide using an [administrative, non-Root AWS user](https://docs.aws.amazon.com/accounts/latest/reference/best-practices-root-user.html).
- Manage your [service quotas](https://repost.aws/knowledge-center/manage-service-limits) for your Retool deployment's AWS Region as you scale.

## System architecture

The following diagram shows the resulting system architecture for your deployment.

## 1. Create a Linux EC2 instance

Create a new Linux EC2 instance in the [Amazon EC2 console](https://console.aws.amazon.com/ec2/) using an Amazon Machine Image (AMI) that meets the minimum requirements. Refer to the [Amazon EC2 documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EC2_GetStarted.html) to learn how to create a new instance.

| VM setting         | Description                                                                                                  |
| ------------------ | ------------------------------------------------------------------------------------------------------------ |
| **Authentication** | Enable **SSH public key** authentication and provide a username. Generate a new key-pair and specify a name. |
| **Network**        | Create or use an existing security group to configure rules for SSH (`22`), HTTP (`80`), HTTPS (`443`), and Custom TCP (`3000`). The source for these rules must be `0.0.0.0/0` and `::/0`. |

Self-hosted Retool initially runs on port `3000`. Once SSL is configured, this port is no longer required.

## 2. Download Self-hosted Retool

You can connect in the [AWS console](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-connect-methods.html) using EC2 Instance Connect, or on the command line using an SSH client, with the SSH key pair you selected in your EC2 dashboard.

```bash title="Connect using SSH"
ssh -i keypair.pem @
```

## 3. Install Docker dependencies

## 4. Back up encryption key

## 5. Configure the instance

## 6. Start the instance

" />

## Additional configuration

## Update your EC2 instance

---

## Deploy Self-hosted Retool on AWS Fargate and ECS with CloudFormation

import TemporalOptions from "../../_partials/_temporal-options.mdx";
import EncryptionKey from "../../_partials/_encryption-key.mdx";
import Egress from "../../_partials/_egress.mdx";
import EnvVarsCloud from "../../_partials/_env-vars-cloud.mdx";
import EnvVarsLocal from "../../_partials/_env-vars-local.mdx";
import SelfhostedLatest from "@site/src/components/SelfhostedLatest";
import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

You can deploy Self-hosted Retool on AWS Fargate using CloudFormation templates. 

## Requirements

To deploy Self-hosted Retool on AWS Fargate and ECS, you need:

- A Retool license key, which you can obtain from the [Retool Self-hosted Portal](https://my.retool.com/) or your Retool account manager.
- An [AWS account](https://aws.amazon.com/marketplace/management/signin).
- An [ECS cluster](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/create-cluster-console-v2.html) with the **AWS Fargate** launch type.

## 1. Verify network configuration

The CloudFormation template you choose depends on whether you need to deploy with or without public subnets.

- Use the [public template](https://raw.githubusercontent.com/tryretool/retool-onpremise/master/cloudformation/fargate.yaml) to deploy Self-hosted Retool in VPCs *with* public subnets.
- Use the [private template](https://raw.githubusercontent.com/tryretool/retool-onpremise/master/cloudformation/fargate.private.yaml) to deploy Self-hosted Retool in VPCs *without* public subnets.

Use the [public template](https://raw.githubusercontent.com/tryretool/retool-onpremise/master/cloudformation/fargate.yaml) to deploy Retool in VPCs with public subnets. This requires the following networking components:

- One VPC with internet access.
- Two public subnets.

You use the [private template](https://raw.githubusercontent.com/tryretool/retool-onpremise/master/cloudformation/fargate.private.yaml) to deploy Retool in VPCs without public subnets. To use this installation, set up a [NAT gateway](https://repost.aws/knowledge-center/nat-gateway-vpc-private-subnet) in front of your private subnets. This requires the following networking components:

- One VPC with internet access
- Two private subnets, attached to NAT gateways

### VPC

1. Open the AWS Management Console and navigate to the VPC service to use. Copy the value of the VPC ID.
2. In the left navigation pane, select **Internet Gateways**.
3. In the list of Internet Gateways, find the Internet Gateway associated with your VPC. If there is a record in this table associated with your VPC, the VPC is public.

### Subnets

Verify that your subnets are public. Subnet IDs are required in a later step.

1. Open the AWS Management Console and navigate to the VPC service.
2. In the left navigation pane, select **Subnets**.
3. Identify at least two subnets which are part of the VPC.
4. Confirm that the subnets are public subnets. For each subnet, select the **Route table** tab. Look for an entry where the **Destination** is `0.0.0.0/0` and the **Target** is an Internet Gateway (`igw-xxxx`). If such an entry exists, the subnet is a public subnet.
5. Confirm that the subnets have auto-assign public IP addresses enabled. For each subnet, select the subnet and go to the **Details** tab in the lower pane. Look for the **Auto-assign public IPv4 address** attribute. Confirm that the value is **Yes**.

Verify that your subnets are connected to a NAT gateway. Subnet IDs are required in a later step.

1. Open the AWS Management Console and navigate to the VPC service.
2. In the left navigation pane, click on **Subnets**.
3. Identify at least two subnets which are part of the VPC.
4. Confirm that the subnets are connected to the NAT gateway. For each subnet, select the **Route table** tab. Look for an entry where the **Destination** is `0.0.0.0/0` and the **Target** is a NAT Gateway. If such an entry exists, the subnet has access to the NAT gateway.

## 2. Configure instance

:::danger Compare your existing template with the newer version

If you previously deployed Self-hosted Retool with an older version of a CloudFormation template, compare your current template with the new one. If the `RetoolRDSInstance` object has changed, this could delete your current database instance.

:::

Download the private CloudFormation template for either **ECS on Fargate** or **ECS on EC2** from the [Self-hosted Retool GitHub repository](https://github.com/tryretool/retool-onpremise). Both of these templates assume a deployment in private subnets of your VPC (with NAT gateway) along with an Application Load Balancer (ALB) to direct external traffic to the Retool ECS service.

```bash title="Template for Workflows"
curl -L -O  https://raw.githubusercontent.com/tryretool/retool-onpremise/master/cloudformation/retool-workflows.fargate.yaml \
&& mv retool-workflows.fargate.yaml fargate.yaml
```

```bash title="Template for Agents"
curl -L -O  https://raw.githubusercontent.com/tryretool/retool-onpremise/master/cloudformation/retool-agents.fargate.yaml \
&& mv retool-agents.fargate.yaml fargate.yaml
```
These two templates are similar, however, the workflows template differs from the agents template in the following ways: 
 - **Parameters**: The agents template declares a `DesiredAgentsCount` and defaults the Retool version to 3.253.2-stable, and the workflows template contains a `Force` parameter, and defaults to Retool version 3.114.7-stable. 
 - **Environment variables**: The workflows template injects `NODE_ENV=production` and a `FORCE_DEPLOYMENT` env var (from the Force parameter) into several task definitions. The agents template sets `AGENTS_ENABLED` and `BASE_DOMAIN` in the `Retool/backend` tasks. 
 - **Temporal**: The temporal versions differ slightly, agents uses 14.6, and workflows uses 14.5. 

Refer to the [System architecture of self-hosted Retool deployments](../../concepts/architecture.mdx) conceptual guide for more information about the containers and services. 

```bash
curl -L -O  https://raw.githubusercontent.com/tryretool/retool-onpremise/master/cloudformation/retool-workflows.ec2.yaml \
&& mv retool-workflows.fargate.yaml fargate.yaml
```

### Configure Temporal

  

Comment out the **Temporal** configuration section in either [fargate.yaml](https://github.com/tryretool/retool-onpremise/blob/master/cloudformation/retool-workflows.fargate.yaml#L603-L1031) or [ec2.yaml](https://github.com/tryretool/retool-onpremise/blob/master/cloudformation/retool-workflows.ec2.yaml#L563-L971), depending on the type of deployment used.

  
    

Comment out the **Temporal** configuration section in either [fargate.yaml](https://github.com/tryretool/retool-onpremise/blob/master/cloudformation/retool-workflows.fargate.yaml#L603-L1031) or [ec2.yaml](https://github.com/tryretool/retool-onpremise/blob/master/cloudformation/retool-workflows.ec2.yaml#L563-L971), depending on the type of deployment used.

  
  

  

### Add your license key

Edit the configuration and include your license key. Replace all values of `LICENSE_KEY` with your license key.

```yaml title="fargate.yaml"
Environment:
  - Name: LICENSE_KEY
// highlight-next-line
    Value: "EXPIRED-LICENSE-KEY-TRIAL"
```

## 3. Configure CloudFormation service

Next, log into the AWS Management Console to configure the CloudFormation service:

1. Navigate to the CloudFormation service and create a new stack.
2. Upload the `fargate.yaml` file.
3. Set the following parameters.

| Parameter      | Value                                                                                                                     |
| -------------- | ------------------------------------------------------------------------------------------------------------------------- |
| `cluster`        | The name of your ECS cluster.                                                                                             |
| `desiredCount`   | `2`                                                                                                                       |
| `environment`    | `Staging`                                                                                                                 |
| `force`          | `false`                                                                                                                   |
| `image`          | The [Docker tag](https://hub.docker.com/r/tryretool/backend/tags) for the version of Retool to install, such as tryretool/backend:. |
| `maximumPercent` | `250`                                                                                                                     |
| `minimumPercent` | `50`                                                                                                                      |
| `subnetID`       | Two subnets you identified in the networking requirements section.                                                        |
| `vpcID`          | The VPC ID to use.                                                                                                        |

After creating the stack, verify its status is `CREATE_COMPLETE`.

## 4. Start the instance

The **Outputs** tab of the CloudFormation stack contains the URL of the load balancer running Self-hosted Retool. Once running, your instance is available at `http://{load-balancer-url}:3000/auth/signup`. When you first visit the page, you must create an admin account.

## Additional steps

Retool strongly recommends you externalize your database, configure SSL, and keep up-to-date with the latest version of Self-hosted Retool. Setting environment variables is often necessary to configure SSO, source control, and other self-hosted features.

### Externalize database

By default, the Retool ECS template creates a new RDS instance to serve as the PostgreSQL database. To set up the ECS deployment to use a different database, you need to update the CloudFormation template by modifying the environment variables for both the `RetoolTask` and the `RetoolJobsRunnerTask`.

1. Modify `fargate.yaml`, setting the following environment variables for both the `RetoolTask` and the `RetoolJobsRunnerTask`.

| Variable            | Description                                          |
| ------------------- | ---------------------------------------------------- |
| `POSTGRES_DB`       | The name of the external database.                   |
| `POSTGRES_HOST`     | The hostname of your external database instance.     |
| `POSTGRES_PORT`     | The port number for your external database instance. |
| `POSTGRES_USER`     | The external database username.                      |
| `POSTGRES_PASSWORD` | The external database password.                      |

2. In the AWS Management Console, navigate to the CloudFormation service. Select the Retool stack.
3. Update the stack. Replace the current template by uploading the new version of `fargate.yaml`.
4. Submit changes, and wait for the stack to redeploy. Verify that the CloudFormation stack has a status of `CREATE_COMPLETE` before continuing.

### Update Retool

1. Back up your database. Amazon RDS provides two different methods for backing up and restoring your DB instances: automated backups and database snapshots.
2. Identify the appropriate release version on [Docker Hub](https://hub.docker.com/r/tryretool/backend). See Retool's [self-hosted release notes](../../../releases/index.mdx) to learn about version-specific features.
3. In the AWS Management Console, navigate to the CloudFormation service. Select the Retool stack.
4. Update the stack. Use the current template. On the parameters screen, change the value of `Image` to the [Docker tag](https://hub.docker.com/r/tryretool/backend/tags) for the version of Retool to use, such as tryretool/backend:.
5. Submit changes, and wait for the stack to redeploy. Verify that the CloudFormation stack has a status of `CREATE_COMPLETE` before continuing.

### Add environment variables

To add [environment variables](https://docs.retool.com/reference/environment-variables), follow the steps below.

1. Modify `fargate.yaml`, setting the environment variables for both the `RetoolTask` and the `RetoolJobsRunnerTask`.
2. In the AWS Management Console, navigate to the CloudFormation service. Select the Retool stack.
3. Update the stack. Replace the current template by uploading the new version of `fargate.yaml` from your computer.
4. Submit changes, and wait for the stack to redeploy. Verify that the CloudFormation stack has a status of `CREATE_COMPLETE` before continuing.

### Configure SSL

To configure SSL, you must first obtain an SSL certificate. You can either purchase an SSL certificate from a Certificate Authority (CA) or generate a free one using Let's Encrypt. AWS also provides a service called AWS Certificate Manager (ACM) for provisioning, managing, and deploying public and private SSL/TLS certificates.

To add your SLS certificate to your instance:

1. In the AWS Management Console, navigate to the EC2 service.
2. Import your SSL certificate to AWS Certificate Manager (ACM).
3. In the AWS Management Console, navigate to the EC2 service. Under **Load Balancing**, choose Load Balancers.
4. Select the load balancer that is part of the Retool deployment.
5. In the **Listeners** tab, select **Add listener**.
6. Select **HTTPS** (port 443). Select the ACM, which you created in step 2.
7. Add the listener. You can now access Retool with a SSL connection.

### Mount volumes

There are several use cases which require the use of [volumes](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/efs-volumes.html). For example, when configuring a [gRPC resource](../../../data-sources/guides/integrations/development/grpc.mdx), you need to mount a volume containing the protos files to the Retool deployment. Follow these instructions to create a persistent volume and copy files from your local machine to the volume.

First, choose a method to upload files to the EFS volume. You can either:

- Mount the EFS volume on an EC2 instance and use standard file operations to upload files directly to the mounted directory on the EFS volume.
- Use [AWS DataSync](https://aws.amazon.com/datasync/) to transfer your local files to an EFS file system by setting up a DataSync agent and configuring a data transfer task.
- Set up an SFTP server using [AWS Transfer](https://docs.aws.amazon.com/transfer/index.html) for SFTP and configure it to use an EFS file system as the storage backend, then upload files using an SFTP client to store them on the EFS volume.

1. In the AWS Management Console, navigate to the EFS service.
2. Create a new file system. Select the same VPC in which you deploy Retool.
3. Upload files to the EFS volume using your data transfer method of choice.
4. In `fargate.yaml`, add the EFS volume as a parameter in the `Parameters` section.

```yaml title="fargate.yaml"
Parameters:
  ...

  EFS:
    Type: AWS::EFS::FileSystem::Id
    Description: Select an existing EFS volume to mount.
```

5. In `fargate.yaml`, add an `EFSMountTarget` to `Resources` section.

```yaml title="fargate.yaml"
Resources:
  EFSMountTarget:
    Type: AWS::EFS::MountTarget
    Properties:
      FileSystemId: !Ref 'EFS'
      SecurityGroups: [!Ref 'ALBSecurityGroup']
      SubnetId: !Select [0, !Ref 'SubnetId']
```

6. In `fargate.yaml`, add the volume to the `RetoolTask`.

```yaml title="fargate.yaml"
  RetoolTask:
    Type: AWS::ECS::TaskDefinition
    Properties:
        ...

      ContainerDefinitions:
        ...

        Volumes:
            - Name: efs-volume
            EFSVolumeConfiguration:
                FileSystemId: !Ref 'EFS'
        MountPoints:
          - SourceVolume: efs-volume
            ContainerPath: /efs
            ReadOnly: false  # Set to 'true' if you want read-only access
```

6. In the AWS Management Console, navigate to the CloudFormation service. Select the Retool stack.
7. Update the stack. Replace the current template by uploading the new version of `fargate.yaml`.
8. Submit changes, and wait for the stack to redeploy.

---

## Deploy Self-hosted Retool on AWS Fargate and ECS

import DocCardList from "@theme/DocCardList";

---

## Deploy Self-hosted Retool on AWS Fargate and ECS with Terraform

import TemporalOptions from "../../_partials/_temporal-options.mdx";
import TemporalConfigure from "../../_partials/_temporal-configure.mdx";
import SelfhostedLatest from "@site/src/components/SelfhostedLatest";
import CodeBlock from '@theme/CodeBlock';
import EncryptionKey from "../../_partials/_encryption-key.mdx";
import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

You can deploy Self-hosted Retool on ECS Fargate and EC2 with a Terraform module.

## Requirements

To deploy Self-hosted Retool on AWS Fargate and ECS, you need:

- A Retool license key, which you can obtain from the [Retool Self-hosted Portal](https://my.retool.com/) or your Retool account manager.
- An [AWS account](https://aws.amazon.com/marketplace/management/signin).
- An [ECS cluster](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/create-cluster-console-v2.html) with the Fargate launch type.

Retool recommends using the Retool-provided [Terraform module](https://github.com/tryretool/terraform-retool-modules/tree/main/modules/aws_ecs) for ECS deployments.

## 1. Update Terraform configuration

:::info Minimum requirements 

A [Retool-managed Temporal Cluster](../../concepts/temporal.mdx#compare-options) requires v3.6.15 or later.

:::

First, update your Terraform configuration to directly use the module. Next, set `ecs_retool_image` to the [Docker tag](https://hub.docker.com/r/tryretool/backend/tags) for the version of Retool to install, such as tryretool/backend:.

:::caution Avoid using `latest`

Specify the exact version to use, such as . This ensures you know exactly which version will be deployed.

:::

If you are not using Fargate, set `launch_type` to `EC2`. Retool recommends `t3.xlarge` instances for EC2-backed ECS as this avoids Elastic Network Interface (ENI) limits since each ECS Service requires an ENI.

Ensure that `workflows_enabled=true`.

```json title="Terraform configuration"
module "retool" {
  source = "git@github.com:tryretool/terraform-retool-modules.git//modules/aws_ecs"

  aws_region = ""
  vpc_id = ""
  subnet_ids = [
      "",
      ""
  ]
  ssh_key_name = ""
  retool_license_key = ""
  ecs_retool_image = ""
  launch_type = "FARGATE" | "EC2"
  // highlight-start
  workflows_enabled = true
  // highlight-end
  ...
}
```

### Configure Terraform for Temporal

Update your Terraform configuration to configure Temporal:

  

```json
module "retool" {
  ...
  workflows_enabled = true
  // highlight-start
  use_existing_temporal_cluster = true
  // temporal_cluster_config = 
  // highlight-end
  ...
}
```

```json
module "retool" {
  ...
  workflows_enabled = true
  // highlight-start
  use_existing_temporal_cluster = true
  temporal_cluster_config = 
  // highlight-end
  ...
}
```

:::info

Use the default values when configuring an existing Temporal Cluster.

:::

```json
module "retool" {
    ...
    workflows_enabled = false
    // highlight-start
    use_existing_temporal_cluster = false
    // temporal_cluster_config = 
    // highlight-end
    ...
}
```

## 2. Install module requirements

Run `terraform init` to install all requirements for the module.

## 3. Verify security settings additional input variables

Ensure that the default security settings in `security.tf` matches your specifications. If you need to restrict access further, you can configure ingress and egress rules into `container_egress_rules`, `container_ingress_rules`, `alb_egress_rules`, and `alb_ingress_rules`.

Check `variables.tf` for any other input variables that may be required.

## 4. Review and apply changes

Run `terraform plan` to view all planned changes to your account. Once complete, run `terraform apply` to apply the changes and deploy Self-hosted Retool.

A load balancer is then associated with the deployment and available in the AWS EC2 Console. The instance address should now be running Self-hosted Retool.

---

## Deploy Retool on Google Compute Engine

import Requirements from "../_partials/_requirements.mdx";
import DockerDownload from "../_partials/_docker_download.mdx";
import DockerInstall from "../_partials/_docker_install.mdx";
import DockerKey from "../_partials/_docker_encryption_key.mdx";
import DockerConfigure from "../_partials/_docker_configure.mdx";
import DockerUpdate from "../_partials/_update.mdx";
import TemporalOptions from "../_partials/_temporal-options.mdx";
import Production from "../_partials/_additional-steps.mdx";
import DockerStart from "../_partials/_docker_start.mdx";
import SelfhostedLatest from "@site/src/components/SelfhostedLatest";
import EnterpriseRequirement from "../_partials/_enterprise-plan-requirement.mdx";

You can deploy an instance of Self-hosted Retool on Google Compute Engine with [Docker Compose](https://docs.docker.com/compose/).

## Requirements

## 1. Create a Linux VM instance

Create a new Linux VM in the [Google Cloud console](https://console.cloud.google.com/) that meets the minimum requirements. Refer to the [Google Cloud documentation](https://cloud.google.com/compute/docs/create-linux-vm-instance) to learn how to create a new virtual machine.

| VM setting  | Description                                                                            |
| ----------- | -------------------------------------------------------------------------------------- |
| **Network** | Configure the VM to allow HTTP and HTTPS traffic.                                      |
| **Storage** | Specify **SSD persistent** with a size that meets or exceeds the minimum requirements. |

Once the VM is initialized, download the private key when prompted. Update the permissions of the key-pair to ensure it has the required permissions for your SSH client.

```shell
chmod 400 
```

## 2. Download Self-hosted Retool

GCP automatically runs the new virtual machine once created. Once the new Linux VM is ready, use the Google Cloud console to connect using SSH.

## 3. Install Docker dependencies

## 4. Back up encryption key

## 5. Configure deployment

## 6. Start the instance

" />

## Additional configuration

## Update your Google Compute Engine instance

---

## Deploy Retool on Kubernetes with Helm

import Requirements from "../../_partials/_kubernetes-requirements.mdx"
import Helm1 from "../../_partials/_helm1.mdx"
import Helm2 from "../../_partials/_helm2.mdx"
import Helm3 from "../../_partials/_helm3.mdx"
import TemporalOptions from "../../_partials/_temporal-options.mdx"
import EncryptionKey from "../../_partials/_encryption-key.mdx"
import Egress from "../../_partials/_egress.mdx"
import EnvVarsCloud from "../../_partials/_env-vars-cloud-helm.mdx"
import EnvVarsLocal from "../../_partials/_env-vars-local-helm.mdx"
import ExternalizeDb from "../../_partials/_externalize-db.mdx"
import LetsEncrypt from "../../_partials/_lets-encrypt.mdx"
import SelfhostedLatest from "@site/src/components/SelfhostedLatest";
import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

You can deploy Self-hosted Retool on Kubernetes with [Helm](https://helm.sh/) 3.3.1 or later.

## Requirements

## 1. Add the Retool Helm chart repository

## 2. Download Helm configuration file

## 3. Update Helm configuration

## 4. Install Self-hosted Retool

After updating the configuration, install Self-hosted Retool.

```shell
helm install my-retool retool/retool -f values.yaml
```

After installing Retool, run `kubectl get pods` to verify you have pods for the main service and `jobs-runner`. If you use the PostgreSQL subchart, there is also a `postgresql` pod. If you have enabled Workflows, there are also `workflow-worker` and `workflow-backend` pods.

```text title="Verify pods"
my-retool-7898474bbd-pr8n6               1/1     Running   1 (8h ago)   8h
my-retool-jobs-runner-74796ddd99-dd856   1/1     Running   0            8h
my-retool-postgresql-0                   1/1     Running   0            8h
```

Once the main service is running, verify the installation by port forwarding to `localhost`.

```shell
kubectl port-forward my-retool-7898474bbd-69zjt 3000:3000
```

You can then access Retool at `http://localhost:3000/`.

## Additional configuration

The following configuration steps are optional but strongly recommended for using Retool in a production environment.

:::note

Whenever you run `helm upgrade`, use the `--version` flag to specify the chart's version number. Otherwise, Helm upgrades to the latest chart version, which may cause compatibility issues. You can check the release version of your deployment with the command `helm list`.

:::

### Add environment variables

[Environment variables](https://docs.retool.com/reference/environment-variables) provide ways to configure a Retool instance. The `values.yaml` file has three locations to add [environment variables](../../reference/environment-variables/index.mdx).

| Object                 | Type                             |
| ---------------------- | -------------------------------- |
| `env`                  | Plain text key-value pairs.       |
| `environmentSecrets`   | Plain text or Kubernetes secrets. |
| `environmentVariables` | Plain text or Kubernetes secrets. |

:::caution

Do not store sensitive information, such as access tokens, in `env`. Use `environmentSecrets` or `environmentVariables` as they can populate environment variables from Kubernetes secrets.

:::

### Mount volumes

There are several use cases which require the use of [volumes](https://kubernetes.io/docs/concepts/storage/volumes/). For example, when configuring a [gRPC resource](../../../data-sources/guides/integrations/development/grpc.mdx), you need to mount a volume containing the protos files to the Retool deployment. Follow these instructions to create a persistent volume and copy files from your local machine to the volume.

#### 1. Enable PersistentVolumeClaim

The Helm chart defines a PersistentVolumeClaim (PVC) which is automatically mounted to the Retool pods, enabling Retool to access files within this volume. The PVC is disabled by default. To enable the `persistentVolumeClaim`, modify your `values.yaml` file:

```yaml
persistentVolumeClaim:
  enabled: true
  existingClaim: ""
```

If you have an existing PVC in your Kubernetes cluster to use, you can specify its name in `existingClaim`. Otherwise, leave `existingClaim` blank.

#### 2. Set security context

In a later step, you use [`kubectl cp`](https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#cp) to copy files from your local machine to the Kubernetes cluster, which requires the pod to run with root privileges. Modify your deployment so the pods run as root by changing the `securityContext` in your `values.yaml` file:

```yaml
securityContext:
  enabled: true
  runAsUser: 0
```

Use `helm` to perform the upgrade and include the Helm chart version number. Retool requires version `6.1.1` or later.

```shell
helm upgrade -f values.yaml my-retool retool/retool --version 6.1.1
```

#### 3. Verify pods

Run `kubetcl get pods` to verify pods are running.

```text
my-retool-7898474bbd-pr8n6               1/1     Running   1 (8h ago)   8h
my-retool-jobs-runner-74796ddd99-dd856   1/1     Running   0            8h
my-retool-postgresql-0                   1/1     Running   0            8h
```

#### 4. Copy files

Next, copy the protos files from your local machine to the PVC. Note from `kubectl get pods` the three pods in the deployment: the main, `jobs-runner`, and `postgresql` containers. Identify the name of the main container.

Ensure you local machine has a folder named `protos` and run the following command, and replacing `my-retool-7c4c89798-fqbh7` with the name of your Retool container.

```shell
kubectl cp protos/ my-retool-7c4c89798-fqbh7:/retool_backend/pv-data/protos
```

#### 4. Set env

If you're configuring gRPC, you need to specify the location of the protos directory. In `values.yaml`, set the `PROTO_DIRECTORY_PATH` environment variable.

```yaml
env:
  PROTO_DIRECTORY_PATH: "/retool_backend/pv-data/protos"
```

#### 5. Reset security context

Revert the security context of your deployment back to a disabled state.

```yaml
securityContext:
  enabled: false
  runAsUser: 1000
```

Use `helm` to perform the upgrade and include the Helm chart version number. Retool requires version `6.1.1` or later.

```shell
helm upgrade -f values.yaml my-retool retool/retool --version 6.1.1
```

### Configure SSL

When configuring SSL, you can use Let's Encrypt to provision a certificate, or provide your own. See [Configure SSL and custom certificates](../../guides/certificates.mdx) for more detail on certificates.

#### 1. Install cert-manager

First, add the `jetstack` Helm repository if you haven't already.

```shell
helm repo add jetstack https://charts.jetstack.io
```

Next, run the following command to install `cert-manager`.

```shell
helm install \
cert-manager jetstack/cert-manager \
--namespace cert-manager \
--create-namespace \
--version v1.11.0 \
--set installCRDs=true --set ingressShim.defaultIssuerName=letsencrypt-prod \
--set ingressShim.defaultIssuerKind=ClusterIssuer \
--set ingressShim.defaultIssuerGroup=cert-manager.io
```

#### 6. Apply changes

After the pods restart, you can access the page in your browser using TLS.

## Update your Kubernetes instance

import Update from "../../_partials/_update-helm-retool.mdx"

---

## Deploy Self-hosted Retool on Kubernetes

You can deploy Self-hosted Retool on Kubernetes using the Helm package manager or manually configure manifests.

import DocCardList from "@theme/DocCardList";

---

## Deploy Retool on Kubernetes with manifests

import Requirements from "../../_partials/_kubernetes-requirements.mdx"
import TemporalConfigure from "../../_partials/_temporal-configure.mdx";
import TemporalTemplate from "../../_partials/_temporal-template.mdx";
import SelfhostedLatest from "@site/src/components/SelfhostedLatest";
import CodeBlock from '@theme/CodeBlock';
import PostgresConfig from "../../_partials/_remaining-postgresql-config.mdx"
import LetsEncrypt from "../../_partials/_lets-encrypt.mdx"
import ExternalizeDb from "../../_partials/_externalize-db.mdx"
import EnterpriseRequirement from "../../_partials/_enterprise-plan-requirement.mdx";

You can deploy Self-hosted Retool on Kubernetes using manually configured manifests. You can use this approach if you do not manage packages with [Helm](helm.mdx).

## Requirements

  

Self-hosted Retool with Workflows deployments on Kubernetes are configured using a [set of manifests](https://github.com/tryretool/retool-onpremise/tree/master/kubernetes). To retrieve a copy of the manifests, download the `retool-onpremise` repository to your local machine. Open the `kubernetes` directory in an IDE to follow along the steps below.

```shell
curl -L -O https://github.com/tryretool/retool-onpremise/archive/master.zip && unzip master.zip \
&& cd retool-onpremise-master/kubernetes
```

  
  

Self-hosted Retool with Workflows deployments on Kubernetes are configured using a [set of manifests](https://github.com/tryretool/retool-onpremise/tree/master/kubernetes). To retrieve a copy of the manifests, download the `retool-onpremise` repository to your local machine. Open the `kubernetes` directory in an IDE to follow along the steps below.

```shell
curl -L -O https://github.com/tryretool/retool-onpremise/archive/master.zip && unzip master.zip \
&& cd retool-onpremise-master/kubernetes
```

  
  

Self-hosted Retool with Workflows deployments on Kubernetes are configured using a [set of manifests](https://github.com/tryretool/retool-onpremise/tree/master/kubernetes). To retrieve a copy of the manifests, download the `retool-onpremise` repository to your local machine. Open the `kubernetes-with-temporal` directory in an IDE to follow along the steps below.

```
curl -L -O https://github.com/tryretool/retool-onpremise/archive/master.zip && unzip master.zip \
&& cd retool-onpremise-master/kubernetes-with-temporal
```

  

## 1. Configure version

:::info

Self-hosted Retool requires 3.6.14 or later for Retool-managed Temporal Cluster.

:::

Set the `image` value in the following files to the [Docker tag](https://hub.docker.com/r/tryretool/backend/tags) for the version of Retool to install, such as tryretool/backend:.

- `retool-container.yaml`
- `retool-jobs-runner.yaml`
- `retool-workflows-worker-container.yaml`
- `retool-workflows-backend-container.yaml`

In `retool-code-executor-container.yaml`, change the `image` tag to indicate the [version of Retool's code executor service](https://hub.docker.com/r/tryretool/code-executor-service/tags) to install. This must match the version of Retool for which you're deploying.

  {`image: tryretool/code-executor-service:`}{}

## 2. Update configuration

  

  
  

  
  

Copy `retool-temporal-secrets.template.yaml` to a new file named `retool-temporal-secrets.yaml`. This file sets the configuration options for Temporal in your deployment, and stores them as [Kubernetes secrets](https://kubernetes.io/docs/concepts/configuration/secret/). These values must be Base64-encoded.

```shell
cp retool-temporal-secrets.template.yaml retool-temporal-secrets.yaml
```

| Setting                  | Description                            |
| ------------------------ | -------------------------------------- |
| `data.postgres_password` | The same database password used above. |

 

## Additional configuration

The following configuration steps are optional but strongly recommended for using Retool in a production environment.

### Add environment variables

[Environment variables](https://docs.retool.com/reference/environment-variables) provide ways to configure a Retool instance.

#### 1. Update manifests

Configure environment variables in the following files:

- `retool-container.yaml`
- `retool-jobs-runner.yaml`
- `retool-workflows-worker-container.yaml`
- `retool-workflows-backend-container.yaml`

The following example configures the [`DBCONNECTOR_QUERY_TIMEOUT_MS`](https://docs.retool.com/self-hosted/reference/environment-variables/index.mdx#dbconnector_query_timeout_ms) variable, but this pattern applies to other [environment variables](https://docs.retool.com/reference/environment-variables) as well.

```yaml
env:
- name: DBCONNECTOR_QUERY_TIMEOUT_MS
  value: 360000
```

#### 2. Apply changes to the manifests

Use `kubectl` to apply manifest changes.

  

```shell
kubectl apply -f -R kubernetes
```

  
  

```shell
kubectl apply -f -R kubernetes
```

  
  

```shell
kubectl apply -f -R kubernetes-with-temporal
```

  

#### 3. Verify pods

Run `kubetcl get pods` to verify pods are running.

```shell
NAME                                     READY   STATUS    RESTARTS     AGE
api-76464f5576-vc5f4                     1/1     Running   1 (8h ago)   8h
jobs-runner-5cfb79cbfd-b49rd             1/1     Running   0            8h
postgres-69c485649c-lkjgc                1/1     Running   0            8h
...
```

### Mount volumes

There are several use cases which require the use of [volumes](https://kubernetes.io/docs/concepts/storage/volumes/). For example, when configuring a [gRPC resource](../../../data-sources/guides/integrations/development/grpc.mdx), you need to mount a volume containing the protos files to the Retool deployment. Follow these instructions to create a persistent volume and copy files from your local machine to the volume.

#### 1. Set security context

In a later step, you use [`kubectl cp`](https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#cp) to copy files from your local machine to the Kubernetes cluster, which requires the pod to run with root privileges.

Modify your deployment so the pods run as root by adding the `securityContext` in the `retool-container.yaml` file:

```yaml
spec:
      securityContext:
        runAsUser: 0
        fsGroup: 2000
```

#### 2. Apply changes to the manifest

Use `kubectl` to apply manifest changes.

```shell
kubectl apply -f retool-container.yaml
```

#### 3. Verify pods

Run `kubectl get pods` to verify that pods are ready.

```shell
NAME                                     READY   STATUS    RESTARTS     AGE
api-76464f5576-vc5f4                     1/1     Running   1 (8h ago)   8h
jobs-runner-5cfb79cbfd-b49rd             1/1     Running   0            8h
postgres-69c485649c-lkjgc                1/1     Running   0            8h
...
```

#### 4. Copy protos files

Next, copy the protos files from your local machine to the PVC. Ensure you local machine has a folder named `protos` and run the following command. Replace `api-76464f5576-vc5f4` with the name of your main Retool container, retrieved from `kubectl get pods`.

```shell
kubectl cp protos/ api-76464f5576-vc5f4:/retool_backend/pv-data/protos
```

#### 5. Set directory path

If you're configuring gRPC, specify the location of the protos directory. In `retool-container.yaml`, set the `PROTO_DIRECTORY_PATH` environment variable.

```yaml
env:
- name: PROTO_DIRECTORY_PATH
  value: "/retool_backend/pv-data/protos"
```

#### 6. Reset security context

Reset the security context of your deployment by removing the `securityContext` field, or by defining a non-root user.

Apply changes to the manifest.

```shell
kubectl apply -f retool-container.yaml
```

### Configure SSL

When configuring SSL, you can use Let's Encrypt to provision a certificate, or provide your own. See [Configure SSL and custom certificates](../../guides/certificates.mdx) for more detail on certificates.

#### 1. Generate self-signed certificate

Generate a self-signed certificate and private key using `openssl`. Replace `KEY_FILE`, `CERT_FILE`, and `HOST` with your key file name, certificate file name, and hostname, or set environment variables naming each.

```shell
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ${KEY_FILE} -out ${CERT_FILE} -subj "/CN=${HOST}/O=${HOST}" -addext "subjectAltName = DNS:${HOST}"
```

#### 2. Create a TLS secret

Create a TLS secret using `kubectl`. Replace `KEY_FILE` and `CERT_FILE` with your key and certificate file names.

```shell
kubectl create secret tls ${CERT_NAME} --key ${KEY_FILE} --cert ${CERT_FILE}
```

#### 3. Install the Ingress-Nginx Controller

If you haven't already, install the [Nginx-Ingress Controller](https://kubernetes.github.io/ingress-nginx/deploy/#installation-guide) using the instructions for your environment. To confirm the installation was successful, run the following command. Its output should contain an entry for the **ingress-nginx-controller** pod.

```shell
kubectl get pods -n ingress-nginx
```

#### 4. Create an ingress resource

Create an ingress resource with the following manifest. Replace `retool.example.com` with your domain, and `testsecret-tls` with your TLS secret. See the [TLS section of the Kubernetes Ingress documentation](https://kubernetes.io/docs/concepts/services-networking/ingress/#tls) for more information.

If you haven't already, install the [Ingress-Nginx Controller](https://kubernetes.github.io/ingress-nginx/deploy/#installation-guide) using the instructions for your environment. To confirm the installation was successful, run the following command. Its output should contain an entry for the **ingress-nginx-controller** pod.

```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: tls-example-ingress
  annotations:
    kubernetes.io/tls-acme: "true"
spec:
  ingressClassName: nginx
  tls:
    - hosts:
      - retool.example.com
  secretName: testsecret-tls
  rules:
  - host: retool.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
          name: api
          port:
            number: 3000
```

#### 1. Install cert-manager

Use `kubectl` to install [cert-manager](https://github.com/cert-manager/cert-manager).

```shell
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.11.0/cert-manager.yam./l
```

#### 6. Apply changes to the manifests

Use `kubectl` to apply manifest changes. After the pods restart, you can access the page in your browser using TLS.

## Update your Kubernetes instance

Follow these instructions to update your Retool instance to a newer release version.

### 1. Back up your database

 If you use a managed database service, your database provider may have a feature to take snapshots or otherwise back up your database. If you use the PostgreSQL subchart, run the following command to export data from the PostgreSQL pod to a `.sql` file.

```shell
kubectl exec -it  -- bash -c 'pg_dump hammerhead_production --no-acl --no-owner --clean -U postgres' > retool_db_dump.sql
```

### 2. Select a new version

Set the `image` value in the following files to the [Docker tag](https://hub.docker.com/r/tryretool/backend/tags) for the version of Retool to install, such as tryretool/backend:.

- `retool-container.yaml`
- `retool-jobs-runner.yaml`
- `retool-workflows-worker-container.yaml`
- `retool-workflows-backend-container.yaml`

  {`image: tryretool/backend:`}{}

### 3. Apply changes to the manifests

Use `kubectl` to apply manifest changes.

  

```shell
kubectl apply -f -R kubernetes
```

  
  

```shell
kubectl apply -f -R kubernetes
```

  
  

```shell
kubectl apply -f -R kubernetes-with-temporal
```

  

### 4. Verify pods

Run `kubectl get pods` to verify that the update has completed.

```
NAME                                     READY   STATUS    RESTARTS     AGE
api-76464f5576-vc5f4                     1/1     Running   1 (8h ago)   8h
jobs-runner-5cfb79cbfd-b49rd             1/1     Running   0            8h
postgres-69c485649c-lkjgc                1/1     Running   0            8h
...
```

---

## Self-hosted Retool tutorials

import Tutorial from '/docs/_partials/_doctypes/_tutorial.mdx';
import EnterpriseRequirement from "./_partials/_enterprise-plan-requirement.mdx";

:::tip Learn more about self-hosted architecture

Read the [system architecture](concepts/architecture.mdx) guide to learn more about the containers, services, and dependencies for self-hosted deployment instances.

:::

---

## Catch-up commits in Source Control

In order to keep your branch up to date, Retool sometimes creates an automatic commit, called a catch-up commit, which keeps your branches up to date with the `main` branch. The commit has the following message:

```
This commit is automatically generated by Retool to update the current branch. 
You don't need to be concerned about it.
```

:::note

You can also use branch merging as an alternative to catch-up commits. Turn on branch merging in **Settings** > **Beta**. When branch merging is turned on, catch-up commits are disabled. Learn more about [branch merging](../guides/branch-merging.mdx).

:::

## Example scenario

The following scenario shows an example of when a catch-up commit is necessary:

1. Joe creates a branch `BranchA` in source control and edits a component `TextInput1`.
2. In a different branch `BranchB`, Emma edits `Chart1`, and the changes are merged to the `main` branch.
3. Joe opens `Chart1` on `BranchA` for the first time.
4. Retool creates a catch-up commit on `BranchA` that contains Emma's changes to `Chart1`.

In step 3 of this example, BranchA does not contain changes to `Chart1` that were made by Emma on `BranchB` and merged to the `main` branch. If the Joe continues to edit without the catch-up commit, he would see the app in an outdated state, and he would encounter merge conflicts when merging `BranchA` with `main`.

In a traditional software development process, this process is handled with a `git rebase`. Retool does not support rebasing, and instead performs a `git cherry-pick` to introduce the changes from `main` with an auto-generated commit.

## Disabling catch-up commits

If you want to ensure that developers in your organization retain complete control of changes within their feature branches, you can disable catch-up commits using the following steps:

:::warning

If you disable catch-up commits, users in your organization must manually, outside of Retool, rebase their branch and resolve conflicts in order to keep their branch up to date.

:::

1. Navigate to **Settings** > **Source Control** and click the **Edit Settings** button on the top right.
2. Toggle on the **Disable auto catch up commits** setting.

## Avoiding merge control issues

To avoid issues with catch-up commits and other merge control mechanisms, refer to Retool's [Merge conflict prevention strategies](../guides/branch-merging.mdx#merge-conflict-prevention-strategies).

---

## Multi-instance development with Source Control

Retool recommends starting app development in an instance representing a staging or development environment; however, an app can be created and protected from any instance of Retool connected to the remote repository (named `retool_source` in the diagram). This is useful if you have a particular resource that's only accessible from your production environment.

Retool also recommends using one branch instead of a "Git Flow” branching model. Since you have the fine-grain control to release each protected app independently at the cadence of your choosing, you can use a single, protected `main` branch that each instance points to.

If batching changes across multiple apps is a requirement, you should manually coordinate releases to synchronize updates after apps have the latest versions. If that is insufficient, you could also point your instances to other integration-like branches such as `staging` and `production` and manually cherry-pick and merge to batch and promote changes.

:::note

By default, releases are exclusive to the Retool instance in which they were created. Creating and publishing releases for an app are not captured in source control, and can be treated as a deploy mechanism. Learn more about [Release Management](../../apps/guides/app-management/releases-history.mdx).

For organizations that need to manage releases across multiple instances, Retool offers [multi-instance releases](../guides/multi-instance-releases.mdx), which enables you to publish different release versions of protected apps and workflows to different instances.

:::

---

## Toolscript

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

Toolscript is a JSX-style markup language built to serialize Retool apps protected using Source Control. Unlike its YAML predecessor, Toolscript is designed to be human reviewable.

Retool recommends you not modify Toolscript files directly and only [make changes](#edit-toolscript) when resolving merge conflicts.

:::info Toolscript availability

Toolscript is available on Retool Cloud and self-hosted Retool versions 3.6.0 and later. Organizations already using Source Control can [migrate YAML files to Toolscript](/changelog/yaml-serialization-deprecation).
Support for YAML will be [removed](/changelog/yaml-serialization-deprecation) in the Q2 2025 [stable release](../../releases/stable/index.mdx) of Self-hosted Retool. Organizations still using YAML serialization of these elements in Source Control must migrate to Toolscript prior to upgrading.

:::

## Syntax

Toolscript uses XML-like syntax to convey positioning and hierarchical information at-a-glance. It uses the `rsx` (Retool Scripting XML) file extension, or "Toolscript" for short.

Toolscript uses nesting to represent parent-child relationships. For example:

- Containers have nested `Header`, `Footer`, `Body` and `View` elements. Child components are nested inside these elements.
- Many components, including `Select` dropdowns and Button groups, have nested `Options`.
- Tables have nested `Columns`.

The ordering of components is based on their position on the canvas: top to bottom, then left to right. When component positions change—for example, a button is moved above another button—their order in the Toolscript file also changes.

## File structure

Toolscript code is split into multiple files for easier comprehension:

- Large groups of components are split into their own files.
- Code blocks are split into own files (e.g., `*.sql`, `*.js`).
- Minor position and size changes are separated from more substantial logical changes.

## Toolscript files

The `apps` directory contains the serialized app contents, split into the following files and directories. Some files and directories, such as `lib`, `src`, and `functions.rsx`, are not always present, depending on the app's structure.

```
app-name
└── .defaults.json
└── .positions
  └── .global_mobile_positions.json
  └── .global_positions.json
  └── .pageName1.positions.json
  └── .pageName2.positions.json
  └── .pageName1.mobilePositions.json
  └── .pageName2.mobilePositions.json
└── lib
  └── sqlQueryName.sql
  └── jsQueryName.js
└── src
  └── container1.rsx
└── functions.rsx
└── main.rsx
└── metadata.json
```

### JSON dotfiles

Toolscript makes use of JSON dotfiles.

- `.defaults.json` contains default component values at time of serialization.
- `.positions` contains one file for each page. Each page contains positioning information for components in the desktop layout.
- `.mobilePositions.json` contains positioning information for components in the mobile layout, if enabled.

Toolscript dotfiles are autogenerated and not designed to be human-readable.

### Core files

- `main.rsx` is the main entry point to the app.
- `functions.rsx` stores functions and queries.
- `metadata.json` contains information about the app version and flags.

### The `lib/` directory

Files in `lib` are extracted from functions and queries. These files contain the contents of the query. The query component has an attribute of the format `query={include("./lib/query1.sql", "string")}`.

### The `src/` directory

Files in `src` are extracted from certain components. They are named after the ID of the extracted component, e.g., `container1.rsx`.

## Edit Toolscript

:::warning

Review these best practices before you directly edit Toolscript files.

:::

Toolscript is a major improvement in readability from YAML, but it is not yet optimized for writing. While manually editing RSX files outside of Retool is possible, it is not currently recommended due to the lack of standard developer tooling, including linting, type-checking, and testing. However, editing library files locally in your IDE and committing the result to your Source Control repository can be a useful way to develop longer scripts and queries.

To provide feedback on your specific use cases for editing Toolscript, contact [support](/support).

### Toolscript elements

The following are recommendations for how to edit specific Toolscript elements.

  

```jsx
// "hello1" is a plugin name

```

Plugin names can be safely edited. However, all references must be renamed as well. This includes the relevant entry in the `.positions.json` file, as well as any other plugins that reference the plugin to be renamed with a Retool expression (e.g., `{{ hello1.value }}` ).

  
  

```jsx
// "Hi there!" is plugin content

```

Plugin content is safe to edit, provided the attribute does not begin with an underscore. Edited content must be the same type as the original type, and it must be well formed. Only simple scalar expressions are allowed (e.g., `{"hi" + "world"}` is not supported).

  
  

```jsx

```

Although it is safe to do so, import statements should not be manually edited because they are currently automatically generated by Retool. As such, any manual changes to imports will be lost the next time the app is exported or updated through Source Control.

Importing files from outside of the app root (e.g., `../other-directory/file`) is not supported.

  
  

```sql
-- lib/query1.sql --
SELECT * from table;
```

You can change the content of library files arbitrarily. If you rename a library file, you must change all import references to it in RSX files as well.

Note that Retool expressions (e.g., `{{ query1.value }}`) are likely not parsed as valid code in library files. Retool expressions prevent the code from being directly executed outside of Retool as well.

  
  

In general, file names correspond to the name of the root plugin or query contained in that file. To rename files, you should rename the corresponding element in the Retool UI to ensure that all dependencies are also renamed.

The name of the `main.rsx`, `functions.rsx`, and JSON metadata files are fixed and cannot be modified.

  

### IDE settings

If you open Toolscript files in an IDE, you can change the language settings for RSX files to React or JSX to enable basic syntax highlighting and auto-formatting.

For example, in VS Code, to change the language setting for RSX, open any RSX file and select **JavaScript (JSX)** on the bottom toolbar.

---

## Source Control concept guides

import Concept from '/docs/_partials/_doctypes/_concept.mdx';

:::tip Get started with self-hosted Retool

Follow a [tutorial](./tutorial/) to set up Source Control with your SCM provider.

:::

---

## Keep branches up to date with branch merging

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

From within the IDE of your app, module, workflow, or query, you can use branch merging to integrate changes from your default branch and resolve merge conflicts. While it's also possible to resolve conflicts through your source control provider's UI or the Git CLI, Retool recommends using the in-product branch merging, as it automatically checks for ToolScript validity.

Branch merging in Retool uses the `git merge` strategy. Read more in the [git documentation](https://git-scm.com/docs/git-merge).

:::note

Enable branch merging by navigating to **Settings** > **Beta**, and toggle on **Enable branch merging**.

:::

## Integrate changes from default branch

Branch merging is enabled when you are developing on a feature branch and have made a commit. When you're ready to update your branch, click the **Merge main into ``** button. Retool handles each of the following three scenarios:

- Your branch is already up to date, and there are no changes to merge.
- Your branch is out of date, and there are no conflicts between the default branch and your feature branch. Click **Commit merge** to update your branch.
- Your branch is out of date, and the changes on the default branch conflict with those on your feature branch. View and resolve the merge conflict from within the IDE. Retool checks that the updated Toolscript is valid. Click **Commit merge** to update your branch. Refer to [Toolscript-specific resolution strategies](#toolscript-specific-resolution-strategies).

export function BranchMerge() {
  return (
    
      
    
  )
}

## Toolscript-specific resolution strategies

:::note

Toolscript is available on Retool Cloud and self-hosted Retool versions 3.6.0 and later. See the [migration guide](/changelog/yaml-serialization-deprecation) to learn how to migrate your apps from YAML to Toolscript.

:::

If your branch has conflicts with the default branch, you may need to edit [Toolscript files](../concepts/toolscript.mdx). It is important to understand how Toolscript files are structured to resolve merge conflicts correctly. Malformed Toolscript files can result in failed Source Control deployments and prevent builders from pushing new commits.

  

Metadata files are located in `app/.metadata.json`. They hold metadata about the application, such as the Retool version the app was edited on and app-level settings.

If the conflict is in the version field, you should keep the later version number.

Most other fields in this file reference app-level settings. If two builders modify the same setting, keep only one valid change in the final commit. If two builders toggle different settings, it is safe to keep both settings.

  

  

Positions files are located in `app/.positions.json`. They encode information about the visual properties for each plugin (e.g., positioning on the canvas).

In this file, each JSON key corresponds to a component's name. As such, all JSON keys must be unique.

### Common causes

There are generally three causes for conflicts in this file.

#### Two builders repositioned the same components

```
>>>>>> main
```

In this case, each block in the conflict has the same JSON key. To maintain the component’s position in your branch, keep your block. Otherwise, keep the block from the main branch.

#### Two builders positioned components at the same location

Both changes modified the same location in `.positions.json`.

```
>>>>>> main
```

In this case, it is safe to keep both blocks.

If both of these causes happen (e.g., two builders repositioned the same components and added different components at the same location), review each conflict area and decide which strategy to apply. The `.positions.json` file is sorted alphabetically by key to help you identify duplicate entries.

  
  

App code is located in `app/main.rsx` and `app/src/*.rsx`. These files contain the Retool components in the app and the app’s overall structure.

### Common causes

There are generally three causes of conflicts in app source code.

#### Two builders edited the same component

In this case, you must only keep the “correct” changes.

#### Two builders added new components at the same location

In this case, it is safe to keep changes from both blocks, but you should pay attention to ordering. See [order of components in app conflicts](#order-of-components-in-app-conflicts).

#### Order of components in app conflicts

When resolving app conflicts, you must carefully consider where your changes should end up in the Toolscript file. The order of components is important, as it helps determine visual placement of the component on the Retool canvas.

In general, your changes can go before, after, or within the other builder’s changes. In the last case, if the other builder introduced an import statement, you might have to place your changes in a different file entirely.

In this example, your changes should go **after** the existing changes.

```jsx
// conflict

=======

>>>>>>> main

// after resolution

```

In this example, your changes should go **within** the existing changes.

```jsx
// conflict

=======

    
    
      
    

>>>>>>> main

// after resolution

    
    
      
      
    

```

In this example, your changes should go within a different file because another builder added an import statement.

```jsx
// conflict - main.rsx

=======

>>>>>>> main

// conflict - container.rsx

    
    
      
    

// after resolution - main.rsx

// after resolution - container.rsx

    
    
      
    

```

  
  

Library code is found in the `lib` directory. It contains user-written executable code, such as JS and SQL files. These are most often used in queries, transformers, and certain code-first components.

Since this code is entirely user-written, there is no Toolscript-specific reasoning or advice for merge conflicts in these files. You may find the section on [merge conflict prevention strategies](#merge-conflict-prevention-strategies) useful.

## Merge conflict prevention strategies

The development processes you use can help prevent merge conflicts from arising in the first place.

### Keep branches and pull-requests short-lived

Trunk-based development best practices recommend branches are used for short-term feature work and kept open for a week or less. The faster a pull request is reviewed and merged into the main branch, the less likely it is for conflicting changes to be merged in during the same period of time.

### Add test deployments to your CI pipeline

[Test deployments](test-deploy.mdx) allow you to programmatically validate a potential change to your Retool repository. You can add a test deployment check to you source control provider (GitHub, etc.) to prevent invalid changes from being merged.

### Use modules to divide work

Retool [modules](../../apps/guides/layout-structure/modules.mdx) are an excellent way to encapsulate a body of work, allowing one individual to develop on an isolated part of an app without affecting others. Before working on a complex app, it can be useful to plan what pieces of the app can be designed as reusable modules.

### Assign different responsibilities to builders

Dividing responsibilities among builders helps ensure that builders make changes in different parts of the codebase. For example, you can assign one builder to work on the executable pieces of the app (e.g., queries and transformers), while another builder works on the visual pieces of the app (e.g., headers, titles, and images). Since the former exclusively affects files in the `lib/` directory while the latter primarily affects files in the `src/` directory, the chances of a conflict are minimal.

---

## Create collaborative branches for app changes

By default, only branch owners can commit changes and merge pull requests on their branches. With collaborative branches, multiple users can commit changes and submit pull requests to protected apps or workflows.

## Create collaborative branches

Create your branch, then select **Allow collaboration** to allow other users in your organization to push changes to it.

## Share branches with collaborators

When you're on a collaborative branch, links copied from the **Share** modal include the branch name. Users sent these links can then push changes to your branch. All users can commit changes and push changes to collaborative branches, but only admins can rename, reset, disallow collaboration, or delete these branches.

## Remove collaboration

From the branch menu in the lower left of the IDE, you can disallow further collaboration.

---

## Manage Source Control branches

Using Source Control, you can:

- Edit multiple apps, modules, workflows, and queries on the same branch.
- Commit changes directly in the editor.
- Switch between branches and apps using the **Branch** menu.

## Multi-element branching

With _multi-element branching_, you can edit apps, modules, workflows, and Query Library queries on a single branch. Each commit contains changes for a single app, module, workflow, or query, but branches can contain commits across apps, modules, and queries. This allows you to combine your dependent changes into a single pull request, instead of creating and merging pull requests for each dependent module or query.

Branches must all be created from `main`.

## Branch menu

From the branch menu in the bottom left status bar, you can:

- View your commit log.
- Switch branches.
- Edit other apps and modules on the same branch.
- Create a pull request.
- Rename, reset, and delete your branch.

## Build apps on and off branches

:::note

When you open a new tab, you are always on the `main` branch.

:::

When you create a new app or module while you're on a local branch, you can create it on or off your branch.

When you select **Create on current branch**, the app or module is created locally on your branch and considered **In development**. Apps and modules in development are only visible to their creators on the local branch until they are merged into source control repositories. After you merge an **In development** app or module to your source control repository, it is visible to the rest of your organization. In development apps and modules are useful when you want to privately develop and iterate on apps before you share them widely.

When **Create on current branch** is not selected, the new app or module is not protected.

## Rebase branches

[Rebasing](https://git-scm.com/book/en/v2/Git-Branching-Rebasing) allows you to integrate changes from one branch to another. It's often used to update a development or feature branch with the latest changes from a more stable branch (e.g.,`main`).

If you need to update your Source Control branch, you can perform a rebase on a branch that you own using the following steps.

1. Ensure all changes are committed on the branch.
2. Rebase the branch within your SCM provider and resolve any merge conflicts.
3. In Retool's app editor, open the branch menu in the lower left and select **•••** > **Reset branch** to update the branch.

:::note

Source Control branches can only be rebased by the branch owner.

:::

## Automatically delete branches

Retool automatically deletes branches after changes are merged if the remote branch no longer exists and there are no uncommitted changes. It can take up to one hour for branches to be automatically deleted. This feature is enabled by default, but you can switch it off in `Settings > Source Control > Edit Settings`.

Otherwise, users can delete branches that they own directly from the branch menu in the app.

## Multi-instance

To preview a branch of an app pushed from another instance, create a new branch in your current instance, and select **Reset branch** in the status bar. Select the relevant branches in the modal to pull in the origin branch changes.

:::caution Do not add files manually

Do not manually add any ToolScript (`*.tsx`), YAML, JSON, or query (`*.sql`, `*.js*`) files to the Source Control repository. All changes to apps, workflows, queries, and resources must be done through the Retool interface. Manually adding these files can cause deploy failures.

Any other files (e.g., `README.md`) are skipped during deploys can be safely added to the repository.

:::

---

## Manage Source Control deployments

:::note Requirements

The Deployment Dashboard is available on [Self-hosted Retool](../../self-hosted/index.mdx) v2.97 or later. Source Control with GitHub and GitLab, including the Deployment Dashboard, is also available on Retool Cloud.

:::

Organizations that use [Source Control](../index.mdx)—a branch-based workflow for managing and deploying changes to Retool apps—can use the Deployment Dashboard to view the status and details of deployments, download deployment logs for further information, and manually deploy the latest changes.

## View Source Control deployment logs

The Deployment Dashboard contains the following details about the most recent deployments (up to 100) for your self-hosted instance.

| Detail       | Description                                                                               |
| ------------ | ----------------------------------------------------------------------------------------- |
| Status       | Whether the deployment was successful or failed.                                          |
| Commit       | Specific commit SHA on the main branch of the Source Control repository that is deployed. |
| Triggered by | Whether the deployment was triggered automatically by Retool or manually by a user.       |
| Started at   | Date and time that the deployment started.                                                |
| Duration     | Total time for the deployment to run.                                                     |
| Logs         | Downloadable log file for the deployment.                                                 |

## Source Control status

The Deployment Dashboard also includes details about your instance's Source Control configuration. You can review the current configuration, test the connection to the source control management (SCM) provider, or edit the configuration.

## Full and Partial deployments

The **Deploy** section displays the commit SHA that is currently deployed.

Admin users can manually trigger **Full deployments** by clicking **Deploy latest**. This syncs the latest state of all protected elements in your remote repository's main branch to Retool.

Retool uses the jobs runner to poll for changes in your remote repository's main branch every 10 seconds. If any changes are detected, Retool automatically kicks off a **Partial deployment** to sync updated elements to Retool.

---

## Migrate from YAML to Toolscript

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

:::warning

This migration is only necessary if your organization initially set up Source Control on self-hosted Retool 3.4.19 or earlier. You must complete this migration before upgrading your legacy deployment to a newer release.

No migration is required if you initially configured Source Control on self-hosted Retool 3.6.0 or later.

:::

To use ToolScript, you must perform a one-time migration of your existing YAML repositories. The entire migration takes approximately 15 minutes to complete, but may take longer for organizations with many protected apps and branches. It does not change the behavior of your apps and queries.

This migration is unidirectional and you cannot migrate from Toolscript back to YAML.

:::tip

If you have a multi-instance deployment, ensure you follow the steps to [migrate multiple instances](#multi-instance-procedure).

:::

## Migration overview

Retool attempts to migrate your main branch and recent feature branches to Toolscript from YAML. During this operation, all source control operations are locked in Retool.

On the main branch, all apps, modules, and resources are migrated. Retool also attempts to migrate all feature branches touched within the last two months, up to a maximum of 300 total. For recent feature branches, only apps and resources changed by the feature branch are migrated. Retool also attempts to delete all recent open branches which rename, move, or unprotect source control elements. These branches are not possible to migrate, but you can recreate them after the migration.

If migration is not possible for all branches, Retool does not push any remote changes, and instead prompts you to fix or delete certain branches. This failure is rare, and usually occurs if a branch has been manually edited outside of Retool and contains an invalid structure or YAML format.

The automatic migration does not currently support migrating workflows. Commits to protected workflows are still in YAML.

### Test runs

You cannot currently specify a single test branch to migrate using the Retool UI. When the migration begins, all remote branches are converted from YAML to Toolscript.

To dry run the migration, you must copy your entire Git repository to a new one, then configure your Source Control configuration on your Retool instance to use the new repository.

### App behavior

The Toolscript migration itself does not change how Retool stores apps, and therefore does not affect the behavior of apps. Similarly, the migration does not affect app releases, history, or queries.

Commits made by builders in Toolscript are functionally equivalent to YAML, with the exception that Toolscript omits default values to produce more readable diffs.
These default values may change across different versions of Retool, but should not lead to significant user experience changes.

## Requirements

To run the migration you must:

- Be on version **3.8.10** or later. See the [errors on earlier versions](#errors-on-versions-pre-3810) section if you're on a version between 3.6.0 and 3.8.10.
- Ensure that branch protection is **disabled** on your SCM provider. Retool requires access to push Toolscript conversion commits directly to your main branch. See [GitHub](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/managing-a-branch-protection-rule#deleting-a-branch-protection-rule) and [GitLab](https://docs.gitlab.com/ee/user/project/protected_branches.html#allow-force-push-on-a-protected-branch) documentation to learn how to toggle these settings. You may re-enable branch protection after the migration is complete.
- Notify all Retool builders in your organization that source control operations are blocked during the migration.

If you run multiple instances, you must ensure your instances all use the same main branch to sync code.

## Run migration

To run the migration, go to **Settings > Source Control** and select **Get Toolscript**.

If the migration runs successfully, no additional input is necessary.
You can confirm a migration was successful by navigating to the main branch on your SCM provider. You should see a new commit entitled "Convert to ToolScript v2.0.0" that replaces `*.yml` files with `*.rsx` and other Toolscript files.
Additionally, the `.retool/protected-apps.yml` metadata file should be updated to read `2.0.0`.

If the migration returns an error, you can download logs for the given run from the Source Control settings page.

## Migration errors

Migration logs can help you identify errors which occurred during your migration. To resolve these errors, use one of the following options:

1. Revert or manually fix the changes introducing errors. Many issues are likely malformed YAML. See [troubleshooting](./troubleshooting.mdx#changes-and-environments-no-longer-in-sync) for more guidance.
2. Delete the remote and Retool branch. You can use the following command to delete the remote branch.

```
git push origin --delete yourfeaturebranch
```

To preserve your work, you can [export and import apps or modules](../../apps/guides/app-management/import-export.mdx) using JSON into a new branch after your migration.

When you've fixed or deleted the erroring branches, rerun the migration by selecting **Get Toolscript** on the Source Control settings page.

### Errors on versions pre-3.8.10

If you are migrating to Toolscript on a version earlier than 3.8.10, you may encounter additional migration errors in the following scenarios:

- **Empty branch migration**: If your migration failure logs contain the line `yourfeaturebranch has no changes when compared to yourmainbranch`, the migration has failed because one or more of your feature branches contain no changes to migrate. Upgrade to 3.8.10+ first, or delete these branches using the Git CLI.
- **SCM branch protection**: You may receive a false success message from Retool without a Toolscript conversion commit on your main branch if you have branch protection enabled on your SCM provider. To resolve this situation, click **Deploy Latest** on the Source Control settings page. This immediately unblocks builders and allows commits to be made in YAML. Next, turn off branch protection and upgrade to version 3.8.10+. You can then re-run the migration from the Source Control settings page.

Upgrading to 3.8.10+ is highly recommended to avoid these scenarios.

## Multi-instance procedure

If you deploy multiple instances of Retool connected by the same source control repository, you must follow special procedures to avoid downtime. The steps below assume you have two instances: development ("dev") and production ("prod"). If you do not have a prod instance, designate the instance that is more sensitive to downtime as the prod instance.

The recommended upgrade procedure depends on whether or not your instances sync from the same main Git branch, or use different branches:

:::warning
Each source control branch can contain Toolscript files or YAML files, but not both. As a result, all Retool instances connected by the same branch must be upgraded to support Toolscript at the same time.
:::

1. Confirm that all instances are already on version **3.8.10+** or later. You can choose to follow an incremental upgrade procedure, such as upgrading and verifying the dev instance first. The Toolscript migration only runs when manually triggered.
2. Choose a time during which no builders are using source control to begin the migration. The safest method to do this is to take all instances offline completely. If downtime is unacceptable, an announcement may suffice. Builders lose any commits or branches created during the migration.
3. Log in to the **dev** instance and run the migration from the Source Control settings page. If there are any errors, address them before moving on; your source control repository is not yet affected.
4. To confirm the migration, look for a successful deployment status in the Source Control settings page and a new conversion commit in your source control repository. The `.retool/protected-apps.yaml` file in your repository should also read `2.0.0`. If this is not the case, stop immediately and contact [Retool Support](/support).
5. After the **dev** migration succeeds, log in to the **prod** instance. Since the prod instance syncs from the same branch as dev, you do not need to perform the migration again. Click **Deploy latest** and confirm that the deployment status shows as healthy. The instance should automatically recognize the new Toolscript code.

1. Confirm that all instances are already on version **3.10.2+** or later. Support for different syncing branches is not available in versions 3.10.2 and earlier.
2. Choose a time during which no builders are using source control on the dev instance to begin the migration. The safest method to do this is to take the dev instance offline completely. If downtime is unacceptable, an announcement may suffice. Builders lose any commits or branches created during the migration.
3. Log in to the **dev** instance and click “Get Toolscript” on the Source Control settings page. In the advanced settings section, check the “Exclude certain branches” option and enter in the branches used by your other instances for syncing. For example, if your “prod” instance syncs from the “production” Git branch, add “production” to the exclusion list. Repeat this process for all instances connected by the same repository.
4. Start the migration. If there are any errors, address them before moving on; your source control repository is not yet affected. To confirm the migration, look for a successful deployment status in the Source Control settings page as well as a new conversion commit in your source control repository. The `.retool/protected-apps.yaml` file in your repository should also read `2.0.0`. If this is not the case, stop immediately and contact [Retool Support](/support).
5. All apps on your dev instance should now be in Toolscript, while your production instance still uses a YAML branch. You can test the Toolscript changes on your dev instance before repeating the above steps for your production instance. Before you migrate your production instance, you cannot promote apps from dev to production. If you need to transfer apps, you can export them as JSON from the dev instance and manually import them into your production instance.

## Migrate YAML manually

Though rare, some directories may remain in YAML after the Toolscript migration has completed. For example:

- A developer merged in a YAML feature branch that was not migrated because it exceeded the 2-month and 300 branch limit.
- A developer merged in a YAML feature branch while the Toolscript migration was running.

In these cases, your deployments may fail because there are both Toolscript and YAML format files in the repository. To resolve this situation:

1. Ensure that the `.retool/protected-apps.yaml` file on the main branch of your source control repository reads `2.0.0`. If not, the Toolscript migration has not ran successfully yet.
2. As a precaution, back up your app by [exporting it to JSON](../../apps/guides/app-management/import-export.mdx).
3. Delete the app folder containing YAML from your source control repository and push the change directly to your main branch.
4. Your app should now show as unprotected on your Retool instance. Protect the app through Retool again to generate the correct Toolscript and commit the change to your source control repository.

A similar procedure can be used to manually migrate an unmerged YAML pull request. These steps can only be executed by the pull request author.

1. Ensure that the `.retool/protected-apps.yaml` file on the main branch of your source control repository reads `2.0.0`. If not, the Toolscript migration has not ran successfully yet.
2. Go to the app changed in the pull request and [export it to JSON](../../apps/guides/app-management/import-export.mdx).
3. Start a new branch on the same app and import the JSON obtained in step 1.
4. Create a pull request on this new branch. The format should be Toolscript. You can now close the YAML pull request.

---

## Multi-instance releases

Organizations can manage the releases of [protected apps](./protect/apps.mdx) and [protected workflows](./protect/workflows.mdx) across multiple deployment instances. You can specify which release version of an app or workflow is available on each deployment that's configured with [Source Control](../index.mdx). This is particularly useful if you want to test a newer version on a staging or development instance before promoting to production.

You can also programmatically manage multi-instance releases and manifests using the [Retool API](../../api/index.mdx). Refer to the **Source Control** section of the Retool API reference for complete API documentation.

## Overview

Multi-instance releases makes use of release artifacts and manifests to manage app and workflow releases across instances.

- Each app or workflow release has its own artifact. This represents a snapshot of the element at the time of release. These are stored in the `dist/apps//.zip` or `dist/workflows//.zip` directories of the Source Control repository.
- Each instance has its own release manifest. Each manifest file contains a list of element UUIDs (apps and workflows) and the release version that is made available for the instance. These are stored in the `manifests/` directory of the Source Control repository.

```text title="Example repository structure"
.
├─ .retool/
│   └─ protected-apps.yaml
├─ apps/
│   ├─ app1
|   |   ├─ 0.1.0.zip
|   |   ├─ 0.2.0.zip
|   |   └─ ...
│   ├─ app2
│   └─ ...
├─ workflows/
│   ├─ workflow1
│   └─ ...
├─ dist/
│   ├─ apps/
│   │   ├─ app1-uuid
│   │   ├─ app2-uuid
│   │   └─ ...
│   └─ workflows/
│       ├─ workflow1-uuid
│       └─ ...
└─ manifests/
    ├─ dev.yaml
    └─ prod.yaml
```

As with protected apps and workflows, you do not directly modify files within the repository. You create and manage multi-instance releases using the Retool UI or, optionally, using the Retool API.

## Demo

Watch the following videos that demonstrate how to create and manage manifests and releases.

## Requirements

To use multi-instance releases:

- You must be using Source Control to protect apps and workflows.
- For Retool Cloud, multi-instance releases are available on all plans with Source Control enabled.
- For self-hosted Retool, each instance must be running version 3.252 or later.
- The Source Control configuration must be the same across all instances, such as source control provider, repository, and branch.

:::note
Multi-instance releases with workflows is enabled by default. To opt out, disable **Multi-instance releases: workflows support** in your organization's **Settings > Beta** page. 
:::

## 1. Create a manifest

Each instance needs its own manifest file stored in the `manifests/` directory of the Source Control repository. The manifest determines the version of apps and workflows that are published on the instance. You create and manage manifests in your organization's **Source Control** settings.

To create a manifest:

1. Navigate to your organization's **Settings > Source Control** page.
1. Click **Create manifest** in the **Releases** pane.
1. Enter a name to use for the manifest.
1. Click **Create branch with release manifest**. Retool then creates a branch and commits the new manifest to it.
1. Click **Open pull request** to start a new pull request.

export function CreateManifest() {
  return (
    
      
    
  )
}

## 2. Select the manifest for an instance

A manifest determines which app and workflow releases to publish on an instance. You assign each manifest to each instance. To configure an instance to use a specific manifest:

1. Sign in to a specific instance (e.g., `dev`).
1. Navigate to **Settings > Source Control**.
1. Click **Edit config** in the **Releases** pane on the right.
1. Select the manifest to associate with the instance (e.g., `dev`).

Once complete, the instance will then publish the release versions associated with the manifest. For example:

| Instance | Manifest |
| --- | --- |
| Development | `dev` |
| User testing | `uat` |
| Production | `prod` |

In the example above, apps and workflows defined in the `dev` manifest are published accordingly to the **Development** instance.

## 3. Create a release

To create a new release for an app, open the app in **View** mode—you cannot create a release whilst editing an app. For workflows, see [Version and publish workflows](../../workflows/guides/version-and-publish.mdx).

1. Click the **Releases** button in the status bar. Note that the button displays the release version currently open. If there have not been any previous releases, the app uses `latest` (current working version).
1. Click **Create new** to create a version for release. Retool uses [Semantic Versioning](https://semver.org/) and automatically increments the version number based on your selection of **Major**, **Minor**, or **Patch** versioning.
1. Click **Create release PR** to create a branch and pull request for the release.
1. Confirm that you want to create a branch and release artifact for the new release. Check that the version and description information is correct, then click **Create branch with release artifact**.
1. Click **Open pull request** to begin opening a PR with your source control provider.

export function CreateRelease() {
  return (
    
      
    
  )
}

The new release is only available once the pull request has merged and its changes deployed. Click the **Releases** button in the status bar once this has been completed to view app releases to see the new release available.

## Manage releases

There are two ways with which you can manage releases:

- View and manage the deployment of all app and workflow releases in the **Releases** tab of the **Source Control** settings page.
- View and manage the deployment of individual app or workflow releases in the **Releases and history** modal.

### View and manage all apps and workflows

You can manage multi-instance app and workflow releases and manifests from the **Source Control** settings page. The **Releases** tab is a graphical interface for managing all release manifests. It contains a list of all protected apps and workflows, and the release version deployed to each instance. Each column corresponds to each instance that uses multi-instance releases.

View and manage all apps and workflows.

To change the deployed release version of an app or workflow for any instance:

1. Click on the version number in the column and select the release to use. You can make multiple changes, if required.
1. Click **Review** to review and confirm the changes.
1. Click **Create pull request** to start a new pull request.

### View and manage an individual app or workflow and its releases

Similar to creating a new release, you can use the **Releases and history** modal to manage the release deployment across instances for both apps and workflows. To deploy a release:

1. Click **•••** to open the contextual menu for the release you want to deploy.
1. Select the instance in the **Publish release** sub-menu. All instances with a manifest are automatically listed.

### Patch a release with a hotfix

:::caution

Only one hotfix branch can be created for each `major.minor` release version at a time.

:::

You can create a _hotfix_ to perform a patch update of an app release. This process creates a collaborative hotfix branch based on the selected release for users to make changes. A hotfix enables you to patch any `major.minor` release version without needing to unprotect it first.

Once you complete a hotfix, a patch release is created and made available.

To create a hotfix branch:

1. Click the **Releases** button in the status bar.
1. Find the release to patch.
1. Click **•••** to open the contextual menu and select **Create hotfix**.

After you complete the necessary changes on the hotfix branch, create a new release:

1. Open the  **Releases and History** pane in the left toolbar.
1. Provide a description of the changes made on the hotfix branch and click **Create patch release PR**.
1. Confirm that you want to create a release artifact for the patch release.

export function PatchRelease() {
  return (
    
      
    
  )
}

---

## Protect agents with Source Control

:::note
To enable Source Control for Agents in version 3.284.0 or 3.284.1, reach out to your account manager. 

In version 3.284.2, Source Control for Agents is available by default, and admins can disable it in **Settings > Beta** by toggling the `AI Agents Source Control` feature flag. 
:::

Source Control for [agents](../../../agents/index.mdx) operates similarly to how it does for [apps](./apps.mdx), but with some key differences. 

- **Agents cannot be moved or renamed**. Protected agents and the folders in which they're located cannot be renamed or moved. You must unprotect agents before making name or location changes. Protected agents must have a unique name.
- **Agent triggers cannot be protected**. You can edit the trigger of a protected agent without creating a commit.
- **Evals and Datasets are incompatible**. You cannot access [evals](../../../agents/quickstart.mdx#evals) and [datasets](../../../agents/quickstart.mdx#datasets-and-test-cases) from a protected agent. 

This guide outlines how to make changes to a protected Retool agent and merge these changes into the `main` branch.

## Requirements

Before you begin, ensure [Source Control is configured](../../index.mdx) for your Retool instance. This guide uses GitHub, but Retool supports [multiple remote SCM providers](../../index.mdx#setup-quickstarts). You must be an admin to configure Source Control.

## 1. Protect the agent

In Retool, _protected_ agents are checked into remote source control repositories. You can protect an agent with Source Control from the  dropdown next to the agent name, or from the  dropdown on the [All agents](../../../agents/quickstart.mdx#all-agents) page. 

After reviewing the confirmation modal, click **Protect agent** and create a pull request. Once merged, your agent is protected.

export function ProtectAgents() {
  return (
    
      
    
  )
}

Your agent shows a badge to the right of the name indicating that it is protected. The protected badge will also show next to the agent name on the [All agents](../../../agents/quickstart#all-agents) page. 

:::note

An agent's name serves as a unique identifier when you use Source Control, so you can't rename a protected agent. To change the name of a protected agent, you must [unprotect](#4-remove-agent-protection) the agent, rename it, and re-protect it.

:::

## 2. Create a branch and commit your changes
You can still chat with and view the logs of a protected agent, but to edit an agent's [configuration](../../../agents/quickstart#configuration), you'll need to create a new branch. 

1. Click **Edit**. 
2. Select **Create new branch**. 
3. Give the branch a descriptive name for your changes. You can also select **Allow collaborators** to [allow other users](../collaborative-branches.mdx) to push changes to your branch.
4. Click **Create**. The branch on which you're editing appears in the top right, along with the commit log for the branch. Changes made on branches are local to the branch.
5. Make any necessary changes to your agent. 
6. Click the **Commit** button in the top right corner.
7. Add a commit message to describe the changes you made to the agent. 
8. Optionally, select **Review changes** to preview the changes in your commit.
9. Push your changes to GitHub by clicking **Commit to \**.

export function Commit() {
  return (
    
      
    
  )
}

## 3. Open a pull request and merge

After you commit all the changes you want to make, incorporate your changes back into the `main` branch.

1. Click branch name in the top right corner, and select **Create pull request**.
2. Notice that the description is already populated with a link to preview your changes on your branch. Make any necessary changes, and create the pull request.
3. Merge the changes into the `main` branch of your repository. The merge button label may differ depending on your repository's configuration for merging (e.g., **Rebase and merge** or **Squash and merge**). Confirm the merge if required by your organization. 
4. After a few seconds, your changes are merged into the `main` branch of the agent.

export function Pull() {
  return (
    
      
    
  )
}

## 4. Remove agent protection

You can remove protection for an agent at any time from the `main` branch. 

1. Ensure you are on the `main` branch. 
2. Click the  dropdown next to the agent name to open the contextual menu. 
3. Select **Remove protection** and confirm that you want to remove protection again. 
4. Click **Open a pull request** in GitHub.
5. Create and merge the pull request into `main` to unprotect your agent. 

export function Remove() {
  return (
    
      
    
  )
}

---

## Protect apps with Source Control

This guide outlines the end-to-end workflow required to make changes to a protected Retool app and merge these changes into the `main` branch.

## Requirements

Before you begin, ensure [Source Control is configured](../../index.mdx) for your Retool instance. This guide uses GitHub, but Retool supports [multiple remote SCM providers](../../index.mdx#setup-quickstarts). You must be an admin to configure Source Control.

## 1. Import the app

If you don't have existing Retool apps to edit, you can import the one used in this guide. First, download the [protected-applications-tutorial.json](https://gist.githubusercontent.com/kyle-retool/870978d09d16b6c83cb762fe0c6a866f/raw/a1de7c06d320ee4bd6ee50617fa98ea755d186e4/protected-applications-tutorial.json) file. Once it's saved, import it into Retool.

## 2. Protect the app

In Retool, _protected_ apps are apps checked into remote source control repositories. To protect your app, go to the app IDE and select **...** > **Protect app**.

Click **Protect application** on the confirmation modal. Follow the flow to open and merge the initial PR of the app's source code into the connected GitHub repository.

Your application now shows a badge and **Protected** below its name.

To rename or move an app, go to the home page and select **Rename** from the action menu. You must commit any renaming changes to your SCM provider before they take effect locally.

## 3. Create a branch and make changes

Click **Edit** and select **Create new branch**. Give the branch a descriptive name for your changes. You can also select **Make collaborative** to [allow other users](../collaborative-branches.mdx) to push changes to it.

After you create the branch, you're directed to the app IDE. The branch on which you're editing appears in the lower left, along with the commit log for the branch. Changes made on branches are local to the branch.

Edit your app. In this example, you can update the table's search filter to include searching by email. Update the `tableFilteredData` transformer to the following query.

```javascript
return formatDataAsArray({{getRows.data}}).filter((row)=> row.name.toLowerCase().includes({{nameAndEmailFilter.value}}) || row.email.toLowerCase().includes({{nameAndEmailFilter.value}}) )
```

Rename the placeholder text of the `nameFilter` to **Search by name and email**, and rename `nameFilter` to `nameAndEmailFilter`.

:::note

Images for protected apps are only stored in the database of the instance in which the app was developed. An image uploaded to the database for one app cannot be accessed by downstream environments.

To ensure that an image appears in all instances of your app, you can:

- Host the image on a public URL that is accessible from all apps, and use the **URL** option for the image source.
- Upload the image to the downstream database environments.

:::

## 4. Commit your changes

After you finish making changes, you need to commit them and push them to GitHub.

1. Click the **Commit** button in the top right corner.
2. Optionally, select **Review changes** to preview the changes in your commit.
3. Write a description of your change.
4. Push your changes to GitHub by clicking **Commit to \**.

## 5. Open a pull request

After you commit your changes, open a pull request to get your changes reviewed. Click branch name in the lower left corner, and select **Create pull request**.

Review any dependencies your app has on other protected objects, such as resources or modules. Click **Open pull request** to visit the pull request on GitHub.

Notice that the description is already populated with a link to preview your changes on your branch.

## 6. Merge changes

After you commit your changes, merge them into the main branch of your repository. The merge button label may differ depending on your repository's configuration for merging (e.g., **Rebase and merge** or **Squash and merge**).

After a few seconds, your changes appear in the main branch of the Retool application.

## Remove application protection

You can remove protection for an application at any time. In the application menu on the Apps page, select **Remove protection**, then confirm changes in the dialog box that appears.

---

## Protect queries with Source Control

:::note

Protected queries is available on Retool Cloud and self-hosted Retool version 3.6 or later.

:::

Using protected queries in Source Control, you can safely share queries in Query Library across multiple instances of Retool, similar to protected resources.

## Prerequisites

You must set up your [Source Control](../../index.mdx) provider to start using protected queries.

You also need to be able to protect underlying resources if you want to share queries that depend on resources across instances. See [protected resources](resources.mdx) for more.

## Protect queries

To protect a query and start tracking its changes, you need to merge an initial commit for the query to your remote repository.

### 1. Start protecting a query

You can start protecting your resource by selecting **Protect query** from the dropdown menu.

When you click **Protect query**, Retool creates a new branch on your SCM provider.

> Note that you will see a warning to protect the underlying resource if it is not protected yet. Protected queries will only work across different instances if their resources are also protected and shared.

### 2. Open a new pull request

After you click **Open pull request**, you’ll be redirected to your SCM provider to create an initial commit for the resource. This commit contains a ToolScript file with relevant details for your query.

### 3. Merge pull request

Query is not protected until your repository contains the files from your initial commit. After you merge this commit, the query syncs to all other instances.

## Edit protected queries

Use the following steps to make changes to protected queries.

### 1. Select a branch to edit

When you edit a protected query, you must select a branch to edit. You can either continue on the previously selected branch or create a new branch from the menu.

### 2. Commit your changes

Make your changes to the query and commit your changes by clicking on the **Commit** button on the top right corner. Note that you have to save your changes first before you can commit. If you have unsaved changes, the commit button will be disabled.

### 3. Open and merge a pull request

Once you've committed your changes, open a pull request to get your changes reviewed. Click branch name in the lower left corner, and select **Create pull request**.
You'll be redirected to your SCM provider to open a pull request. Once it's reviewed, you can merge the changes into the main branch of your repository.

After a few seconds, your changes appear in the main branch of the Retool application.

## Unprotect queries

You can unprotect a query by selecting **Unprotect query** from the dropdown menu.

A query must be unprotected before you can delete it.

---

## Protect resources with Source Control

Using protected resources in Source Control, you can safely replicate resource configurations across multiple instances of Retool, similar to protected applications. You can use protected resources with any of Retool's supported source control management (SCM) providers:

- [GitHub](../../tutorials/github.mdx)
- [GitLab](../../tutorials/gitlab.mdx)
- [AWS CodeCommit](../../tutorials/aws-codecommit.mdx)
- [Bitbucket](../../tutorials/bitbucket.mdx)
- [Azure Repos](../../tutorials/azure-repos.mdx)

On Retool Cloud and self-hosted Retool versions 3.6.0 and later, Retool uses ToolScript to serialize resources. On earlier versions of Retool, protected resource configurations are represented as YAML files.

## Prerequisites

Before you configure protected resources, you must set up your [Source Control](../../index.mdx) provider.

If your resources depend on secret values, set the secrets using one of the following:

- Secret [configuration variables](../../../org-users/guides/configuration/config-vars.mdx).
- Secrets manager integration, such as [AWS Secrets Manager](../../../self-hosted/guides/secrets/aws.mdx) or [HashiCorp Vault](../../../self-hosted/guides/secrets/hashicorp-vault.mdx).
- Environment variables with the [`RETOOL_EXPOSED`](../../../self-hosted/reference/environment-variables/index.mdx#retool_exposed_name) prefix.

### Resources in multiple instances

To differentiate resources across instances, set the same variable names with different values per environment. For example, given a resource with different resource URLs for development and prod—e.g., the development URL `dev-api.your-company.com` and prod `prod-api.your-company.com`—you can use the following setup. You'd define URLs in this example in config vars or secrets manager settings, or wherever you store environment variables.

| Type                                   | Example setting             | Example usage in resource config         |
| -------------------------------------- | --------------------------- | ---------------------------------------- |
| `RETOOL_EXPOSED` environment variables | `RETOOL_EXPOSED_YOUR_COMPANY_API_URL` | `%RETOOL_EXPOSED_YOUR_COMPANY_API_URL%`            |
| Config var                             | `your_company_api_url`                | `{{ environment.variables.your_company_api_url }}` |
| Value from secrets manager             | `your_company_api_url`                | `{{ secrets.your_company_api_url }}`               |

The same resource can reference `RETOOL_EXPOSED_YOUR_COMPANY_API_URL`, `{{ environment.variables.your_company_api_url }}`, and `{{ secrets.your_company_api_url }}`, and different URL values are injected for each instance.

## Protect resources

To protect a resource and start tracking its changes, you need to merge an initial commit for the resource to your remote repository.

### 1. Create a branch

You can start protecting your resource by selecting **Protect resource** from either:

- The **Resources** overview page (`/resources`).
- The individual **Edit resource** page.

When you click **Protect Resource**, Retool creates a new branch on your SCM provider.

### 2. Open a new pull request

After you click **Open pull request**, you’ll be redirected to your SCM provider to create an initial commit for the resource. This commit contains a YAML file with configuration details for your resource.

### 3. Merge pull request

Resources are not protected until your repository contains the resource YAML file from your initial commit. After you merge this commit, the resource syncs to all other instances.

## Edit protected resources

When you edit a protected resource, the **Resources** overview and the **Edit resource** pages display a warning that your resource has unmerged changes. This warning displays until you merge your changes to your remote repository.

To create a pull request, click the **Open PR** button next to the resource. Follow your source control workflow to merge the pull request. After it’s merged, you’ll see the resource in all your instances, with the resource updated as necessary.

To rename a resource, from the Resources overview page, select **Rename** next to the resource. You must commit any renaming changes to your SCM provider before they take effect locally.

## Unprotect resources

You can unprotect a resource from either:

- The **Resources** overview page (`/resources`).
- The individual **Edit resource** page.

A resource must be unprotected before you can delete it.

## Protected resources with multiple environments

Only the changes on the _production_ environment of a protected resource are tracked in source control, even if you use multiple environments. This is because multi-instance Retool deployments typically only use a single environment in their downstream (non-development) instances. If your setup uses another configuration, contact Retool Support.

---

## Protect themes with Source Control

Using protected themes in Source Control, you can safely replicate [organization-level theme configurations](../../../apps/guides/presentation-styling/themes.mdx) across multiple instances of Retool. You can use protected themes with any of Retool's supported source control management (SCM) providers:

- [GitHub](../../tutorials/github.mdx)
- [GitLab](../../tutorials/gitlab.mdx)
- [AWS CodeCommit](../../tutorials/aws-codecommit.mdx)
- [Bitbucket](../../tutorials/bitbucket.mdx)
- [Azure Repos](../../tutorials/azure-repos.mdx)

Protected themes work similarly to protected apps or resources, with a few key differences. Keep in mind the following considerations:

- _Themes cannot be deleted or renamed._ Protected themes cannot be renamed or deleted. You must unprotect themes before making deletions or name changes.
- _Changes are branchless._ Unlike with Retool apps, you do not create a branch from which to make changes. Instead, you create a pull request from the theme IDE after you make changes.

## Prerequisites

Before you configure protected themes, you must set up your [Source Control](../../index.mdx) provider.

## Protect themes

To protect a theme and start tracking its changes, you need to merge an initial commit for the theme to your remote repository. Complete the following steps.

1. Navigate to an organization-level theme and select **Protect theme** from the menu `•••`.
2. Confirm by clicking the **Protect theme** button. At this point Retool creates a new branch on your SCM provider.
3. Click the **Open pull request** button. You'll be redirected to your SCM provider to create an initial commit for the theme. This commit contains a JSON file with configuration details for your theme. Themes are not protected until your repository contains the JSON file from your initial commit.
4. Create the pull request and merge the commit. After you merge this commit, the theme syncs to all other instances.

The following example shows this process using GitHub as the SCM.

## Edit protected themes

When you edit a protected theme, you must click **Save** and click the **Open PR** button next to the theme to commit your changes. Follow your source control workflow to merge the pull request. After it’s merged, you’ll see the theme in all your instances, with the theme updated as necessary.

:::note

While protected themes are managed using source control, you do not create a separate branch on which to make changes.

:::

## Unprotect themes

Use the same process to unprotect a theme as you did to [protect it](#protect-themes).

A theme must be unprotected before you can delete it or rename it.

---

## Protect workflows with Source Control

This guide outlines how to use Source Control to protect workflows.

:::note

Protected workflows and the folders in which they're located cannot be renamed or moved. You must unprotect workflows before making name or location changes. Protected workflows must have a unique name.

:::

## Requirements

Before you begin, ensure [Source Control is configured](../../index.mdx) for your Retool instance. This guide uses GitHub, but Retool supports [multiple remote SCM providers](../../index.mdx#setup-quickstarts). You must be an admin to configure Source Control.

## 1. Protect the workflow

In Retool, _protected_ workflows are workflows checked into remote source control repositories. You can protect a workflow with Source Control in either the **Workflows** tab of your organization or the workflow IDE:

- **Workflows tab**: Click `•••` to open the contextual menu for the workflow you want to protect, then select **Protect workflow**.
- **Workflow IDE**: Click `•••` in the toolbar to open the contextual menu, then select **Protect workflow**.

After reviewing the confirmation modal, click **Protect workflow** and create a pull request. Once merged, your workflow is protected.

export function ProtectWorkflow() {
  return (
    
      
    
  )
}

Your workflow now shows a badge to the right of the name indicating that it is protected.

:::note

An workflow's name serves as a unique identifier when you use Source Control, so you can't rename a protected workflow. To change the name of a protected workflow, you must [unprotect](#remove-workflow-protection) the workflow, rename it, and re-protect it.

:::

## 3. Create a branch and make changes

Click **Edit** and select **Create new branch**. Give the branch a descriptive name for your changes. You can also select **Make collaborative** to [allow other users](../collaborative-branches.mdx) to push changes to it.

After you create the branch, you're directed to the workflow IDE. The branch on which you're editing appears in the lower left, along with the commit log for the branch. Changes made on branches are local to the branch.

Edit your workflow.

export function CreateBranch() {
  return (
    
      
    
  )
}

## 4. Commit your changes

After you finish making changes, you need to commit them and push them to GitHub.

1. Click the **Commit** button in the top right corner.
2. Write a description of your change.
3. Optionally, select **Review changes** to preview the changes in your commit.
4. Push your changes to GitHub by clicking **Commit to \**.

export function CommitChanges() {
  return (
    
      
    
  )
}

## 5. Open a pull request and merge

After you commit all the changes you want to make, incorporate your changes back into the `main` branch.

1. Click branch name in the lower left corner, and select **Create pull request**.
2. Review any dependencies your workflow has on other protected objects, such as resources. Click **Open pull request** to visit the pull request on GitHub.
3. Notice that the description is already populated with a link to preview your changes on your branch. Make any necessary changes, and create the pull request.
4. Merge the changes into the `main` branch of your repository. The merge button label may differ depending on your repository's configuration for merging (e.g., **Rebase and merge** or **Squash and merge**).
5. After a few seconds, your changes are merged into the `main` branch of the workflow. [Publish a release](../../../workflows/guides/version-and-publish.mdx) for your changes to be reflected in the workflow's live release.

export function MergeChanges() {
  return (
    
      
    
  )
}

## Multi-instance releases

If your organization manages multiple deployment instances (such as development, staging, and production), you can use [multi-instance releases](../multi-instance-releases.mdx) to control which version of a protected workflow is deployed to each instance.

With multi-instance releases, you can:

- Test new workflow versions on development or staging instances before promoting to production.
- Deploy different versions of workflows to different instances.
- Manage workflow releases across instances using release manifests.

:::note
Multi-instance releases with workflows is enabled by default. To opt out, disable **Multi-instance releases: workflows support** in your organization's **Settings > Beta** page. Learn more about [multi-instance releases](../multi-instance-releases.mdx).
:::

## Remove workflow protection

You can remove protection for a workflow at any time in either the **Workflows** tab of your organization or the workflow IDE:

- **Workflows tab**: Click `•••` to open the contextual menu for the workflow you want to protect, then select **Remove protection**.
- **Workflow IDE**: Click `•••` in the toolbar to open the contextual menu, then select **Unprotect workflow**. You can also choose to **Unprotect triggers** if you want to remove protection from your workflow triggers.

Confirm changes in the dialog box that appears.

---

## Test Source Control deployments

Test deployments serve as a method to confirm that a pull request can be successfully deployed to Retool. They perform a "dry run" of your changes, ensuring that the modifications can be applied to your Retool instance without encountering errors.

The following video shows an overview of how the test deployment process works:

In this guide, we’ll refer to a placeholder Retool organization at `https://exampleorg.retool.com`. Be sure to substitute this placeholder with the actual URL where your Retool instance is hosted. Additionally, replace `RETOOL_TEST_DEPLOY_API_KEY` with your access token.

## Create an access token

Complete the following steps to create an access token with permissions for test deployment:

1. Navigate to **Settings** > **Retool API** and click **Create new**.
2. Enter a name and description for your new access token.
3. Select the `Read` and `Test Deploy` scopes from the **Source Control** section.
4. Click **Create token** and save the API key.

## Using the Test Deploy endpoint

Use the [Test source control changes](../../api/test-source-control-changes.api.mdx) endpoint in the Retool API to check whether a commit can be successfully deployed. Pass the full commit SHA to the endpoint:

```bash
curl -X POST -H "Content-Type: application/json" \
  -H "Authorization: Bearer " \
  -d '{"deploy_params": {"commit_sha": ""}}' \
  http://exampleorg.retool.com/api/v2/source_control/test_deploy
```

## Add a CI check

You can also add a check that runs the test deploy as part of your continuous integration process.

This section includes an example that uses GitHub as the source control provider and Buildkite as the CI/CD provider, although the steps outlined here can be applied to any Git-based hosting service.

### Using Github Actions

Follow [GitHub's guide on storing information in variables](https://docs.github.com/en/actions/learn-github-actions/variables) to configure the `RETOOL_TEST_DEPLOY_API_KEY` environment variable on GitHub. The following code snippet shows an example of a workflow file

```yaml
on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  deploy:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Test Deploy Commit
        run: |
          COMMIT_SHA="${{ github.event.pull_request.head.sha }}"
          echo "Test deploying commit $COMMIT_SHA on https://exampleorg.retool.com/"
          response=$(curl -X POST -H "Content-Type: application/json" -H "Authorization: Bearer ${{ secrets.RETOOL_TEST_DEPLOY_API_KEY }}" -d "{\"deploy_params\": {\"commit_sha\": \"$COMMIT_SHA\"}}" https://exampleorg.retool.com/api/v2/source_control/test_deploy)
          echo "API Response:"
          echo "$response"
          if [[ $(echo $response | jq '.data.success') != "true" ]]; then
            echo "Test deployment failed"
            exit 1
          fi
```

### Using Buildkite

Follow Buildkite's [Managing pipeline secrets guide](https://buildkite.com/docs/pipelines/secrets) to set the `RETOOL_TEST_DEPLOY_API_KEY` environment variable on Buildkite. Add the following command to your `pipeline.yml` file.

```yaml title="pipeline.yml"
steps:
  - command: scripts/run_test_deploy.sh
```

Buildkite [recommends](https://buildkite.com/docs/pipelines/secrets#anti-pattern-referencing-secrets-in-your-pipeline-yaml) that you avoid directly referencing secrets in your `pipeline.yml` file. Instead, utilize a script stored in your repository for this purpose:

```bash title="scripts/run_test_deploy.sh"
#!/bin/bash
set -euo pipefail

COMMIT_SHA=$BUILDKITE_COMMIT
echo "Deploying commit SHA $COMMIT_SHA on https://exampleorg.retool.com/settings/api/v2/source_control/test_deploy"
response=$(curl -X POST -H "Content-Type: application/json" -H "Authorization: Bearer $RETOOL_TEST_DEPLOY_API_KEY" -d "{\"deploy_params\": {\"commit_sha\": \"$COMMIT_SHA\"}}" https://exampleorg.retool.com/settings/api/v2/source_control/test_deploy)
echo "API Response:"
echo "$response"
if [[ $(echo $response | jq '.data.success') != "true" ]]; then
  echo "Test deployment failed"
  exit 1
fi
```

---

## Troubleshoot Source Control issues

With Source Control, your Retool instance syncs and deploys changes made to the designated main branch of your Git repository. As an administrator, you can diagnose and troubleshoot most Source Control issues.

## Identify Source Control issues

Issues that arise with Source Control should be resolved quickly to minimize disruption. There are a number of methods available for identifying symptoms as you investigate the root cause.

### Deployment Dashboard

Use [Deployment Dashboard](./manage-deployment.mdx#source-control-status) to check the status of deployments and the SCM connection. Each deployment has either a successful or failed state. Failed deployments can indicate a potential issue with Source Control, such as malformed repository contents or an inconsistent Git history.

Kicking off a [full deployment](./manage-deployment.mdx#full-and-partial-deployments) by clicking **Deploy latest** updates all protected elements in your Retool instance to the latest state of your main branch. This overrides and can help fix failing intermediary partial deployments.

The SCM details area displays the current connection status. If the status is not **Connected**, this indicates that your Retool instance is not able to connect to your SCM provider. To resolve the connection, verify your SCM configuration by clicking **Edit config**, environment variables, and if using GitHub, your Base64-encoding of your private key.

### Changes and environments no longer in sync

There may be issues with Source Control if there is a mismatch between the current state of your Retool instance and the main branch of your Git repository. This can be apparent if protected apps, modules, or resources fail to update. When these issues occur, they can prevent newer changes from being deployed, resulting in the changes being overwritten. New objects cannot be protected return an error or indicate an `unmerged` status.

Similarly, if changes from one Retool instance are not in sync with others (e.g., `dev` to `prod`) then there may be an issue somewhere in the Source Control workflow.

### jobs-runner instability

If Source Control cannot successfully deploy changes, it may cause the [jobs-runner](../../self-hosted/concepts/architecture.mdx) container to continuously crash or restart. This container performs background tasks, such as Source Control syncing, and database migrations.

You can also monitor the status of the `jobs-runner` container using its health check endpoint: `http://[container_ip]:3003/api/checkJobsRunnerHealth`.

## Resolve common Source Control issues

For Source Control to operate:

- The Git repository and its contents must have a stable history and valid file contents.
- Both your Retool instance and your source control management (SCM) provider must be configured correctly.
- Retool must be able to connect to your SCM provider.

Any problems or disruptions that affect these dependencies can impact the operation of Source Control.

### Git repository and content

Retool instances periodically poll your Git repository for new commits and attempt to apply changes. If the integrity of the Git repository is affected, such as invalid or malformed data, your Retool instance may fail to deploy successfully.

#### Malformed data

Source Control breaks up applications into multiple YAML files to simplify code review and prevent merge conflicts. Invalid changes to YAML files can prevent successful deploys. Possible YAML issues include:

- **Invalid formatting**. Manually adding commits, or editing commits automatically generated by Retool, can introduce formatting errors (e.g., different whitespace or tab settings).

If there appears to be malformed data or invalid files, you need to identify which commits contain the malformed files.

First, identify the commit with malformed files. You can do this by either:

- Using the Deployment Dashboard to identify the commit of the first failing build.
- Reviewing recent commits in the Git history with the SCM provider. This may be the most recent commit if you're able to respond to the issue quickly.
- Reviewing the `jobs-runner` container logscontainer logs for lines with the terms `Target sha`, `SOURCE_CONTROL_DEPLOYMENT`, and `error`.

Next, revert or fix-forward the offending commit. You can revert the commit and push the reversion to your Git repository to roll back the change. To preserve the changes, you can fix-forward by correcting the error as a new commit.

Finally, run a full deployment. You can perform this using the Deployment Dashboard by clicking **Deploy latest**. Retool instances using Self-hosted Retool earlier than v2.97 must restart the `jobs-runner` container manually.

#### Inconsistent Git history

Retool requires a linear, append-only Git history, and tracks the latest commit SHA at all times. If the Git history is rewritten manually (e.g., `git push --force`, `git rebase -i`, or `git reset`), the commit currently tracked by Retool may no longer exist. Retool may attempt to open an invalid commit with the SCM provider, which will return an error.

If the Git history has been rewritten manually, one way to resolve this is to:

1. Export any apps as [JSON](../../apps/guides/app-management/import-export.mdx#export-apps) that cannot be pushed using the Retool UI.
2. Create a new branch in the Retool UI and import the exported JSON.
3. Retry the commit and push operations.
4. Verify that the generated pull request contains all relevant changes.

#### Unexpected files

The file system structure in the Git repository must match the expected directory structure of your Retool instance. Any unknown files added to the repository, or moving existing files to different locations unexpectedly, can cause deploys to fail. Non-YAML files that may be autogenerated by operating systems, such as `.DS_Store`, can also impact the repository's integrity.

If unexpected files have been added to the repository, remove them and commit the changes.

### Instance configuration

Retool uses [environment variables](../../self-hosted/reference/environment-variables/index.mdx) to configure deployments on self-hosted Retool. Any misconfigured environment variables can cause issues with your Retool instance.

#### SCM and repository connection

The Deployment Dashboard reflects the connection status to the SCM provider. If this reports a failed connection, check that the connection details are correct.

Source Control requires the full URL for some SCM providers. If you are using one of these SCM providers, make sure to include the full `http/https` URL.

#### Nested repositories

Some SCM providers support nested repositories (e.g., GitLab). If the path is incorrect, the Deployment Dashboard indicates a successful connection but Source Control is not syncing from the correct repository path. If you are using nested repositories, check the repository details use the full path.

### SCM configuration

Retool requires read and write permissions for branches, commits, and pull requests. Incorrect permissions can prevent Retool from creating new protected objects or editing existing ones. Refer to the [Source Control](../index.mdx) documentation for your SCM provider to learn what permissions are required and how to configure them.

#### Infrastructure instability

Infrastructure failures or instability can be caused by a misconfigured environment. There can be many reasons for this, but common root causes include:

- **Minimum resource requirements not met**. Each Retool instance requires 4+ CPU cores and 4GB+ of memory. As your organization grows, the number of commits or concurrent users increases, which requires more resources. If your Retool instances lack resources, the `jobs-runner` container performance degrades and can become unresponsive. When Retool detects that the `jobs-runner` container is not responsive, it can automatically restart the container in some deployment modes (e.g., Retool Kubernetes Helm chart).
- **Network firewall restrictions**. Retool requires an open connection directly to your SCM provider. If your firewall blocks these connections, Git operations may become unresponsive or time out. You can use the Deployment Dashboard to check the connection to your SCM provider by clicking **Test connection**.

### OAuth callback URL

If you see issues with the OAuth callback URL for protected resources, make sure:

1. The `BASE_DOMAIN` environment variable is persisted on both the `api` and `jobs-runner` containers.
2. Both the `api` and `jobs-runner` containers were rebooted.
3. The source control repo is redeployed through the `/settings/sourcecontrol`.

### Image display

Images for protected apps are only stored in the database of the instance in which the app was developed. An image uploaded to the database for one app cannot be accessed by downstream environments.

If an image does not appear where you expect it to in your protected app, you can:

- Host the image on a public URL that is accessible from all apps, and use the **URL** option for the image source.
- Upload the image to the downstream database environments.

---

## Source Control how-to guides

import Howto from '/docs/_partials/_doctypes/_howto.mdx';

:::tip Get started with Source Control

If you want to get started with Source Control at Retool:

- Read the [quickstart](./quickstart) to learn about the fundamental concepts.
- Follow a [tutorial](./tutorial/) to set up Source Control with an SCM provider.

:::

---

## Source Control documentation

Source Control allows organizations to manage changes to apps, workflows, resources, and themes using pull requests on remote source control management (SCM) providers, such as GitHub, GitLab, AWS CodeCommit, Bitbucket, and Azure Repos. Instead of making changes directly to an app, changes are made on a separate branch.

---

## Source Control quickstart

This guide serves as an introduction to Source Control. It covers many of the concepts and terminology you would come across when using version control to manage apps, workflows, and more. After reading this page, you should have a good understanding of the fundamentals for using Source Control.

## Introduction

Source Control enables organizations to manage changes using remote source control management (SCM) providers, such as GitHub, GitLab, AWS CodeCommit, Bitbucket, and Azure Repos. Using this distributed approach, your users can:

- Protect changes to apps, workflows, queries, resources, and themes.
- Work collaboratively on apps.
- Review changes using [Toolscript](concepts/toolscript.mdx)—a readable, JSX-like markup language.
- Sync changes across [multiple instances](concepts/multi-instance-development.mdx).

## Version control and protection

Source Control provides a distributed approach to managing your organization using a remote repository. Users can check in (protect) apps, workflows, queries, resources, and themes. Once protected, all changes are made on branches. Users can create multiple branches and work collaboratively on changes.

All of this occurs within Retool. Reviewers then use the source control provider (e.g., GitHub) to review the pull request. Once merged, Retool deploys the changes automatically.

## Monitor deployment status

The Deployment Dashboard provides details about recent deployments triggered by Source Control. It includes details about your instance's Source Control configuration.

As with any version control system, you can roll back changes by reverting to an earlier commit.

## Collaborate on app changes

[Collaborative branches](guides/collaborative-branches.mdx) enables multiple users to make app changes on the same branch.

## Get started

Set up Source Control using a supported SCM provider.

import DocCardList from "@theme/DocCardList";

---

## Source Control environment variables

import SourceControl from "@site/docs/self-hosted/reference/environment-variables/source-control.mdx";

---

## Source Control glossary

## A

## B

## C

## D

## E

## F

## G

## H

## I

## J

## K

## L

## M

## N

## O

## P

## Q

## R

## S

## T

## U

## V

## W

## X

## Y

## Z

---

## Organization and users reference

import Reference from '/docs/_partials/_doctypes/_reference.mdx';

:::tip Get started with your organization and users

If you are unfamiliar with managing your organization and want to get started, read the [quickstart](./quickstart) to learn about the fundamental concepts.

:::

---

## Configure Source Control with AWS CodeCommit

## Setup instructions

### Disable git syncing

If you've enabled [Git Syncing](../index.mdx), disable it:

- In your `docker.env` file, set `DISABLE_GIT_SYNCING=true` and `VERSION_CONTROL_LOCKED=false`.
- In the **Settings** > **Advanced** tab in Retool, remove the repository URL and branch name from your Git Syncing configuration.

### 1. Create a new IAM Role in your AWS account

Enter a name for the user and check the **Access key - Programmatic access** setting.

Next, select **Attach existing policies directly**, search for "codecommit", and select the **AWSCodeCommitFullAccess** policy name. This is the only permission required for this new user.

Save the **Access key ID** and **Secret access key** in a secure location locally. You'll use these as environment variables in step 3.

Go to the newly created user on your IAM console. Select the **Security credentials** tab, scroll to **HTTPS Git Credentials for AWS CodeCommit**, and click **Generate Credentials**. Download and save these credentials in a secure location. You'll use these HTTPS credentials as environment variables in step 3.

### 2. Create a new CodeCommit repository

Go to CodeCommit on your AWS console and create a new repository for Retool syncing. You can select any region of your preference.

Add a `README.md` file to this repository. The repository needs to contain at least one file to sync.

### 3. Configure AWS CodeCommit repository settings

Go to the [Source Control settings](https://login.retool.com/auth/login?source=docs&redirectOnLogin=settings/source-control), and select **Set up AWS CodeCommit**. Enter the following settings.

| **Setting**               | **Description**                                    | **Example**           |
| ------------------------- | -------------------------------------------------- | --------------------- |
| AWS CodeCommit Repository | The name of the CodeCommit repository.             | retool-apps           |
| AWS CodeCommit Branch     | The default branch for your CodeCommit repository. | main                  |
| AWS Region                | The region of the CodeCommit repository.           | us-east-1             |
| Access Key Id             | The **Access key ID** you generated in step 1.     | AKIAWS3BACWHP6QW6VB2  |
| Secret Access Key         | The **Secret access key** you generated in step 1. | loDJlwRetoolTYXOFbO   |
| HTTPS Username            | The HTTPS username you generated in step 1.        | retool-https-username |
| HTTPS Password            | The HTTPS password you generated in step 1.        | retool-https-password |

Set the following [environment variables](../../self-hosted/reference/environment-variables/index.mdx#source-control) on your Retool instance on the `api` and `jobs-runner` containers.

| **Variable name**                 | **Description**                                    | **Example value**     |
| --------------------------------- | -------------------------------------------------- | --------------------- |
| CODE_COMMIT_AWS_ACCESS_KEY_ID     | The **Access key ID** you generated in step 1.     | AKIAWS3BACWHP6QW6VB2  |
| CODE_COMMIT_AWS_SECRET_ACCESS_KEY | The **Secret access key** you generated in step 1. | loDJlwRetoolTYXOFbO   |
| CODE_COMMIT_AWS_DEFAULT_REGION    | The region of the CodeCommit repository.           | us-east-1             |
| CODE_COMMIT_REPOSITORY_NAME       | The name of the CodeCommit repository.             | retool-apps           |
| CODE_COMMIT_MAIN_BRANCH           | The default branch for your CodeCommit repository. | main                  |
| CODE_COMMIT_HTTPS_USERNAME        | The HTTPS username you generated in step 1.        | retool-https-username |
| CODE_COMMIT_HTTPS_PASSWORD        | The HTTPS password you generated in step 1.        | retool-https-password |

If you use your own [SSL certificates](../../self-hosted/guides/certificates.mdx), set the `SSL_CERT_FILE` and `NODE_EXTRA_CA_CERTS` environment variables on the `jobs-runner` and `api` containers to the path to your SSL certificate.

### 4. Verify your settings

After you set up your environment variables, visit the **Settings** > **Source Control** on your Retool instance.

If your environment is correctly configured, the page will show a **Deployment Dashboard**. Click **Test connection** under the **AWS CodeCommit** section to test your connection and confirm the sync works as expected.

If you don't see the Deployment Dashboard and your AWS CodeCommit commits, go back to step 3 and confirm your environment variables are correctly set.

You are now ready to use source control with AWS CodeCommit. Read the [source control getting started guide](../index.mdx) to learn more about source control workflows.

---

## Configure Source Control with Azure Repos

With Source Control with [Azure Repos](https://azure.microsoft.com/en-us/products/devops/repos), you can use pull requests on [Azure Repos](https://azure.microsoft.com/en-us/products/devops/repos#get-started) to manage changes to your Retool applications.

## Setup instructions

### Disable git syncing

If you've enabled [Git Syncing](../index.mdx), disable it:

- In your `docker.env` file, set `DISABLE_GIT_SYNCING=true` and `VERSION_CONTROL_LOCKED=false`.
- In the **Settings** > **Advanced** tab in Retool, remove the repository URL and branch name from your Git Syncing configuration.

### Prerequisites

This guide requires access to an Azure DevOps account with permissions to create repos and access tokens.

### 1. Create a new repository in Azure Repos

1. [Create or use an existing project under Azure Repos](https://learn.microsoft.com/en-us/azure/devops/organizations/projects/create-project?view=azure-devops&tabs=browser). Ensure its name does not contain any spaces.
2. [Create a new Git repository under Azure Repos](https://learn.microsoft.com/en-us/azure/devops/repos/git/create-new-repo?view=azure-devops) for your Retool apps. Ensure that it has a `README.md` file at the root level directory.

### 2. Create a personal access token

1. [Create an Azure Repos personal access token](https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=Windows).
2. Under the **Code** section, check **Full** and **Status**. With this token, Retool will be able to programmatically manipulate your repository, generate pull requests, and see their statuses. Accordingly, use the most appropriate admin in your organization to generate it.

### 3. Configure Azure repository settings

Go to the [Source Control settings](https://login.retool.com/auth/login?source=docs&redirectOnLogin=settings/source-control), and select **Set up Azure Repos**. Enter the following settings.

| **Setting**           | **Description**                                                                                                                                                            | **Example**                                                |
| --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------- |
| Azure Repository      | The repo name from step 1. Ensure this name does not contain any spaces.                                                                                                   | retool-apps                                                |
| Azure Organization    | The Azure DevOps organization.                                                                                                                                             | acme-co                                                    |
| Azure Branch          | Retool automatically tracks changes to this branch.                                                                                                                        | main                                                       |
| Azure Project         | The new or existing Azure DevOps project from step 1. Ensure its name does not contain any spaces.                                                                         | proj-retool                                                |
| Azure User            | The Azure Repos username.                                                                                                                                                  | retool                                                     |
| Azure URL             | Your base Azure URL. For Azure Cloud, this is always [http://dev.azure.com](http://dev.azure.com/). For Azure self-managed, this is the URL where your instance is hosted. | [https://azure.mycompany.com](https://azure.mycompany.com) |
| Personal Access Token | The Azure [project access tokens](https://docs.searchunify.com/Content/Content-Sources/Azure-Generate-Personal-Access-Token.htm) to authenticate to the Azure API.         | AKIAWS3BACWHP6QW6VB2                                       |
| Use Basic Auth        | Set this to true if you are using self-hosted Azure Repos.                                                                                                                 | true                                                       |

| **Variable**                        | **Description**                                                                                    | **Example**                                          |
| ----------------------------------- | -------------------------------------------------------------------------------------------------- | ---------------------------------------------------- |
| `AZURE_REPOS_USER`                  | The Azure Repos username                                                                           | retool                                               |
| `AZURE_REPOS_PERSONAL_ACCESS_TOKEN` | The personal access token generated previously                                                     | mpoqd2zy7jklzfbhmuzev46vbbcpkeeqminb4wcvwigsrldasdfa |
| `AZURE_REPOS_ORGANIZATION`          | The Azure DevOps organization                                                                      | acme-co                                              |
| `AZURE_REPOS_PROJECT`               | The new or existing Azure DevOps project from step 1. Ensure its name does not contain any spaces. | proj-retool                                          |
| `AZURE_REPOS_REPO`                  | The repo name from step 1. Ensure its name does not contain any spaces.                            | retool                                               |
| `AZURE_REPOS_MAIN_BRANCH`           | Retool automatically tracks changes to this branch                                                 | main                                                 |

If you use your own [SSL certificates](../../self-hosted/guides/certificates.mdx), set the `SSL_CERT_FILE` and `NODE_EXTRA_CA_CERTS` environment variables on the `jobs-runner` and `api` containers to the path to your SSL certificate.

### 4. Test your configuration

After you set up your environment variables, if you’re an admin on your Retool instance, you can visit **Settings** > **Source Control**.

If your Azure Repos environment is correctly configured, the page will show your Azure Repos commits in the **Deployment Dashboard** section. You can also click the **Test connection** button, which sends a request to test the connection.

If you don't see the **Deployment Dashboard** and your Azure Repos commits on **Settings** > **Source Control**, check that your environment variables are correctly configured.

## What's next?

You are now ready to use source control. Learn how to create branches and pull requests, merge changes, and use other source control features in the [getting started guide](../index.mdx).

---

## Configure Source Control with Bitbucket

You can use Source Control with [Bitbucket Cloud](https://bitbucket.org) to manage changes with pull requests.

:::note
Source Control does not currently support self-hosted [Bitbucket Data Center](https://www.atlassian.com/software/bitbucket/enterprise/data-center) instances.
:::

## Setup instructions

- In your `docker.env` file, set `DISABLE_GIT_SYNCING=true` and `VERSION_CONTROL_LOCKED=false`.
- In the **Settings** > **Advanced** tab in Retool, remove the repository URL and branch name from your Git Syncing configuration.

### Prerequisites

This guide requires access to a Bitbucket account with permissions to create workspaces and repositories.

### 1. Create a new Bitbucket workspace and repository

1. [Create a Bitbucket workspace](https://support.atlassian.com/bitbucket-cloud/docs/create-your-workspace/) for your Retool repository.
2. [Create a Git repository in Bitbucket](https://support.atlassian.com/bitbucket-cloud/docs/create-a-git-repository/) for your Retool apps.

### 2. Create an access token for the repository

1. [Create a Bitbucket access token for the repository](https://support.atlassian.com/bitbucket-cloud/docs/create-a-repository-access-token/).
2. Under **Scopes** > **Repositories**, select **Read**, **Write**, **Admin**, and **Delete**. This gives the access token the ability to access and modify the repository containing your Retool apps.

### 3. Configure Bitbucket repository settings

Go to the [Source Control settings](https://login.retool.com/auth/login?source=docs&redirectOnLogin=settings/source-control), and select **Set up Bitbucket**. Enter the following settings.

| **Setting**                  | **Description**                                                                                          | **Example**                     |
| ---------------------------- | -------------------------------------------------------------------------------------------------------- | ------------------------------- |
| Bitbucket Workspace          | The name of the workspace you created for the Retool repository. This variable is case-sensitive.        | `retool-workspace`              |
| Bitbucket Repository         | The repo name in step 2.                                                                                 | `retool-apps`                   |
| Bitbucket Branch             | Retool automatically tracks changes to this branch                                                       | `main`                          |
| Bitbucket URL                | The domain used to access your self-hosted Bitbucket instance.                                           | `https://bitbucket.org/`        |
| Bitbucket Enterprise API URL | The REST API route for your self-hosted Bitbucket instance. Defaults to `https://api.bitbucket.org/2.0`. | `https://api.bitbucket.org/2.0` |
| Bitbucket Access Token       | The access token you created.                                                                            | `loDJlwRetoolTYXOFbO`           |

Set the following [environment variables](../../self-hosted/reference/environment-variables/index.mdx#source-control) on your Retool instance on the `api` and `jobs-runner` containers.

| **Variable**            | **Description**                                                                           | **Example**         |
| ----------------------- | ----------------------------------------------------------------------------------------- | ------------------- |
| `BITBUCKET_TOKEN`       | The access token you generated in [step 2](#2-create-an-access-token-for-the-repository). | loDJlwRetoolTYXOFbO |
| `BITBUCKET_WORKSPACE`   | The workspace name in step 2. This variable is case-sensitive.                            | retool-space        |
| `BITBUCKET_REPO`        | The repo name in step 2                                                                   | retool-apps         |
| `BITBUCKET_MAIN_BRANCH` | Retool automatically tracks changes to this branch                                        | main                |

If you use your own [SSL certificates](../../self-hosted/guides/certificates.mdx), set the `SSL_CERT_FILE` and `NODE_EXTRA_CA_CERTS` environment variables on the `jobs-runner` and `api` containers to the path to your SSL certificate.

### 4. Test your configuration

After you set up your environment variables, if you’re an admin on your Retool instance, you can visit **Settings** > **Source Control**.

If your Bitbucket environment is correctly configured, the page will show your Bitbucket commits in the **Deployment Dashboard** section. You can also click the **Test connection** button, which sends a request to test the connection.

If you don't see the **Deployment Dashboard** and your Bitbucket commits on **Settings** > **Source Control**, check that your environment variables are correctly configured.

## What's next?

You are now ready to use source control. Learn how to create branches, pull requests, merging changes, and use other source control features in the [getting started guide](../index.mdx).

---

## Configure Source Control with GitHub

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

You can use Source Control with GitHub to manage changes using pull requests. You create a repository for Source Control to use, then configure a GitHub App for your Retool instance that commits, pushes, and pulls changes.

## Requirements

To configure Source Control with GitHub, you must have:

- Administrator permissions in your Retool organization.
- Permissions to [create an app](https://github.com/settings/apps/new) in GitHub.

## 1. Create a Git repository

:::warning

Use a new, dedicated Git repository. Source Control does not work with an existing [Git Syncing](../index.mdx) repository.

:::

To begin, [create a new repository](https://docs.github.com/en/free-pro-team@latest/github/getting-started-with-github/create-a-repo) for Source Control. Ensure it contains a **README.md** file and a `main` branch. Source Control uses this repository to store and track all changes.

If you have [multiple Retool instances](../concepts/multi-instance-development.mdx), use this same repository across all instances.

## 2. Create and install a GitHub App

[Create a GitHub App](https://github.com/settings/apps/new) with the following settings. Ensure this app is created by the same owner as the Git repository. Refer to [GitHub's guide](https://docs.github.com/en/developers/apps/creating-a-github-app) for more information.

If you have [multiple Retool instances](../concepts/multi-instance-development.mdx), use this same GitHub App across all instances.

| Setting                                                    | Value                                                       |
| ---------------------------------------------------------- | ----------------------------------------------------------- |
| **GitHub App name**                                        | The name of your GitHub App.                                |
| **Homepage URL**                                           | The URL of your GitHub App. This field is unused by Retool. |
| **Expire user authorization tokens**                       | Unchecked                                                   |
| **Request user authorization (OAuth) during installation** | Unchecked                                                   |
| **Webhook** > **Active**                                   | Unchecked                                                   |
| **Permissions** > **Contents**                             | Read and write                                              |
| **Permissions** > **Pull requests**                        | Read and write                                              |
| **Where can this GitHub App be installed?**                | Only on this account                                        |

### Install the GitHub App

After you create your GitHub App, it should appear in the [GitHub Apps section of your Developer Settings](https://github.com/settings/apps).

To install your GitHub App:

1. Click **Edit** next to your app to see its settings.
2. Click on **Install App**.
3. Click the **Install** button next to your organization.
4. Choose the **Only select repositories** option and select the repository created in the first section.
5. You’ll be redirected to the installation page. The number at the end of the URL is the installation ID. Save this for later.

```shell
https://github.com/settings/installations/:installation_id
```

### Save the app owner and app ID

On the same page, click on **App settings** underneath the GitHub App name. Save the **Owned by** and **App ID** fields for later.

## 3. Generate a private key

1. Navigate to the **Private Keys** section and click **Generate a private key**.
2. Download and save the `.pem` key. The filename should use the format `{github-app-name}.YYYY-MM-DD.private-key.pem`.
3. Base64-encode the contents of the file to use when configuring your instance. If you use Kubernetes Secrets, you must base64-encode this value twice:

```shell title="Base64-encode twice for Kubernetes Secrets"
base64 -i path/to/{github-app-name}.YYYY-MM-DD.private-key.pem > github-private-key.txt
```

## 4. Configure GitHub repository settings

Next, configure the GitHub repository in Retool.

Navigate to the [Source Control settings](https://login.retool.com/auth/login?source=docs&redirectOnLogin=settings/source-control) in Retool and select **Set up GitHub**.

Enter the settings using the details you saved when creating your GitHub app.

| Setting                   | Description                                                                                                                                                                                                                                             |
| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| GitHub owner              | The username or organization who owns the GitHub App, without the `@` prefix, e.g., `tryretool`.                                                                                                                                                        |
| GitHub repository         | The name of the repository you created to use with Retool.                                                                                                                                                                                              |
| GitHub branch             | The default branch, e.g., `main`.                                                                                                                                                                                                                       |
| GitHub Enterprise URL     | The domain used to access your self-hosted GitHub instance. If you use GitHub Cloud, you can leave this blank.                                                                                                                                          |
| GitHub Enterprise API URL | The [REST API route](https://docs.github.com/en/enterprise-server@2.21/rest/reference/enterprise-admin#endpoint-urls) for your self-hosted GitHub instance. Defaults to `https://[hostname]/api/v3`. If you use GitHub Cloud, you can leave this blank. |
| App ID                    | The [GitHub App ID](./github.mdx#save-the-app-owner-and-app-id).                                                                                                                                                                |
| Installation ID           | The [GitHub installation ID](./github.mdx#install-the-github-app). This can be found at the end of the installation URL.                                                                                                        |
| App Private Key           | The base64-encoded private key.                                                                                                                                                                                                                         |

:::note

You can choose to use a GitHub Personal Access Token for authentication instead of the GitHub app. To do so, navigate to **Settings** > **Beta**, and toggle on the **GitHub Personal Access Token** toggle. Return to the configuration page, and set **Config type** to **Personal Access Token**.

:::

Click **Test connection** to confirm your app is connected, then click **Save and deploy**.

Configure the following [Retool environment variables](../../self-hosted/reference/environment-variables/index.mdx) on the `api` and `jobs-runner` containers and then restart your containers.

| Environment variable         | Description                                                                                                  |
| ---------------------------- | ------------------------------------------------------------------------------------------------------------ |
| `GITHUB_APP_ID`              | The [GitHub App ID](#save-the-app-owner-and-app-id).                                                         |
| `GITHUB_APP_INSTALLATION_ID` | The [GitHub installation ID](#install-the-github-app). This can be found at the end of the installation URL. |
| `GITHUB_APP_PRIVATE_KEY`     | The base64-encoded private key.                                                                              |

If you use your own [SSL certificates](../../self-hosted/guides/certificates.mdx), set the `SSL_CERT_FILE` and `NODE_EXTRA_CA_CERTS` environment variables on the `jobs-runner` and `api` containers to the path to your SSL certificate.

Deployments using self-hosted Retool v2.91 or above can optionally prevent the instance from pushing changes to GitHub. To lock version control, set the [`VERSION_CONTROL_LOCKED` environment variable](../../self-hosted/reference/environment-variables/source-control.mdx#version_control_locked) to `true`. This is useful for creating a "read-only" production instance of Retool.

For v2.97 and later, navigate to the [Source Control settings](https://login.retool.com/auth/login?source=docs&redirectOnLogin=settings/source-control) and select **Set up GitHub**.

For v2.96 and earlier, navigate to the **Advanced** settings and select **Protected Applications**.

Enter the settings using the details you saved when creating your GitHub app.

| Setting                   | Required | Description                                                                                                                                                                                                                                             |
| ------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| GitHub owner              | Yes      | The username or organization who owns the GitHub App, without the `@` prefix, e.g., `tryretool`.                                                                                                                                                        |
| GitHub repository         | Yes      | The name of the repository you created to use with Retool.                                                                                                                                                                                              |
| GitHub branch             | Yes      | The default branch, e.g., `main`.                                                                                                                                                                                                                       |
| GitHub Enterprise URL     | No       | The domain used to access your self-hosted GitHub instance. If you use GitHub Cloud, you can leave this blank.                                                                                                                                          |
| GitHub Enterprise API URL | No       | The [REST API route](https://docs.github.com/en/enterprise-server@2.21/rest/reference/enterprise-admin#endpoint-urls) for your self-hosted GitHub instance. Defaults to `https://[hostname]/api/v3`. If you use GitHub Cloud, you can leave this blank. |

Click **Test connection** to confirm your app is connected, then click **Save and deploy**.

## Troubleshooting

The following errors are issues common to Source Control with GitHub. Refer to the guide to [resolve common Source Control issues](../guides/troubleshooting.mdx) to learn about other common errors.

### Internal Server Error

If the system time on the machine that Retool is running on is out of sync or has "drifted," Retool can have trouble authenticating with the GitHub API.

If you see `500 Internal Server Error` when making a commit on a branch, or see errors such as `'Issued at' claim ('iat') must be an Integer representing the time that the assertion was issued` in Retool API server logs, check that the system time where the Retool image is running is accurate. Restarting the system can solve this issue.

### Cookies must be enabled to use GitHub

This error occurs when the **GitHub Enterprise API URL** provided is not an API route. Check the URL provided and confirm that it is a GitHub API route.

### Unable to access GitHub

If your Retool instance is behind a firewall and cannot access the GitHub API, add `.github.com` to your allowlist to enable outbound requests.

---

## Configure Source Control with GitLab

You can use Source Control with GitLab to manage changes using pull requests. You create a repository for Source Control to use, then configure a GitLab project for your Retool instance that commits, pushes, and pulls changes.

Source Control requires the use of [GitLab project access tokens](https://docs.gitlab.com/ee/user/project/settings/project_access_tokens.html). See GitLab's documentation to verify you have access to these tokens.

## Prerequisites

Self-hosted Retool deployments that currently use [Git Syncing](../index.mdx) to sync with GitLab must disable this feature before switching to Source Control. To disable:

- Set `DISABLE_GIT_SYNCING=true` and `VERSION_CONTROL_LOCKED=false` in the `docker.env` file.
- Navigate to **Settings** > **Advanced**, then Remove the GitLab repository URL and branch name from your Git Syncing configuration.

## 1. Create a GitLab project

Create a new project on GitLab. This project repository stores the apps under Source Control from your Retool deployment. Use the following settings:

- Set the correct group under the **Project URL** dropdown.
- Select the **Initialize repository with a README** checkbox.

Next, follow GitLab's [Project access token](https://docs.gitlab.com/ee/user/project/settings/project_access_tokens.html) documentation to create an access token. Set the following scopes for the token:

- `api`
- `read_api`
- `read_repository`
- `write_repository`.

Set the role to `maintainer` or `developer`.

## 2. Configure settings in Retool

Configure the GitLab repository settings.

:::note
Ensure your GitLab domain contains an IPv6 (AAAA) DNS record to prevent server errors when Retool attempts to connect to your GitLab instance. 

You can confirm whether you have IPv6 configured by running:
```
dig  AAAA
```
If the output of the command is `ANSWER: 0`, then the IPv6 DNS record is missing. 
:::

Go to the [Source Control settings](https://login.retool.com/auth/login?source=docs&redirectOnLogin=settings/source-control), and select **Set up GitLab**. Enter the following settings.

| **Setting**          | **Description**                                                                                                                                                         | **Example**                                                  |
| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------ |
| GitLab URL           | Your base GitLab URL. On GitLab Cloud, this is always [https://gitlab.com](https://gitlab.com/). On GitLab self-managed, this is the URL where your instance is hosted. | [https://gitlab.mycompany.com](https://gitlab.mycompany.com) |
| Project access token | The GitLab [project access tokens](https://docs.gitlab.com/ee/user/project/settings/project_access_tokens.html) to authenticate to the GitLab API.                      | glpat-123xyzabc456                                           |
| GitLab project ID    | The numerical project ID for your GitLab project. Find this ID listed below the project's name on the project's homepage.                                               | 278964                                                       |
| GitLab branch        | The default branch for your GitLab project.                                                                                                                             | main                                                         |
| GitLab organization  | The name of your GitLab organization. This can be a username if the project is not part of an organization.                                                             | company, username, engineering, etc.                         |
| GitLab repository    | The name of the GitLab project.                                                                                                                                         | retool-apps                                                  |

Set the following environment variables on your `api` and `jobs-runner` containers.

| **Environment variable**      | **Description**                                                                                                                                    | **Example**        |
| ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ |
| `GITLAB_PROJECT_ACCESS_TOKEN` | The GitLab [project access tokens](https://docs.gitlab.com/ee/user/project/settings/project_access_tokens.html) to authenticate to the GitLab API. | glpat-123xyzabc456 |

If you use your own [SSL certificates](../../self-hosted/guides/certificates.mdx), set the `SSL_CERT_FILE` and `NODE_EXTRA_CA_CERTS` environment variables on the `jobs-runner` and `api` containers to the path to your SSL certificate.

Go to the [Source Control settings](https://login.retool.com/auth/login?source=docs&redirectOnLogin=settings/source-control), and select **Set up GitLab**.

| **Setting**         | **Description**                                                                                                                                                           | **Example**                                                  |
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------ |
| GitLab URL          | Your base GitLab URL. For GitLab Cloud, this is always [https://gitlab.com](https://gitlab.com/). For GitLab self-managed, this is the URL where your instance is hosted. | [https://gitlab.mycompany.com](https://gitlab.mycompany.com) |
| GitLab project ID   | The numerical project ID for your GitLab project. Find this ID listed below the project's name on the project's homepage.                                                 | 278964                                                       |
| GitLab branch       | The default branch for your GitLab project.                                                                                                                               | main                                                         |
| GitLab organization | The name of your GitLab organization. This can be a username if the project is not part of an organization.                                                               | company, username, engineering, etc.                         |
| GitLab repository   | The name of the GitLab project.                                                                                                                                           | retool-apps                                                  |

Set the following environment variables on your `api` and `jobs-runner` containers.

| **Environment variable**      | **Description**                                                                                                                                                           | **Example**                                                  |
| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------ |
| `GITLAB_URL`                  | Your base GitLab URL. For GitLab Cloud, this is always [https://gitlab.com](https://gitlab.com/). For GitLab self-managed, this is the URL where your instance is hosted. | [https://gitlab.mycompany.com](https://gitlab.mycompany.com) |
| `GITLAB_PROJECT_ACCESS_TOKEN` | The GitLab [project access tokens](https://docs.gitlab.com/ee/user/project/settings/project_access_tokens.html) to authenticate to the GitLab API.                        | glpat-123xyzabc456                                           |
| `GITLAB_PROJECT_ID`           | The numerical project ID for your GitLab project. Find this ID listed below the project's name on the project's homepage.                                                 | 278964                                                       |
| `GITLAB_MAIN_BRANCH`          | The default branch for your GitLab project.                                                                                                                               | main                                                         |
| `GITLAB_ORGANIZATION_NAME`    | The name of your GitLab organization. This can be a username if the project is not part of an organization.                                                               | company, username, engineering, etc.                         |
| `GITLAB_REPOSITORY_NAME`      | The name of the GitLab project.                                                                                                                                           | retool-apps                                                  |
| `GITLAB_PROJECT_SLUG`         | The URL path to your GitLab project.                                                                                                                                      | retool/eng/retool-apps                                       |
| `VERSION_CONTROL_LOCKED`      | When set to `true`, the instance becomes a "read-only" instance and users cannot create or edit apps. Available on Retool versions 2.91 and later.                        | false                                                        |

If you use your own [SSL certificates](../../self-hosted/guides/certificates.mdx), set the `SSL_CERT_FILE` and `NODE_EXTRA_CA_CERTS` environment variables on the `jobs-runner` and `api` containers to the path to your SSL certificate.

Your Retool instance uses GitLab-specific environment variables to recreate the host URL for your GitLab project. You must provide values for either `GITLAB_ORGANIZATION_NAME` and `GITLAB_REPOSITORY_NAME`, or `GITLAB_PROJECT_SLUG`. For example, if your project URL is `https://gitlab.com/retool-source/gitlab-retool-dev`, you can set `GITLAB_ORGANIZATION_NAME=retool-source` and `GITLAB_REPOSITORY_NAME=gitlab-retool-dev`, or `GITLAB_PROJECT_SLUG=retool-source/gitlab-retool-dev`.

## 3. Verify GitLab settings

On self-hosted Retool versions 2.97 and later, go to the [Source Control settings page](https://login.retool.com/auth/login?source=docs&redirectOnLogin=settings/source-control) to verify your GitLab project is correctly configured. If you still see the **Set up GitLab** option, confirm your environment variables are set.

To confirm Retool can connect to your GitLab project, select **Test connection**.

On self-hosted Retool versions 2.96 and earlier, confirm GitLab is correctly set up by [protecting your first application](../guides/protect/apps.mdx).

## 4. Save your settings

Click **Save and deploy** to save your settings.

---

## Source Control tutorials

import Tutorial from '/docs/_partials/_doctypes/_tutorial.mdx';

:::tip Learn more about self-hosted architecture

Read the [system architecture](../self-hosted/concepts/architecture.mdx) guide to learn more about the containers, services, and dependencies for self-hosted deployment instances.

:::

---

## Enforce SSO

You can disable Retool's built-in authentication method (email address and password) and require that all users log in using SSO credentials.

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

:::danger Verify your SSO configuration is correct

Cloud-hosted Retool organizations must configure and test SSO before enabling this setting. Disabling Retool's standard authentication can lock all users out of your organization.

:::

Navigate to your organization's [Single Sign On (SSO)](https://login.retool.com/auth/login?source=docs&redirectOnLogin=settings/custom-sso) settings and toggle **Disable Login with Email and Password**.

Set the `DISABLE_USER_PASS_LOGIN` [environment variable](../../../self-hosted/reference/environment-variables/index.mdx#disable_user_pass_login) and restart your deployment.

---

## Restrict SSO domains

You can restrict SSO authentication so that only users with email addresses from these domains are allowed. Separate multiple domains with commas.

On self-hosted deployments, this setting also disables username and password login. It is set using the `RESTRICTED_DOMAIN` [environment variable](../../../self-hosted/reference/environment-variables/index.mdx#restricted_domain).

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

1. Navigate to your organization's [Single Sign On (SSO)](https://login.retool.com/auth/login?source=docs&redirectOnLogin=settings/custom-sso) settings.
2. Select your configured SSO provider.
3. Specify one or more domains as a comma-separated list.

1. Navigate to **Settings > Single Sign-on (SSO)**.
2. Select your configured SSO provider.
3. Specify one or more domains as a comma-separated list.

This setting [disables username and password login](enforce-sso.mdx). It is also set using the `RESTRICTED_DOMAIN` [environment variable](../../../self-hosted/reference/environment-variables/index.mdx#restricted_domain).

---

## Configure SSO session duration

SSO sessions have a one week duration. Once authenticated, a user can remain signed in for up to a week before they need to sign in again. If you need to restrict session durations further, you can reduce this to 12 hours.

Navigate to your organization's [Single Sign On (SSO)](https://login.retool.com/auth/login?source=docs&redirectOnLogin=settings/custom-sso) settings and toggle **Use short session**.

---

## Automatically trigger SSO authentication

Retool can automatically prompt users to sign in with SSO and begin the OAuth 2.0 or SAML authentication flow.

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

1. Navigate to your organization's [Single Sign On (SSO)](https://login.retool.com/auth/login?source=docs&redirectOnLogin=settings/custom-sso) settings.
2. Select your configured SSO provider.
3. Toggle the **Trigger Login Automatically** setting.

1. Navigate to **Settings > Single Sign-on (SSO)**.
2. Select your configured SSO provider.
3. Toggle the **Trigger Login Automatically** setting.

You can also set one of the `TRIGGER_OAUTH_2_SSO_LOGIN_AUTOMATICALLY` or `TRIGGER_SAML_LOGIN_AUTOMATICALLY` [environment variables](../../../self-hosted/reference/environment-variables/index.mdx#trigger_oauth_2_sso_login_automatically).

---

## Sync Google Groups

Organizations using Google Sign-in SSO that manage users with [Google Workspace](https://workspace.google.com/) can map Google Group memberships to Retool permission groups.

## Requirements

To configure Google Groups in Retool, you must have administrator access to your Google Workspace.

## 1. Set up Google as your SAML Identity Provider

To sync Google Groups to Retool, you must first set up Google as your SAML provider on Retool. If you already use Google SAML, you can skip ahead to [configure Retool to use your SAML app](#configure-retool-to-use-the-saml-app).

### Create a custom SAML app on Google

Follow the steps in Google's [instructions to create a custom SAML app](https://support.google.com/a/answer/6087519?hl=en) using the configuration below.

On the **Service provider details** screen, enter the following fields.

| Setting       | Value                                                                                                                                  |
| ------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| **ACS URL**   | On Retool Cloud, `https://your-company.retool.com/api/saml/login`. On self-hosted Retool,`https://retool.your-company.com/saml/login`. |
| **Entity ID** | `https://your-company.retool.com`                                                                                                      |

On the **Attribute mapping** screen, map your Google user details to the field names to show on Retool users. Under **App attributes**, include `email`, `firstName`, and `lastName`.

By default, Google turns off SAML apps. To turn on the app for all users, in **User access**, change the Service status to **On for everyone**.

### Configure Retool to use the SAML app

After creating your custom SAML app, you need to configure Retool to use it for SSO.

1. On Retool Cloud, go to **Settings > Single Sign-On (SSO)**. On self-hosted deployments, go to **Settings > Advanced**.
2. Copy the contents of the IdP Metadata file you downloaded when creating your SAML app and paste them into the **IdP Metadata XML** field.

3. Enable **JIT user provisioning** to allow Retool to automatically provision an account when users sign in for the first time using SAML. This step is optional, but recommended.

4. Set the `DOMAINS` environment variable to match the entity ID you set in your SAML settings. If you enabled JIT user provisioning, you also need to set the `DEFAULT_GROUP_FOR_DOMAINS` [environment variable](../../../self-hosted/reference/environment-variables/index.mdx#default_group_for_domains).

```
DOMAINS=retool.your-company.com
```

## 2. Set up Google Secure LDAP service

Retool uses the Google Secure LDAP Service to fetch a user's Google Group memberships to sync with Retool.

1. Log in to your Google Admin console.
2. Navigate to **Apps** > **LDAP** and click **Add LDAP client**.
3. Enter a client name in the setup wizard, then add the following permissions in the **Access permissions** settings. Retool recommends the following access scopes, but you can also limit to selected OUs and groups.

- Under **Verify user credentials**, select **Entire domain**.
- Under **Read user information**, select **Entire domain**.
- Toggle on **Read group information**.

4. Create the client and download the generated certificate bundle.

5. Switch the **Service status** to **On**.

### Configure Retool to use the LDAP service

Configure the following settings to retrieve Google Groups from your LDAP service.

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";
import SettingsOverride from "../../_settings-override.mdx";

On Retool Cloud and self-hosted Retool versions 3.16 and later, go to **Settings > Single Sign-On (SSO)**, select **Google LDAP** under **Group syncing**, and enter the following settings.

| **Setting**            | **Value**                     | **Description**                                       |
| ---------------------- | ----------------------------- | ----------------------------------------------------- |
| Server URL             | `ldaps://ldap.google.com:636` | The LDAP server URL for Google's Secure LDAP Service. |
| Server name            | `ldap.google.com`             | The LDAP server name.                                 |
| Base domain components | `dc=example,dc=com`           | Your organization's email domain in DC syntax.        |
| Server certificate     | _Encoded certificate_         | The certificate from the downloaded bundle.           |
| Server key             | _Encoded private key_         | The private key from the downloaded bundle.           |

On self-hosted Retool versions earlier than 3.16, add the following [environment variables](../../../self-hosted/reference/environment-variables/index.mdx#authentication) on your Retool instance.

| **Environment variable**      | **Value**                     | **Description**                                       |
| ----------------------------- | ----------------------------- | ----------------------------------------------------- |
| `LDAP_SERVER_URL`             | `ldaps://ldap.google.com:636` | The LDAP server URL for Google's Secure LDAP Service. |
| `LDAP_SERVER_NAME`            | `ldap.google.com`             | The LDAP server name.                                 |
| `LDAP_BASE_DOMAIN_COMPONENTS` | `dc=example,dc=com`           | Your organization's email domain in DC syntax.        |
| `LDAP_SERVER_CERTIFICATE`     | _Encoded certificate_         | The certificate from the downloaded bundle.           |
| `LDAP_SERVER_KEY`             | _Encoded private key_         | The private key from the downloaded bundle.           |
| `LDAP_SYNC_GROUP_CLAIMS`      | True                          | Enable the sync.                                      |

Your Google Workspace domain must use domain component (DC) syntax.

| **Email Domain**   | **Domain Component (DC) format** |
| ------------------ | -------------------------------- |
| `example.com`      | `dc=example,dc=com`              |
| `mail.company.com` | `dc=mail,dc=company,dc=com`      |

### Role mapping

import GroupsAdminWarning from "../../_groups_admin_warning.mdx";

Role mapping allows you to map group email names from Google Groups to group names on Retool. The `LDAP_ROLE_MAPPING` environment variable uses an arrow syntax for the mapping. For example, to map the group `retool-admins@yourcompany.com` to the default `Admin` group on Retool and `support@yourcompany.com` to a `Support` Retool group, you might set the following.

| **Setting**       | **Environment variable** | **Example value**                          |
| ----------------- | ------------------------ | ------------------------------------------ |
| LDAP Role Mapping | `LDAP_ROLE_MAPPING`      | retool-admins -> admin, support -> Support |

import RoleMappingCaseSensitivity from "../../_role-mapping-case-sensitivity.mdx";

Retool automatically creates the explicitly mapped groups on Retool if they don't already exist. If `LDAP_ROLE_MAPPING` is not set, Retool attempts to sync the user's group slugs with existing Retool groups.

On self-hosted deployments, when you first configure group syncing, it's useful to set the `LDAP_ROLE_MAPPING_DISABLED` environment variable to true. When true, Retool does not actually sync or create any new groups when a user logs in, but does show you logs for the steps it _would_ have performed. This allows you to debug your setup before actually changing groups for users.

## Troubleshoot group syncing

Use the following information to resolve common issues when configuring LDAP group syncing.

### Insufficient access rights

If you see an error indicating `Insufficient Access Rights`, ensure the LDAP service status is set to **On** in the Google Admin Console.

### Error configuring app for user

If you see an error which contains `app_not_configured_for_user`, ensure the LDAP service status is set to **On** in the Google Admin Console. Verify that you correctly set your [DOMAINS variable](#configure-retool-to-use-the-saml-app) and that it matches the [entity ID](#create-a-custom-saml-app-on-google) you previously set.

---

## Configure group syncing and role mapping

import GroupsAdminWarning from "../../_groups_admin_warning.mdx";

Retool can sync groups from your SSO provider for authorization. The approach you use depends on your configured SSO provider.

| Provider                                                                | Supported group sync methods                                                                                                     |
| ----------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| [Sign in with Google](../../tutorials/google/google-sign-in.mdx)       | [LDAP Google Group sync](google-groups.mdx)                                                              |
| [Google OIDC](../../tutorials/google/oidc.mdx)                         |                                                                                                                                  |
| [Okta OIDC](../../tutorials/okta/oidc.mdx)                             | [Role mapping](../../tutorials/okta/oidc.mdx#role-mapping-with-okta-group-claims)                                               |
| [OneLogin OIDC](../../tutorials/onelogin.mdx)                          | [Role mapping](../../tutorials/custom/oidc.mdx#role-mapping)                                                                    |
| [Auth0 OIDC](../../tutorials/auth0.mdx)                                | [Role mapping](../../tutorials/custom/oidc.mdx#role-mapping)                                                                    |
| [Microsoft Entra ID OIDC](../../tutorials/microsoft-entra-id/oidc.mdx) | [Role mapping](../../tutorials/microsoft-entra-id/oidc.mdx#optional-settings)                                                   |
| [Generic OIDC](../../tutorials/custom/oidc.mdx)                        | [Role mapping](../../tutorials/custom/oidc.mdx#role-mapping)                                                                    |
| [Okta SAML](../../tutorials/okta/saml.mdx)                             | [SAML group sync and role mapping](saml-group-sync.mdx), [SCIM provisioning](../scim-user-provisioning.mdx) |
| [OneLogin SAML](../../tutorials/custom/saml.mdx)                       | [SAML group sync and role mapping](saml-group-sync.mdx)                                                              |
| [Microsoft Entra ID SAML](../../tutorials/microsoft-entra-id/saml.mdx) | [SAML group sync and role mapping](saml-group-sync.mdx)                                                              |
| [Active Directory Federation Services SAML](../../tutorials/adfs.mdx)  | [SAML group sync and role mapping](saml-group-sync.mdx)                                                              |
| [Generic SAML](../../tutorials/custom/saml.mdx)                        | [SAML group sync and role mapping](saml-group-sync.mdx), [SCIM provisioning](../scim-user-provisioning.mdx) |

## Group sync

If your groups are named the same in your IdP and Retool, group syncing happens automatically when users log in. For example, if you have an OIDC group claim of **Engineers** and a Retool group named **Engineers**, the users in the OIDC group are automatically added to the Retool group on login.

Group syncing occurs when users log in, so if you change groups in your IdP, users need to log out and in again for changes to be reflected. OIDC groups are created in Retool automatically. You assign group membership in your IdP. Manual edits to group memberships are overwritten with IdP groups on subsequent logins, so manual editing is not recommended.

## Role mapping

To map the groups from your IDP to differently named groups in Retool, use role mapping. For example, if your LDAP/SAML group claim is **Admins** and in Retool the corresponding group is **RetoolAdmins**, you can map groups so members of the **Admins** group are added to the **RetoolAdmins** group on login.

Role mapping occurs when users log in, so if you change groups in your IdP, users need to log out and in again for changes to be reflected. OIDC groups are created in Retool automatically. Editing Retool group membership in Retool is disabled. You assign group membership in your IdP.

With [SCIM provisioning](../scim-user-provisioning.mdx), groups are pushed automatically from your IdP to Retool using API requests. This means you can push group membership on an automated schedule or manually from your IdP.

SCIM calls specific API endpoints to add users to groups, remove users from groups, and create groups.

SCIM matches groups by name, so user groups in your IdP and Retool need to have the same name. You can map a group name to one of Retool's 4 default groups (**Admin**, **Viewer**, **Editor**, **All Users**).

:::note

SCIM requires your Retool instance is open to API requests from your IdP. You should add your IdP's IP addresses to your instance’s allowlist.

:::

### Role mapping and Spaces

If you use [Retool Spaces](../../../org-users/tutorials/spaces.mdx), you may want to namespace your IdP groups by Space if they're in the same IdP instance. For example, if you have distinct engineering teams building apps in different spaces, Retool recommends splitting "Engineering" into "Engineering - Treasury" and "Engineering - Issuing" IdP groups for your Treasury and Issuing Spaces.

---

## Sync SAML group memberships

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

Retool supports syncing group memberships at the time of SAML login. This requires passing group membership for users to Retool in your SAML assertion. Retool parses these memberships and adds or removes users to the groups when a user logs in to Retool.

import GroupsAdminWarning from "../../_groups_admin_warning.mdx";

## Requirements

To use SAML group syncing, you must:

- Have [SAML SSO](../../tutorials/custom/saml.mdx) configured on Retool.
- Create Retool groups prior to syncing SAML groups. Retool groups are **not** created in the login flow.

:::note

Upon login, users are **removed** from any groups that do not have a corresponding IdP group. This includes users assigned to the Admin group. You should test this flow with a non-admin or test user.

:::

## 1. Include groups in your SAML application

Configure your SAML application to include group attributes in the SAML response.

### Okta

SAML Group Syncing directly matches the user groups by name, so Retool copies the naming convention you use in Okta.

In your Okta SAML application, configure the response to include Group Attributes. Set the group filter as desired. For example, you might set the name to `groups` and the regex filter to `.*` to include all of a user's groups.

See [Okta documentation](https://help.okta.com/en-us/Content/Topics/Apps/define-group-attribute-statements.htm) for more details about configuring Okta group attributes.

### Microsoft Entra ID

For Microsoft Entra ID, use [group claims](https://learn.microsoft.com/en-us/entra/identity/hybrid/connect/how-to-connect-fed-group-claims) to configure group attributes. When configuring group claims:

1. Set the group attribute name to `http://schemas.microsoft.com/ws/2008/06/identity/claims/groups` to use the Retool default.
1. Configure the group claim to [include the group display name for cloud-only groups](https://learn.microsoft.com/en-us/entra/identity/hybrid/connect/how-to-connect-fed-group-claims#emit-cloud-only-group-display-name-in-token). Set the **Source attribute** to **Cloud-only group display names**.

## 2. Update configuration

Add your SAML configuration to your Retool instance through environment variables or the settings page.

import SettingsOverride from "../../_settings-override.mdx";

On Retool Cloud and self-hosted Retool versions 3.16 and later, on the **Settings > Single Sign-On (SSO)** page, select **SAML Group Attribute** under **Sync Groups to Retool**. In **Groups attribute**, enter the attribute defined in your SAML assertion. Save your settings.

On self-hosted instances earlier than 3.16, configure the `SAML_SYNC_GROUP_CLAIMS` [environment variable](../../../self-hosted/reference/environment-variables/index.mdx#saml_sync_group_claims) to enable SAML group sync.

```
SAML_SYNC_GROUP_CLAIMS=true
```

Set the `SAML_GROUPS_ATTRIBUTE` environment variable to the group attribute defined in your SAML assertion. `SAML_GROUPS_ATTRIBUTE` defaults to `groups` if the environment variable is not set.

```
SAML_GROUPS_ATTRIBUTE="groups"
```

You must restart your Retool instance whenever you make changes to environment variables.

## 3. Role mapping

If your permission group names differ in Retool and your IdP, you can use role mapping to map your IdP groups to Retool groups. Role mapping is not necessary if your group names are the same, e.g., your admin group is named `admin` in both your IdP and Retool.

Roles are case sensitive. This means:

1. Roles set in your IdP that you specify in the following settings need to match exactly. For example, if you have a `Retool Admin` role in your IdP, you need to pass `Retool Admin`.
2. Roles within Retool are always lowercase. For example, if you have a `Retool Admin` role within your IdP, and you want to map it to Retool's `admin` role, you need to set it using `Retool Admin → admin`.

On Retool Cloud and self-hosted Retool versions 3.16 and later, from **Settings > Single Sign-On (SSO)** under **SAML Groups Attribute**, add role mapping as a comma-separated list to the **Roles mapping** field. For example, to map the group `retool-admins@yourcompany.com` to the default `Admin` group on Retool and `support@yourcompany.com` to a `Support` Retool group, you can set:

```
retool-admins -> admin, support -> Support
```

Members of the `retool-admin` group in your IdP are now automatically added to the `admin` group when they log in, and `support` members added to `Support`.

On self-hosted Retool, set the `LDAP_ROLE_MAPPING` [environment variable](../../../self-hosted/reference/environment-variables/index.mdx#ldap_role_mapping) to a comma-separated list of mapped groups. For example, to map the group `retool-admins@yourcompany.com` to the default `Admin` group on Retool and `support@yourcompany.com` to a `Support` Retool group, you can set:

```
LDAP_ROLE_MAPPING=retool-admins -> admin, support -> Support
```

Members of the `retool-admin` group in your IdP are now automatically added to the `admin` group when they log in, and `support` members added to `Support`.

You must restart your Retool instance whenever you make changes to environment variables.

## 4. Confirm group sync

You can test out your configuration by logging in from an incognito window.

On self-hosted instances, you can also confirm group syncing by reviewing the `api` container logs after users log in. The logs contain SAML group memberships for each user who has logged in.

```
api_1 | [SAML] - Received SAML Login Response, parsing...
api_1 | [SAML] - Validating response...
api_1 | [SAML] - Validated response, and received the following attributes {
api_1 | email: 'johndoe@retool.com',
api_1 | firstName: 'John',
api_1 | lastName: 'Doe',
api_1 | groups: [ 'samlgroup', 'Everyone', 'anothergroup' ]
api_1 | }
```

---

## Enable JIT user provisioning for SSO

JIT user provisioning is optional, but recommended. When you enable this setting, Retool provisions user accounts when users sign in over SSO for the first time. This saves time for admins, who then don't need to manually invite users to Retool. The identity provider is still the source of truth and determines which users have access to Retool. Users must also still be granted access in the identity provider before an account is created in Retool.

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

1. Navigate to your organization's [Single Sign On (SSO)](https://login.retool.com/auth/login?source=docs&redirectOnLogin=settings/custom-sso) settings.
2. Select your configured SSO provider.
3. Toggle **Enable JIT user provisioning**.

1. Navigate to **Settings > Single Sign-on (SSO)**.
2. Select your configured SSO provider.
3. Toggle **Enable JIT user provisioning**.

If you're setting up SSO with Google, you also need to set the `DEFAULT_GROUP_FOR_DOMAINS` [environment variable](../../self-hosted/reference/environment-variables/index.mdx#default_group_for_domains).

---

## Migrate from SAML to OIDC

You can use both [OpenID and SAML SSO methods](../index.mdx) to log users in to Retool. OIDC SSO has additional functionality you may want to enable based on your use case. For example, with OIDC, you can use ID tokens from your IdP for authentication in Retool resources.

On self-hosted instances, ensure you follow the order outlined in this guide. On Retool Cloud, you can update settings directly in **Settings > Single Sign-On (SSO)**.

:::note

To minimize downtime, apply all changes to environment variables at the same time. When you restart the Retool server, your instance immediately replaces the SAML-based SSO with the new OIDC-based SSO.

:::

## 1. Enable OIDC SSO

Use the configuration guides—[Okta](../tutorials/okta/oidc.mdx), [Google](../tutorials/google/oidc.mdx), [OneLogin](../tutorials/onelogin.mdx), or [another provider](../tutorials/custom/oidc.mdx)—to enable OIDC authentication on your instance. This involves creating an OIDC app for your Retool instance and setting `CUSTOM_OAUTH2_SSO_*` [environment variables](../../self-hosted/reference/environment-variables/index.mdx#authentication). You can set these variables in Retool under **Settings > Configuration variables**. Do not restart your Retool instance yet.

## 2. Remove SAML metadata

:::note

If you store SAML metadata in the Retool settings UI, ensure you added the OIDC configuration in the previous step. Settings stored in the UI are applied immediately.

:::

SAML IDP metadata is located in either the **Settings** > **Advanced** page in the Retool UI, or stored in the `SAML_IDP_METADATA` environment variable. Remove or comment out the metadata.

## 3. Restart your Retool instance

Restart your Retool instance and select **Login with SSO** to test your changes. If you cannot log in, comment out the `RESTRICTED_DOMAIN` environment variable to temporarily enable username and password logins. Confirm the rest of your environment variables are correctly set. You may also want to enable [JIT user provisioning](jit-provisioning.mdx).

---

## Provision users with SCIM

SCIM, or System for Cross-domain Identity Management, is the industry standard for automatically provisioning user accounts. It's useful when using a third-party SSO provider for authentication, such as Okta, because it allows you to automatically create, update, and deactivate user accounts in applications such as Retool.

Refer to the [SCIM reference](../../scim/index.mdx) to learn how to make requests to the SCIM endpoints Retool implements.

import GroupsAdminWarning from "../../sso/_groups_admin_warning.mdx";

## Requirements

You can use SCIM provisioning on Retool Cloud or self-hosted Retool versions **2.32.1** and later.

If you use Okta, follow the [configuration steps](#configuration-steps) to create an application. If you use another provider, see the documentation for your SSO provider to learn how to add Retool as an application.

If your Retool instance is behind a firewall and you use Okta, [add Okta's IP addresses to your allowlist](https://help.okta.com/en-us/Content/Topics/Security/ip-address-allow-listing.htm) so it can connect to Retool. Optionally, you can add a rule to your load balancer to only expose the needed paths: `/api/scim/v2/*`.

## SCIM server authentication

To use SCIM, you first add Retool as an application in your SSO provider. After you assign the application to a user, an account in Retool is automatically created. That happens because your SSO provider makes an API request to Retool and instructs Retool to create an account for your user.

To ensure accounts are only created by authorized users, Retool uses a **Bearer Token** authentication scheme. You define a secret that only Retool and your SSO provider know.

### Generate tokens in Retool

You can create API tokens with the **SCIM API** scope in the **API** settings for your organization.

import Tokens from '/docs/_partials/_tokens.mdx';

### Create tokens outside of Retool

Use the following instructions to set the `SCIM_AUTH_TOKEN` environment variable:

- For organizations on versions 3.4.0 or earlier, generate your own token and set it as the `SCIM_AUTH_TOKEN` token [environment variable](../../self-hosted/reference/environment-variables/#property-SCIM_AUTH_TOKEN). 
- For organizations on versions after 3.4.0, you can use one of the following options: 
  * Set `SCIM_AUTH_TOKEN` environment variable to the the same token that was generated in the UI.
  * Omit the `SCIM_AUTH_TOKEN` environment variable if the token was also generated in the UI.
  * Use the Retool API to [create an access token](../../api/create-an-access-token).

```
SCIM_AUTH_TOKEN=A_SECRET_TOKEN
```

:::warning 
The secret must be at least 10 characters long.

The SCIM API endpoints that Retool makes available are highly sensitive and should be protected. Use a long and secure secret token that nobody can guess.

:::

After creating your token, restart your Retool instance.

### Test your token

After creating your token, test it by opening `https://yourRetoolAppDomain/api/scim/v2/Schemas`. There, you should see an error that looks like this:

```
{"detail":"SCIM request not authorized","schemas":["urn:ietf:params:scim:api:messages:2.0:Error"]}
```

After that's been set, only requests to Retool's SCIM API with the header `Authorization: Bearer YOUR_SECRET_TOKEN` will be accepted.

## Microsoft Entra ID requirements

In Microsoft Entra ID, you must add `aadOptscim062020` to your **Tenant URL** for your requests to be SCIM 2.0 compliant and work in Retool. See the [Microsoft Entra ID documentation](https://learn.microsoft.com/en-us/azure/active-directory/app-provisioning/application-provisioning-config-problem-scim-compatibility#flags-to-alter-the-scim-behavior) for more.

## Okta-specific requirements

When using Okta as an SSO provider, the SCIM API offers the following features:

- **Create Users:** New or existing users in Okta are pushed to Retool as new users.
- **Update User Attributes:** Updates to user profiles in Okta are pushed to Retool.
- **Deactivate Users:** Users deactivated in Okta are automatically disabled in Retool. They are immediately logged out and do not count towards the billable user count. If a user is reactivated, they regain access to Retool and keep all previously specified access controls.
- **Importing Users:** Users that are created manually in Retool can be imported into Okta as Okta users.
- **Group Push:** Group Push lets you take existing Okta groups and their memberships and push them to Retool.

### Configuration steps

Before you begin, [create an access token](#scim-server-authentication).

1. In Okta, add the Retool application to your Okta account.

2. Enter your Retool domain in the following format: `YOUR.RETOOL.INSTANCE.COM`. For example, if you log into `https://YOUR.RETOOL.INSTANCE.COM/`, enter: YOUR.RETOOL.INSTANCE.COM”.

:::note

Do not include the `https://` prefix in the Retool domain—it is automatically included.

:::

3. Go to the **Provisioning** tab and click **Configure API Integration**.

4. Enable the **Enable API integration** setting, and enter your [Retool SCIM API access token](#scim-server-authentication).

5. Click **Test API credentials** to ensure the connection is working.

:::note

Do not include a `Bearer` prefix in the Auth Token as this is automatically included.

:::

6. Click **Save** to enable provisioning for the Retool app.

7. Activate the desired provisioning features. In the provisioning settings for the app, under **To App**, click **Edit** and enable **Create Users**, **Update User Attributes**, and **Deactivate Users**. Save your settings.

8. If you use [Retool user attributes](../../org-users/guides/user-management/user-attributes.mdx#assign-attribute-values-to-users) and want to connect Okta custom attributes to the **metadata** user attribute, set **External namespace** to **metadata** on the Okta custom attribute.

You can now start assigning users to the Retool app and they will be automatically created in Retool.

### Enable group push

If you're setting up Okta for the first time, you will get access to Group Push with the rest of the Okta SCIM features, and you can ignore the rest of this guide. If you've already set up Retool with Okta, you will have to re-authenticate your account to enable Group Push. To do this, follow these steps.

1. Log in to your Okta org as an Admin.
2. Open the Admin UI.
3. Open your Retool application instance.
4. Go to the **Provisioning** tab.
5. On the **Settings** sidebar, click **Integration**.
6. Click **Edit** and then **Test API Credentials**, and then click **Save**. If you've followed the **Configuration Steps** of this guide, your credentials should already be in the **Integration** section.

SCIM should now be enabled for your Retool instance.

### Push to default Retool groups

:::note

This feature is available on self-hosted Retool versions 2.94 and later.

:::

You cannot rename Retool's default groups using SCIM group mapping, but you can map your custom groups to the four default Retool groups (`admin`, `editor`, `viewer`, `All Users`). Okta is used as an example, but a similar process also works for other identity providers.

1. Follow instructions in [Okta](https://help.okta.com/en-us/content/topics/users-groups-profiles/usgp-import-users-app.htm) to import groups using the **Import Now** button. After importing, the four Retool default groups show in Okta as `Retool production - [group name]`. These groups are read-only in Okta.

2. To push custom Okta groups to Retool default groups, follow the [Okta instructions](https://help.okta.com/en-us/content/topics/users-groups-profiles/usgp-enable-group-push.htm). Check **Push group members immediately** if you want to immediately sync group memberships.

3. If you disabled the automatic push in step 2, you can manually push your groups. Confirm your group has a **Push Status** of **Active** in the **Push Groups** tab.

4. Log in to Retool and go to the **Users** page to verify your group members are synced.

---

## SSO how-to guides

import Howto from '/docs/_partials/_doctypes/_howto.mdx';

:::tip Get started with SSO

If you want to get started with SSO at Retool:

- Read the [quickstart](./quickstart) to learn about the fundamental concepts.
- Follow a [tutorial](./tutorial/) to set up an SSO provider.

:::

---

## SSO documentation

Single sign-on (SSO) is a user authentication tool that enables users to securely access multiple applications and services using one set of credentials. Rather than require users to create additional usernames and passwords for Retool, you can centralize logins to a single identity provider (IdP).

---

## SSO quickstart

This guide serves as an introduction to single sign-on (SSO). It covers many of the concepts and terminology you would come across when configuring and authenticating with SSO. After reading this page, you should have a good understanding of the fundamentals for using SSO to authenticate users.

## Introduction

SSO is a user authentication tool to securely access multiple applications and services using one set of credentials. Rather than require users to create additional usernames and passwords for Retool, you can centralize logins to a single identity provider (IdP). SSO is primarily used for authentication, though Retool also supports syncing groups for authorization.

## Requirements

Retool supports OpenID Connect (OIDC) and Security Assertion Markup Language (SAML) identity providers. Organizations can map IdP roles to Retool permission groups using OIDC role mapping or SAML group sync. Retool also supports LDAP Google Group sync when using Sign in with Google.

Setup tutorials are available for the following identity providers.

| Provider                                                                                                                       | Protocol                                                                                                 |
| ------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------- |
| [Active Directory Federation Services (AD FS)](https://learn.microsoft.com/en-us/windows-server/identity/ad-fs/ad-fs-overview) | [SAML](./tutorials/adfs.mdx)                                                                           |
| [Auth0](https://auth0.com/)                                                                                                    | [OIDC](./tutorials/auth0.mdx)                                                                          |
| [Google](https://cloud.google.com/architecture/identity/single-sign-on)                                                        | [OIDC](./tutorials/google/oidc.mdx) and [Sign In with Google](./tutorials/google/google-sign-in.mdx) |
| [Microsoft Entra ID](https://learn.microsoft.com/en-us/entra/fundamentals/whatis)                                              | [OIDC](./tutorials/microsoft-entra-id/oidc.mdx) and [SAML](./tutorials/microsoft-entra-id/saml.mdx)  |
| [Okta](https://www.okta.com/)                                                                                                  | [OIDC](./tutorials/okta/oidc.mdx) and [SAML](./tutorials/okta/saml.mdx)                              |
| [OneLogin](https://www.onelogin.com/)                                                                                          | [OIDC](./tutorials/onelogin.mdx)                                                                       |
| Other IdP providers                                                                                                            | [OIDC](./tutorials/custom/oidc.mdx) and [SAML](./tutorials/custom/saml.mdx)                          |

If your organization uses [Google Workspace](https://workspace.google.com/) and all users share the same domain, users can log in automatically to the same Retool organization.

## SSO authentication flow

The following steps illustrate the authentication flow between the IdP and Retool when a user signs in.

1. User navigates to a Retool organization URL.
2. Retool redirects the user to the identity provider to sign in.
3. The user signs in with their credentials.
4. The user confirms with the IdP that they want to sign in to Retool.
5. The user is redirected back to Retool.
6. Retool checks with the IdP to confirm the user is authenticated.
7. The IdP validates the authentication request.
8. The user successfully signs in and can access Retool.

## Enforce and restrict SSO

Retool includes support for enforcing SSO and disabling Retool's built-in authentication method. Organizations can also set session durations, restrict which email address domains are allowed, and trigger SSO automatically.

Each [Retool Space](../org-users/tutorials/spaces.mdx) can only have a single authentication method, and it's not possible to set the login method to _username and password_ for some users and _SSO_ for others within a single space. Refer to [Spaces and SSO](../org-users/tutorials/spaces.mdx#spaces-and-sso) for more information.

:::note

To provide multiple authentication methods, you could utilize multiple spaces. For example, create one space for SSO authentication and another space for username and password authentication. Sync both spaces via [Source Control](../source-control/index.mdx), maintaining the same apps and resources for both. Each space requires independent setup of users, permissions, and other settings.

:::

## Group syncing and user provisioning

---

## SSO environment variables

import Sso from "@site/docs/self-hosted/reference/environment-variables/authentication.mdx";

---

## SSO glossary

import SharedGlossary from '../../_partials/_glossary.mdx';

---

## Organization and users reference

import Reference from '/docs/_partials/_doctypes/_reference.mdx';

:::tip Get started with your organization and users

If you are unfamiliar with managing your organization and want to get started, read the [quickstart](./quickstart) to learn about the fundamental concepts.

:::

---

## Configure Active Directory Federation Services SAML SSO

import Test from "./_partials/_test-connection.mdx"

Use the following guide to integrate Retool with Active Directory Federation Services 3.0.

## 1. Create a relying party trust

Follow the [Relying Party Trust wizard](https://learn.microsoft.com/en-us/windows-server/identity/ad-fs/operations/create-a-relying-party-trust#to-create-a-claims-aware-relying-party-trust-manually) in Active Directory with the following settings.

1. In **Select Data Source**, select **Enter data about the relying party manually**.
2. In **Choose Profile**, select **AD FS profile**.
3. In **Configure Certificate**, do not upload a certificate.
4. In **Configure URL**, select **Enable support for SAML 2.0 WebSSO Protocol**. On Retool Cloud, enter `https://your-sso-url.retool/api/saml/login`. On self-hosted Retool, enter `https://your-sso-url.retool/saml/login`. Replace `your-sso-url` with your Retool single-sign on domain. This is often `retool.yourcompany.com`.
5. In **Configure Identifiers**, add your single-sign on domain without the protocol as a **Relying party trust identifier**. For example, use `retool.yourcompany.com` instead of `https://retool.yourcompany.com`.
6. Finish the wizard.

## 2. Send LDAP attributes as claims

Follow the steps to [send LDAP attributes as claims](https://learn.microsoft.com/en-us/windows-server/identity/ad-fs/operations/create-a-rule-to-send-ldap-attributes-as-claims).

1. On the **Choose rule type** page, select **Send LDAP Attributes as Claims**.
2. On the **Configure claim rule** page, choose **Active Directory** as the attribute store. Fill in the following settings.

| LDAP Attribute  | Outgoing Claim Type     |
| --------------- | ----------------------- |
| Email addresses | `email`                 |
| Email addresses | AD FS 1.x Email address |
| Given Name      | `firstName`             |
| Surname         | `lastName`              |

3. Select **Transform an Incoming Claim** and select the following settings.

| Setting                  | Value                   |
| ------------------------ | ----------------------- |
| Incoming claim type      | AD FS 1.x Email Address |
| Outgoing claim type      | Name ID                 |
| Outgoing claim ID format | Email                   |

4. Select **Pass through all claim values** and save the settings.

## 3. Configure Retool with IdP metadata

Export the metadata to an XML file from your IdP. There is usually a button to download this from your IdP dashboard. Additionally, you can often find this by navigating to `https://your.identityprovider.com/federationmetadata/2007-06/federationmetadata.xml`.`

Copy the entire XML file to your clipboard and log in to Retool as an admin user.

- **Self-hosted Retool**: Go to **Settings > Advanced**.
- **Retool Cloud**: Go to **Settings > Single Sign-On (SSO)**, select **SAML SSO**, and paste the XML file contents to the **Identity Provider Metadata** field.

## 4. Test the connection

---

## Configure Auth0 OIDC SSO

import Test from "./_partials/_test-connection.mdx"

Use this guide to configure Auth0 SSO with OpenID Connect (OIDC) on Retool. Once configured, users can log in to Retool with their Auth0 credentials.

## Requirements

To configure Auth0 SSO, you must:

- Have admin permissions on Retool Cloud or permissions to add environment variables on self-hosted Retool instances.
- Have permissions to create an OIDC application in Auth0.

## 1. Create a new Auth0 application

In Auth0, create a new application for Retool. Go to your application **Settings** and save the **Client ID** and **Client secret**.

Next, save the **OAuth Authorization URL** and **OAuth Token URL**. In Auth0, this is found in **Settings > Advanced Settings > Endpoints**.

Add `https://.retool.com/oauth2sso/callback` as the callback URL, replacing `` with your subdomain. In Auth0, the callback URL is set in **Settings > Application URIs**.

## 2. Configure settings in Retool

Configure your SSO settings in Retool.

import SettingsOverride from "../_settings-override.mdx";
import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

On Retool Cloud and self-hosted Retool versions 3.16 and later, enter settings on **Settings > Single Sign-On (SSO)**.

| Setting        | Example                                |
| -------------- | -------------------------------------- |
| Client ID      | yypLZ44LxEz0XlQZBu5k2Nq9XsdOv4f5       |
| Client secret  | xxxxxxxxxxxxxxxxxxxxxxxxxxxxx          |
| Scopes         | openid email profile offline_access    |
| Auth URL       | `https://retool.auth0.com/authorize`   |
| Token URL      | `https://retool.auth0.com/oauth/token` |
| Email key      | idToken.email                          |
| First name key | idToken.given_name                     |
| Last name key  | idToken.family_name                    |

On self-hosted Retool versions earlier than 3.16, configure the following [environment variables](../../self-hosted/reference/environment-variables/index.mdx#authentication) on your Retool instance.

| Environment variable                   | Example                                |
| -------------------------------------- | -------------------------------------- |
| `CUSTOM_OAUTH2_SSO_CLIENT_ID`          | `yypLZ44LxEz0XlQZBu5k2Nq9XsdOv4f5`     |
| `CUSTOM_OAUTH2_SSO_CLIENT_SECRET`      | `xxxxxxxxxxxxxxxxxxxxxxxxxxxxx`        |
| `CUSTOM_OAUTH2_SSO_SCOPES`             | `openid email profile offline_access`  |
| `CUSTOM_OAUTH2_SSO_AUTH_URL`           | `https://retool.auth0.com/authorize`   |
| `CUSTOM_OAUTH2_SSO_TOKEN_URL`          | `https://retool.auth0.com/oauth/token` |
| `CUSTOM_OAUTH2_SSO_JWT_EMAIL_KEY`      | `idToken.email`                        |
| `CUSTOM_OAUTH2_SSO_JWT_FIRST_NAME_KEY` | `idToken.given_name`                   |
| `CUSTOM_OAUTH2_SSO_JWT_LAST_NAME_KEY`  | `idToken.family_name`                  |

### Optional settings

To provide authorization to access resources when a user logs in with SSO, specify the API audience that corresponds to the resource as configured in Auth0. Find the API audience in the Auth0 UI under **Applications > APIs**. Set this value as the `CUSTOM_OAUTH2_SSO_AUDIENCE` environment variable in your Retool deployment, or in the **SSO Audience** field in the Retool UI.

:::warning

If you don't configure the `CUSTOM_OAUTH2_SSO_AUDIENCE` setting, Retool receives an opaque token, and you won't be able to use the `accessToken` to control access to components and resources.

:::

## 3. Test the connection

If you added environment variables, restart your Retool instance.

---

## Configure SSO with OIDC authentication

import Test from "../_partials/_test-connection.mdx"

Retool's OpenID integration uses the [Authorization Code Flow](https://auth0.com/docs/flows/authorization-code-flow). Retool, at minimum, expects either an ID token or access token to be a JSON Web Token (JWT) that contains the email of the authenticated user.

## Requirements

To configure SSO with OIDC, you need:

- The OAuth client ID for your application
- The OAuth client secret for your application
- A list of scopes to grant to Retool
- The authorization endpoint for your OpenID provider
- The token endpoint for your OpenID provider

You should confirm how your SSO provider formats its ID and access tokens. Retool attempts to decode these tokens as JWTs. You must provide Retool the path in the decoded JWT that corresponds with your identifying user information.

:::note

Some SSO providers don't return both an ID token and access token by default. You might need to provide additional metadata to obtain both tokens. See the provider-specific guides and generic process to [obtain a fat token](#thin-tokens-and-fat-tokens) for more details.

:::

### Required settings

Configure your SSO settings in Retool.

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";
import SettingsOverride from "../../_settings-override.mdx";

On Retool Cloud and self-hosted Retool versions 3.16 and later, enter settings on **Settings > Single Sign-On (SSO)**.

| Setting       | Example value                                            |
| ------------- | -------------------------------------------------------- |
| Client ID     | `XXXXXXXXXX`                                             |
| Client secret | `XXXXXXXXXX`                                             |
| Scopes        | `openid email offline_access profile`                    |
| Auth URL      | `https://yourcompany.idprovider.com/oauth2/v1/authorize` |
| Token URL     | `https://yourcompany.idprovider.com/oauth2/v1/token`     |
| Email key     | `idToken.email`                                          |

On self-hosted Retool versions earlier than 3.16, configure the following [environment variables](../../../self-hosted/reference/environment-variables/index.mdx#authentication) on your Retool instance.

| Environment variable                   | Example                                |
| -------------------------------------- | -------------------------------------- |
| `CUSTOM_OAUTH2_SSO_CLIENT_ID`          | yypLZ44LxEz0XlQZBu5k2Nq9XsdOv4f5       |
| `CUSTOM_OAUTH2_SSO_CLIENT_SECRET`      | xxxxxxxxxxxxxxxxxxxxxxxxxxxxx          |
| `CUSTOM_OAUTH2_SSO_SCOPES`             | openid email profile offline_access    |
| `CUSTOM_OAUTH2_SSO_AUTH_URL`           | `https://retool.auth0.com/authorize`   |
| `CUSTOM_OAUTH2_SSO_TOKEN_URL`          | `https://retool.auth0.com/oauth/token` |
| `CUSTOM_OAUTH2_SSO_JWT_EMAIL_KEY`      | idToken.email                          |
| `CUSTOM_OAUTH2_SSO_JWT_FIRST_NAME_KEY` | idToken.given_name                     |
| `CUSTOM_OAUTH2_SSO_JWT_LAST_NAME_KEY`  | idToken.family_name                    |

On self-hosted deployments, you should also set the `BASE_DOMAIN` environment variable to ensure links using your domain—for example, new user invitations and forgotten password resets—are correct. Retool's backend tries to determine the `BASE_DOMAIN` if it is not set, but it can be incorrect if your website uses a proxy.

```
BASE_DOMAIN=https://retool.yourcompany.com
```

## Configure SSO for your provider

Follow the configuration guides to set up SSO with OIDC for [Auth0](../auth0.mdx), [Google](../google/oidc.mdx), [Okta](../okta/oidc.mdx), or [Microsoft Entra ID](../microsoft-entra-id/oidc.mdx).

## Role mapping

import GroupsAdminWarning from "../../_groups_admin_warning.mdx";

You can also map the roles returned from the OpenID response to your Retool permission groups. Any Retool groups that are not specified in the role mapping are overwritten. Groups are created automatically in Retool when a user logs in.

The following example maps the `devops` and `support` roles to specific Retool permission groups.

| Setting      | Environment variable              | Example value                        |
| ------------ | --------------------------------- | ------------------------------------ |
| Roles key    | `CUSTOM_OAUTH2_SSO_JWT_ROLES_KEY` | `idToken.groups`                      |
| Role mapping | `CUSTOM_OAUTH2_SSO_ROLE_MAPPING`  | `devops -> admin, support -> viewer` |

After setting these environment variables, a user's groups sync the next time they log in to Retool over SSO.

import RoleMappingCaseSensitivity from "../../_role-mapping-case-sensitivity.mdx";

To disable role mapping, set `CUSTOM_OAUTH2_SSO_ROLE_MAPPING_DISABLED` to `true`.

### Examples

| Roles                 | Role mapping setting | Result                                                                                                                            |
| --------------------- | -------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| ["admin", "editor"]   | ""                   | The user is added to the "admin", "editor", and "All Users" groups.                                                               |
| \[]                   | ""                   | The user is added to the "All Users" group.                                                                                       |
| ["admin", "support"]  | ""                   | A new custom group called "support" is created. The user is added to the "admin", "support", and "All Users" groups.  |
| ["support", "devops"] | "devops -> editor"   | A new custom group called "support" is created. The user is added to the "editor", "support", and "All Users" groups. |

## Use authentication JWTs in resources

Using SSO with OIDC, you can reuse the tokens obtained throughout the SSO process in API calls you make from Retool to your backend services.

In the following example, `USER_OAUTH2_ACCESS_TOKEN` is replaced with the access token obtained in the auth flow, and `USER_OAUTH2_ID_TOKEN` is replaced with the ID token obtained in the auth flow.

:::note

If your OpenID Provider returned a `refresh token` in the initial login flow, Retool automatically uses it to refresh the access and ID tokens every two hours by default. You can set a custom refresh time using the `CUSTOM_OAUTH2_SSO_ACCESS_TOKEN_LIFESPAN_MINUTES` environment variable.

:::

## Reference JWT claims in Retool apps

You can use JWT claims returned by your SSO provider to personalize Retool apps or [control component permissions](../../../permissions/guides.mdx#hide-or-disable-components-for-users-and-groups). Retool automatically includes these claims as the values of the `current_user.metadata.idToken` and `current_user.metadata.accessToken` keys. Access them using curly braces anywhere in your Retool app: `{{current_user.metadata.idToken.picture}}`

### Custom claims

You can add custom claims following instructions from your IdP. See the guides to your IdP for instructions on where to add these claims. In Retool, custom claims are accessible in `current_user.metadata.`

## Thin tokens and fat tokens

Some OIDC identity providers, such as Okta, don't send all the claims associated with a user during the authentication flow. This is called a [thin token](https://developer.okta.com/docs/concepts/api-access-management/#tokens-and-scopes) and is used in place of a fat token, which returns all claims, for performance reasons. If your ID token is missing certain claims, you can tell Retool to make an additional request to a given endpoint for the fat token.

You can configure this endpoint in **Settings > Custom SSO > Fat token URL** on Retool Cloud, or using the `CUSTOM_OAUTH2_SSO_USERINFO_URL` environment variable on self-hosted deployments. For example, Okta uses the `/userinfo` endpoint.

```
CUSTOM_OAUTH2_SSO_USERINFO_URL=https://your-company.okta.com/oauth2/v1/userinfo
```

During the authentication flow, Retool replaces the thin token with the fat token returned from this endpoint.

## Test the connection

---

## Configure SSO with SAML authentication

import Test from "../_partials/_test-connection.mdx"

Retool Cloud and Self-hosted Retool deployments support [Okta](../okta/saml.mdx), [Microsoft Entra ID](../microsoft-entra-id/saml.mdx), [Active Directory Federation Services](../adfs.mdx), and other SAML SSO providers.

If you don't use Okta or Active Directory, use the following steps to configure your SAML identity provider service.

## 1. Set your Entity ID in Retool

By default, Retool uses the Entity ID `https://tryretool.com`.

Add the following [environment variable](../../../self-hosted/reference/environment-variables/index.mdx#domains) to your `docker.env` file, replacing `retool.yourcompany.com` with your domain. Note: adding a new environment variable requires restarting the container for it to take effect.

```
DOMAINS=retool.yourcompany.com
```

## 2. Configure your Identity Provider

You should reference the provided documentation from your identity provider to complete its setup. However, you will likely be asked to supply values for the **Sign on URL** and **Reply URL** fields. Use the following pattern, replacing `example.retool.com` with the **Entity ID** you supplied in step 1:

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

- Sign on URL: `https://example.retool.com/api/saml/login`
- Reply URL: `https://example.retool.com/api/saml/login`

- Sign on URL: `https://example.retool.com/saml/login`
- Reply URL: `https://example.retool.com/saml/login`

## 3. Match user attributes and claims

Retool requires exactly the following attributes to be asserted for each user on login:

- `email`: The identifier for a user
- `firstName`: The user's first name
- `lastName`: The user's last name

## 4. Assign users access to Retool

Use your identity provider to assign users to have access to login to Retool.

## 5. Configure Retool with the Identity Provider Metadata

Export the metadata to an XML file from your identity provider and copy it. There's usually a button to trigger a download from your IdP dashboard. Additionally, you can often find this data by navigating to `https://your.identityprovider.com/federationmetadata/2007-06/federationmetadata.xml`.

You can configure Retool with the IdP metadata in the dashboard for Retool Cloud, or with the [SAML IDP METADATA](https://docs.retool.com/sso/reference/environment-variables.mdx#saml_idp_metadata) environment variable on self-hosted deployments. To use the dashboard, log in to Retool as an admin user.

On Retool Cloud, go to **Settings > Single Sign-On (SSO)**, select **SAML SSO**, and paste the XML file contents to the **Identity Provider Metadata** field. On self-hosted deployments, this setting is on **Settings > Advanced**.

## 6. Test the connection

---

## Configure Sign in with Google

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

You can configure SSO using the [Sign in with Google](https://developers.google.com/identity/gsi/web/guides/overview) following this guide. This guide applies to Retool Cloud and self-hosted deployments.

:::note

Follow this guide to configure Sign-in with Google. Refer to the [Google SSO with OpenID Connect guide](oidc.mdx) to configure Google SSO using OIDC.

:::

## 1. Configure Google OAuth client

Navigate to the **Credentials** page of your [Google APIs console](https://console.developers.google.com/apis). Create or select your project.

Click **Create Credentials > OAuth client ID** to create a [Google OAuth client](https://developers.google.com/identity/gsi/web/guides/get-google-api-clientid). Use the following settings if you have a custom domain set on your organization.

| Setting name                  | Setting values                                                                                                                                                                                   |
| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Application type              | Web application                                                                                                                                                                                  |
| Authorized JavaScript origins | `https://.com`                                                                                                                                                                 |
| Authorized redirect URIs      | `https://.com/oauthcallback``https://.com/oauth``https://.com/oauthcallback/mobile` (for Retool Mobile only) |

If you are using Retool Cloud, you must also set the following authorized redirect URIs to enable Sign-in with Google:

- `https://login.retool.com/oauthcallback`
- `https://login.retool.com/oauthcallback/cli`
- `https://login.retool.com/oauth/oauthcallback`

If you are prompted to configure a consent screen, select **Internal** as the **User type**.

For more information on setup needed in the Google APIs console, refer to [Google's setup guide](https://developers.google.com/identity/gsi/web/guides/get-google-api-clientid).

## 2. Update settings in Retool

Click on your new OAuth application in the **Credentials** tab, and copy the **Client ID** and **Client secret**.

Visit **Settings > Custom SSO**, select Google SSO, and add your Client ID and Client Secret.

In your `docker.env` file, set your Client ID and Client Secret as the values of the `CLIENT_ID` and `CLIENT_SECRET` environment variables. Set the [BASE_DOMAIN](../../../self-hosted/reference/environment-variables/index.mdx#variable-BASE_DOMAIN) environment variable as well, so Google handles redirect requests correctly.

If you use [Kubernetes Secrets](https://kubernetes.io/docs/concepts/configuration/secret/), place the Base64-encoded version of these strings inside your Kubernetes secrets file instead of in `docker.env`.

```
CLIENT_ID={YOUR_GOOGLE_CLIENT_ID}
CLIENT_SECRET={YOUR_GOOGLE_CLIENT_SECRET}
```

To automatically provision users on sign-in, set the `DEFAULT_GROUP_FOR_DOMAINS` [environment variable](../../../self-hosted/reference/environment-variables/index.mdx#variable-DEFAULT_GROUP_FOR_DOMAINS).

```
DEFAULT_GROUP_FOR_DOMAINS=example1.org -> admin, example2.com -> viewer
```

If you want to restrict sign-in to SSO and remove the option for users to sign in with a username and password, add the `RESTRICTED_DOMAIN` environment variable.

```
RESTRICTED_DOMAIN=yourcompany.com
```

When setting `RESTRICTED_DOMAIN`, do not include the protocol or subdomain where you have Retool deployed. Only use `yourcompany.com`, which should match the email address users use to sign in. See the [environment variables](../../../self-hosted/reference/environment-variables/index.mdx#authentication) reference for more details.

Restart your Retool instance to reload the configuration and enable Google SSO. If you use Docker, you can restart the instance with `docker compose up`.

```
sudo docker compose up -d
```

---

## Configure Google OIDC SSO

:::note

Follow this guide to configure Google SSO using OpenID Connect (OIDC). Refer to the [Sign in with Google](google-sign-in.mdx) to configure SSO using Sign in with Google.

:::

## Requirements

To configure Google OIDC SSO, you must:

- Have admin permissions on Retool Cloud or permissions to add environment variables on self-hosted Retool instances.
- Have permissions to create a Google OAuth Client.

## 1. Create a Google OAuth Client ID

Go to your [Google Developer Console](https://developers.google.com/identity/gsi/web/guides/get-google-api-clientid) and create an **OAuth client ID**.

If you are asked to configure an OAuth consent screen, select **Internal**. Configure the app as a **Web application** and enter `https://retool.your-company.com/oauth2sso/callback` under **Authorized redirect URIs > URIs**.

Save your **Client ID** and **Client secret**.

## 2. Configure settings in Retool

:::note

Google requires the URL parameters `access_type=offline` and `prompt=consent` to obtain refresh tokens, so you should include these in your Auth URL variable.

:::

Configure SSO settings in Retool.

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";
import SettingsOverride from "../../_settings-override.mdx";

On Retool Cloud and self-hosted Retool versions 3.16 and later, enter settings on **Settings > Single Sign-On (SSO)**.

| Setting        | Example                                                                           |
| -------------- | --------------------------------------------------------------------------------- |
| Client ID      | 22222222222-dq62o6pidgmgrem34fb07klc8qa1308t.apps.googleusercontent.com           |
| Client secret  | xxxxxxxxxxxxxxxxxxxxxxxxxxxx                                                      |
| Scopes         | openid email profile https://www.googleapis.com/auth/userinfo.profile             |
| Auth URL       | `https://accounts.google.com/o/oauth2/v2/auth?access_type=offline&prompt=consent` |
| Token URL      | `https://oauth2.googleapis.com/token`                                             |
| Email key      | idToken.email                                                                     |
| First name key | idToken.given_name                                                                |
| Last name key  | idToken.family_name                                                               |

On self-hosted Retool versions earlier than 3.16, configure the following [environment variables](../../../self-hosted/reference/environment-variables/index.mdx#authentication).

| Environment variable                   | Example                                                                           |
| -------------------------------------- | --------------------------------------------------------------------------------- |
| `CUSTOM_OAUTH2_SSO_CLIENT_ID`          | `22222222222-dq62o6pidgmgrem34fb07klc8qa1308t.apps.googleusercontent.com`         |
| `CUSTOM_OAUTH2_SSO_CLIENT_SECRET`      | `xxxxxxxxxxxxxxxxxxxxxxxxxxxx`                                                    |
| `CUSTOM_OAUTH2_SSO_SCOPES`             | `openid email profile https://www.googleapis.com/auth/userinfo.profile`           |
| `CUSTOM_OAUTH2_SSO_AUTH_URL`           | `https://accounts.google.com/o/oauth2/v2/auth?access_type=offline&prompt=consent` |
| `CUSTOM_OAUTH2_SSO_TOKEN_URL`          | `https://oauth2.googleapis.com/token`                                             |
| `CUSTOM_OAUTH2_SSO_JWT_EMAIL_KEY`      | `idToken.email`                                                                   |
| `CUSTOM_OAUTH2_SSO_JWT_FIRST_NAME_KEY` | `idToken.given_name`                                                              |
| `CUSTOM_OAUTH2_SSO_JWT_LAST_NAME_KEY`  | `idToken.family_name`                                                             |

You can also configure an access token refresh time. Google's tokens expire after one hour. By default, Retool's integration refreshes tokens older than two hours. This means it's recommended to set `CUSTOM_OAUTH2_SSO_ACCESS_TOKEN_LIFESPAN_MINUTES` to refresh the tokens more frequently.

```
CUSTOM_OAUTH2_SSO_ACCESS_TOKEN_LIFESPAN_MINUTES=45
```

---

## Configure Microsoft Entra ID OIDC SSO

import Test from "../_partials/_test-connection.mdx"

To configure SSO with Microsoft Entra ID OIDC, you must:

- Have permission to create an Microsoft Entra ID Enterprise application.
- Have admin permissions on your Retool instance. For self-hosted deployments, you must also have the ability to configure environment variables.

## 1. Create an Microsoft Entra ID Enterprise application

To create an Microsoft Entra ID Enterprise application, follow the steps in [Azure's documentation](https://learn.microsoft.com/en-us/azure/active-directory/manage-apps/add-application-portal-setup-oidc-sso).

1. In the **Microsoft Entra ID portal**, add a new **Enterprise application**.

2. Retool is not listed in the Microsoft Entra ID Gallery, so select **Create your own application**.

3. Name the application.

4. Select **Register an application to integrate with Microsoft Entra ID (App you're developing)**.

5. Under **Supported account types**, select **Accounts in this organizational directory only (Default Directory Only - Single tenant)**.

6. Under **Redirect URI**, select **Web**. Enter `https://retool.yourcompany.com/oauth2sso/callback` under the path, replacing `retool.yourcompany.com` with your Retool instance domain. This specifies the path where Microsoft Entra ID redirects users after they complete authentication.

## 2. Configure secrets

1. In the settings for the new Retool enterprise application, select the **Single sign-on** menu. Select the **App registrations experience**.

2. Select the **Certifications & secrets** menu. Add a new client secret and set an expiration period. You must update your Retool deployment when the secret expires, so you should set the maximum allowable period to **24 months**.

3. Save this secret for use in a later step.

## 3. Configure claims

1. In the **Azure app registration experience**, select the **Token configuration** menu.

2. Select **Add optional claim** for the **ID token**. At a minimum, add the following claims:

- `acct`
- `email`
- `family_name`
- `given_name`

3. When you save the claims, **turn on the Microsoft Graph email, profile permissions**.

4. Optionally, specify additional claims to include for the **Access token**.

## 4. Configure optional group claims

You can optionally map Microsoft Entra ID groups to Retool groups to automatically assign users to groups when they authenticate using SSO. This requires adding group claims to the **ID token**.

1. In the **Azure app registration experience**, select the **Token configuration** menu.

2. Select **Add optional claim** for the **ID token**.

- In the claim, include the groups you want to map to Retool groups.
- Include the **Group ID** for **ID**, **Access**, and **SAML**.

## 5. Retrieve connection details

1. In the **Azure app registration experience**, select the **Overview** menu and select **Endpoints**.

2. Save the following fields:

- Application (client) ID
- OAuth 2.0 authorization endpoint (v2)
- OAuth 2.0 token endpoint (v2)

## 6. Configure settings in Retool

Configure your Microsoft Entra ID settings in Retool.

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";
import SettingsOverride from "../../_settings-override.mdx";

On Retool Cloud and self-hosted Retool versions 3.16 and later, enter settings on **Settings > Single Sign-On (SSO)**.

| Setting                       | Example                                                            |
| ----------------------------- | ------------------------------------------------------------------ |
| Client ID                     | `CLIENT_ID`                                                        |
| Client secret                 | `CLIENT_SECRET`                                                    |
| Scopes                        | `openid profile email offline_access`                              |
| Auth URL                      | `https://login.microsoftonline.com//oauth2/v2.0/authorize` |
| Token URL                     | `https://login.microsoftonline.com//oauth2/v2.0/token`     |
| Email key                     | `idToken.email`                                                    |
| User info URL (Fat token URL) | `https://yourcompany.idprovider.com/oauth2/v1/userinfo`            |

:::caution

For Microsoft Entra ID OIDC, leave **User info URL (Fat token URL)** unset.

:::

On self-hosted Retool versions earlier than 3.16, add the following [environment variables](../../../self-hosted/reference/environment-variables/index.mdx#authentication) to your Retool instance. Be sure to include the `BASE_DOMAIN` environment variable so links using your domain are correct. Retool's backend tries to determine the `BASE_DOMAIN` if it is not set, but it can be incorrect if your website uses a proxy.

| Environment variable              | Example                                                            |
| --------------------------------- | ------------------------------------------------------------------ |
| `CUSTOM_OAUTH2_SSO_CLIENT_ID`     | `CLIENT_ID`                                                        |
| `CUSTOM_OAUTH2_SSO_CLIENT_SECRET` | `CLIENT_SECRET`                                                    |
| `CUSTOM_OAUTH2_SSO_SCOPES`        | `openid profile email offline_access`                              |
| `CUSTOM_OAUTH2_SSO_AUTH_URL`      | `https://login.microsoftonline.com//oauth2/v2.0/authorize` |
| `CUSTOM_OAUTH2_SSO_TOKEN_URL`     | `https://login.microsoftonline.com//oauth2/v2.0/token`     |
| `CUSTOM_OAUTH2_SSO_JWT_EMAIL_KEY` | `idToken.email`                                                    |
| `CUSTOM_OAUTH2_SSO_USERINFO_URL`  | `https://yourcompany.idprovider.com/oauth2/v1/userinfo`            |
| `BASE_DOMAIN`                     | `https://retool.yourcompany.com`                                   |

See [thin tokens and fat tokens](../custom/oidc.mdx#thin-tokens-and-fat-tokens) for more detail on the **User Info URL** or `CUSTOM_OAUTH2_SSO_USERINFO_URL` [environment variable](../../reference/environment-variables.mdx#custom_oauth2_sso_userinfo_url).

### Optional settings

To pass the user's first name and last name to Retool, set the following settings.

| Setting        | Example               |
| -------------- | --------------------- |
| First name key | `idToken.given_name`  |
| Last name key  | `idToken.family_name` |

import GroupsAdminWarning from "../../_groups_admin_warning.mdx";

If you configured group claims, construct a [role mapping string](../custom/oidc.mdx#role-mapping) to map Microsoft Entra ID group object IDs to Retool group names. Find Microsoft Entra ID group object IDs in the **Azure Groups application**.

For example, given an Microsoft Entra ID group called `Retool Editors` with an object ID of `fd951-f454-4b7a`, use the mapping string `fd951-f454-4b7a -> editor` to assign its members to the Editor group in Retool.

To add role mapping, set the following environment variables in your Retool instance.

| Setting      | Example                     |
| ------------ | --------------------------- |
| Roles key    | `idToken.groups`             |
| Role mapping | `fd951-f454-4b7a -> editor` |

| Environment variable                   | Example               |
| -------------------------------------- | --------------------- |
| `CUSTOM_OAUTH2_SSO_JWT_FIRST_NAME_KEY` | `idToken.given_name`  |
| `CUSTOM_OAUTH2_SSO_JWT_LAST_NAME_KEY`  | `idToken.family_name` |

If you configured group claims, construct a [role mapping string](../custom/oidc.mdx#role-mapping) to map Microsoft Entra ID group object IDs to Retool group names. Find Microsoft Entra ID group object IDs in the **Azure Groups application**.

For example, given an Microsoft Entra ID group called `Retool Editors` with an object ID of `fd951-f454-4b7a`, use the mapping string `fd951-f454-4b7a -> editor` to assign its members to the Editor group in Retool.

To add role mapping, set the following environment variables in your Retool instance.

| Environment variable              | Example                     |
| --------------------------------- | --------------------------- |
| `CUSTOM_OAUTH2_SSO_JWT_ROLES_KEY` | `idToken.groups`            |
| `CUSTOM_OAUTH2_SSO_ROLE_MAPPING`  | `fd951-f454-4b7a -> editor` |

## 7. Test the connection

---

## Configure Microsoft Entra ID SAML SSO

import Test from "../_partials/_test-connection.mdx"

Follow these steps to configure SAML SSO with Microsoft Entra ID for your Retool instance.

## 1. Set your Entity ID in Retool

By default, Retool uses the Entity ID `https://tryretool.com`.

Add the following [environment variable](../../../self-hosted/reference/environment-variables/index.mdx#domains) to your `docker.env` file, replacing `yourcompany.retool.com` with your domain. Note: adding a new environment variable requires restarting the container for it to take effect.

```
DOMAINS=yourcompany.retool.com
```

## 2. Create an Microsoft Entra ID Enterprise application

In the **Microsoft Entra ID admin center**, add a new **Enterprise application**. Retool is not listed in the Microsoft Entra ID Gallery, so you must select **Create your own application**.

Name the application “Retool” and select **Integrate any other application you don’t find in the gallery (Non-gallery)**.

## 3. Assign users to the Retool application in Azure

For users to access Retool using Microsoft Entra ID SSO, they must:

- Be assigned to the application
- Have a First Name, Last Name, User Principal Name, and Email defined on their profile

Assign users to the Retool application and confirm their required attributes in the **Microsoft Entra ID admin center**.

## 4. Configure SAML settings in Azure

In the **Microsoft Entra ID admin center**, select the **Retool** Enterprise application. Set up single sign on for the Retool application, selecting **SAML** as the sign-on method. Use the following SAML settings, replacing `yourcompany.com` with your domain. Leave **Relay state** and **Logout URL** blank.

| Setting                                    | Value                                                                                                                                  |
| ------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------- |
| Identifier (Entity ID)                     | `yourcompany.retool.com`                                                                                                               |
| Reply URL (Assertion Consumer Service URL) | On Retool Cloud, `https://your-company.retool.com/api/saml/login`. On self-hosted Retool, `https://retool.your-company.com/saml/login`. |
| Sign on URL                                | On Retool Cloud, `https://your-company.retool.com/api/saml/login`. On self-hosted Retool, `https://retool.your-company.com/saml/login`. |

Set the following attributes and claims.

| Setting                          | Value                    |
| -------------------------------- | ------------------------ |
| Unique User Identifier (Name ID) | `user.mail`              |
| `firstName`                      | `user.givenname`         |
| `lastName`                       | `user.surname`           |
| `email`                          | `user.userprincipalname` |

You must also edit each claim and clear the value for the **Namespace** field.

## 5. Import Azure Federation Metadata into Retool

On the same page you configured SAML settings in the **Microsoft Entra ID admin center**, download the **Federation Metadata** XML file (listed under the SAML Signing Certificate).

Open the XML file in a code editor and copy the contents to your clipboard. Once complete:

1. Navigate to **Settings > Single Sign-On (SSO)**.
2. Select **SAML SSO**.
3. Paste the XML file contents to the **Identity Provider Metadata** field.

## 6. Test the connection

---

## Configure Okta OIDC SSO

import Test from "../_partials/_test-connection.mdx"

To configure Okta OIDC SSO, you need:

- An Okta account with permissions to create an OIDC integration.
- The ability to set environment variables on self-hosted Retool instance, or admin permissions for Retool Cloud.

## 1. Create a new app integration

Follow the steps in Okta to [create a new OIDC integration](https://help.okta.com/en-us/Content/Topics/Apps/Apps_App_Integration_Wizard_OIDC.htm). Use the following settings, replacing `YOUR_RETOOL_DOMAIN` with your Retool instance.

| Setting                | Value                                           |
| ---------------------- | ----------------------------------------------- |
| Sign-on method         | OIDC - OpenID Connect                           |
| Application type       | Web Application                                 |
| Sign-in redirect URIs  | `https://YOUR_RETOOL_DOMAIN/oauth2sso/callback` |
| Sign-out redirect URIs | `https://YOUR_RETOOL_DOMAIN/api/logout`         |

To add a Retool tile in Okta, set the `TRIGGER_OAUTH_2_SSO_LOGIN_AUTOMATICALLY` environment variable to `true` or enable **Trigger login automatically** on **Settings > Single Sign-On (SSO)**. In Okta, use the following settings:

| Setting            | Value                                              |
| ------------------ | -------------------------------------------------- |
| Initiate login URI | `YOUR_RETOOL_DOMAIN`                               |
| Login initiated by | Either Okta or App                                 |
| Login flow         | Redirect to app to initiate login (OIDC Compliant) |

## 2. Configure settings in Retool

Configure your SSO settings in Retool.

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

import SettingsOverride from "../../_settings-override.mdx";

On Retool Cloud and self-hosted Retool versions 3.16 and later, enter settings on **Settings > Single Sign-On (SSO)**. Depending on your setup, you might not need to configure all of these values. If you aren't sure where to find token and auth URLs, try using the [/.well-known/openid-configuration](https://developer.okta.com/docs/reference/api/oidc/#well-known-openid-configuration) endpoint.

| Setting        | Example                                                                  |
| -------------- | ------------------------------------------------------------------------ |
| Client ID      | `CLIENT_ID`                                                              |
| Client secret  | `CLIENT_SECRET`                                                          |
| Scopes         | `openid email offline_access profile`                                    |
| Fat token URL  | `https://yourcompany.okta.com/oauth2/YOUR_AUTH_SERVER_NAME/v1/userinfo`  |
| Auth URL       | `https://yourcompany.okta.com/oauth2/YOUR_AUTH_SERVER_NAME/v1/authorize` |
| Token URL      | `https://yourcompany.okta.com/oauth2/YOUR_AUTH_SERVER_NAME/v1/token`     |
| Email key      | `idToken.email`                                                          |
| First name key | `idToken.given_name`                                                     |
| Last name key  | `idToken.family_name`                                                    |

After you save your settings, you can log in using Okta SSO. You can test out your configuration by logging in from an incognito window.

On self-hosted Retool versions earlier than 3.16, configure the following [environment variables](../../../self-hosted/reference/environment-variables/index.mdx#authentication) on your Retool instance.

| Environment variable                   | Example                                                                  |
| -------------------------------------- | ------------------------------------------------------------------------ |
| `CUSTOM_OAUTH2_SSO_CLIENT_ID`          | `CLIENT_ID`                                                              |
| `CUSTOM_OAUTH2_SSO_CLIENT_SECRET`      | `CLIENT_SECRET`                                                          |
| `CUSTOM_OAUTH2_SSO_SCOPES`             | `openid email offline_access profile`                                    |
| `CUSTOM_OAUTH2_SSO_USERINFO_URL`       | `https://yourcompany.okta.com/oauth2/YOUR_AUTH_SERVER_NAME/v1/userinfo`  |
| `CUSTOM_OAUTH2_SSO_AUTH_URL`           | `https://yourcompany.okta.com/oauth2/YOUR_AUTH_SERVER_NAME/v1/authorize` |
| `CUSTOM_OAUTH2_SSO_TOKEN_URL`          | `https://yourcompany.okta.com/oauth2/YOUR_AUTH_SERVER_NAME/v1/token`     |
| `CUSTOM_OAUTH2_SSO_JWT_EMAIL_KEY`      | `idToken.email`                                                          |
| `CUSTOM_OAUTH2_SSO_JWT_FIRST_NAME_KEY` | `idToken.given_name`                                                     |
| `CUSTOM_OAUTH2_SSO_JWT_LAST_NAME_KEY`  | `idToken.family_name`                                                    |

You can also configure an access token refresh time using the `CUSTOM_OAUTH2_SSO_ACCESS_TOKEN_LIFESPAN_MINUTES` environment variable.

You should also set the `BASE_DOMAIN` environment variable to ensure links using your domain are correct. Retool's backend tries to determine the `BASE_DOMAIN` if it is not set, but it can be incorrect if your website uses a proxy.

```
BASE_DOMAIN=https://retool.yourcompany.com
```

After you restart your Retool instance, you can log in using Okta SSO. You can test out your configuration by logging in from an incognito window.

## Role mapping with Okta Group Claims

import GroupsAdminWarning from "../../_groups_admin_warning.mdx";

Use the following instructions to automatically map your Okta groups with Retool permission groups when a user logs in.

### 1. Create a new scope

Follow the instructions in Okta to create a new [API access scope](https://help.okta.com/en-us/Content/Topics/Security/api-config-scopes.htm) for your authorization server.

### 2. Create a new claim

Follow the instructions in Okta to [create a new claim](https://help.okta.com/en-us/Content/Topics/Security/api-config-claims.htm) and name it `groups`.

In the **Add claim** form, select **ID Token** and **Always** in the dropdown. In the **Value type** section, select **Groups**.

You can add an optional **Filter** to limit the groups to sync.

In the **Scopes** section, add in the API access scope you previously created.

#### Custom claims

To send custom Okta attributes—for example, employee number or department—to Retool, you can configure [custom claims in Okta](https://support.okta.com/help/s/article/How-to-add-custom-attributes-of-user-profile-as-claims-in-token).

1. Add attributes to the Retool profile from **Profile editor** > **Add attributes**, and from **Profile editor** > **Mappings**, map them to the correct fields.
2. Add claims to the authorization server. You may want to limit the scope of these claims.
3. To confirm your custom attributes were correctly added, you can preview a sample token from **Security** > **API** > **Default** > **Preview** in Okta.
4. When you next log in to Retool using SSO, you can view attributes in `current_user.metadata.userInfoResponse`.

### 3. Add additional settings

On Retool Cloud and self-hosted Retool versions 3.16 and later, enter settings on **Settings > Single Sign-on (SSO)**. Add the `groups` scope to the **Scopes** fields.

Specify the roles key—for example, `idToken.groups` —in the **Roles key** field.

Specify any additional remapping in the **Role mapping** field. For example, `Retool devops->admin` maps members of the "Retool devops" group to Retool admins.

On self-hosted Retool versions earlier than 3.16, use [environment variables](../../../self-hosted/reference/environment-variables/index.mdx) to configure additional settings.

Add the `groups` scope to the `CUSTOM_OAUTH2_SSO_SCOPES` environment variable.

```
CUSTOM_OAUTH2_SSO_SCOPES=openid email profile offline_access groups
```

Specify that the groups can be read in the `idToken`.

```
CUSTOM_OAUTH2_SSO_JWT_ROLES_KEY=idToken.groups
```

Specify any additional remapping. The following example maps members of the "Retool devops" group to Retool admins.

```
CUSTOM_OAUTH2_SSO_ROLE_MAPPING=Retool devops -> admin
```

import RoleMappingCaseSensitivity from "../../_role-mapping-case-sensitivity.mdx";

You can also now use `USER_OAUTH2_ACCESS_TOKEN` and `USER_OAUTH2_ID_TOKEN` as [authentication in resources](../custom/oidc.mdx#use-authentication-jwts-in-resources).

## 3. Test the connection

---

## Configure Okta SAML SSO

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";
import Test from "../_partials/_test-connection.mdx"

To configure Okta SAML SSO, you must:

- Be in **Admin** mode in Okta.
- Have group names that match exactly between Okta and SAML.
- Have admin permissions in Retool.
- For organizations on Retool Cloud, the ability to create a custom SAML application.

## Configuration

1. In your Okta admin dashboard, click **Add Application**.
2. Search for `Retool` and follow the wizard.
3. Navigate to the Okta application you created. Click on the **Sign On** tab, then **Actions > View IdP Metadata** in the **SAML Signing Certificates** section.
4. Save the page as an XML file. Consult Okta's documentation to confirm how to [view the IdP metadata](https://support.okta.com/help/s/article/Location-to-download-Okta-IDP-XML-metadata-for-a-SAML-app-in-the-new-Admin-User-Interface?language=en_US).
5. Copy the contents of the XML file and log in to your Retool instance. Go to the **Single-Sign On (SSO) > Custom SSO** settings, select **SAML SSO**, and paste the XML file contents to the **Identity Provider Metadata** field.
6. If not set already, [assign your app to your user](https://help.okta.com/en-us/content/topics/provisioning/lcm/lcm-assign-app-user.htm) in Okta.

1. Create a [custom SAML application](https://help.okta.com/oag/en-us/Content/Topics/Access-Gateway/add-app-saml-pass-thru-add-okta.htm) in Okta. Use the following settings.

| Setting                     | Value                              |
| --------------------------- | ---------------------------------- |
| Single sign-on URL          | `/api/saml/login` |
| Audience URI (SP Entity ID) | `your-org-domain-without-https`    |
| `firstName` attribute       | `user.firstName`                   |
| `lastName` attribute        | `user.lastName`                    |
| `email` attribute           | `user.email`                       |

2. In the **Feedback** tab, check **I'm a software vendor. I'd like to integrate my app with Okta**.
3. In your app's settings, go to the **Sign On** tab. Under **SAML Signing Certificates** > **SHA-2**, click **Actions** > **View IdP metadata**.
4. Copy the contents of the XML file and log in to Retool. Go to the [Single-Sign On (SSO) settings](https://login.retool.com/auth/login?source=docs&redirectOnLogin=settings/custom-sso), select **SAML SSO**, and paste the XML file contents to the **Identity Provider Metadata** field.
5. On the same page, enter `firstName` and `lastName` in the **Attributes** section.
6. In your Okta app under **Assignments**, assign users or groups to your app.

## Test the connection

---

## Configure OneLogin OIDC SSO

import Test from "./_partials/_test-connection.mdx"

Use this guide to configure OneLogin SSO with OpenID Connect (OIDC) on Retool. Once configured, users can log in to Retool with their OneLogin credentials.

## Requirements

To configure OneLogin SSO, you must:

- Have permissions to add environment variables to your Retool instance.
- Have permissions to create an OIDC application in OneLogin.

## 1. Create an OIDC application in OneLogin

Follow the steps in the [OneLogin OpenID Connect Customer Connector guide](https://onelogin.service-now.com/support?id=kb_article&sys_id=ed442c25db46255057fac24505961975&kb_category=93e869b0db185340d5505eea4b961934) to create a new OIDC application. Use the following settings.

### Configuration page

On the **Configuration** page, under **Redirect URIs**, enter `https:///oauth2sso/callback`.

### Parameters page

On the **Parameters** page, select **Configured by admin** under **Credentials**. In this section, you can add custom claims—for example, `user_id`.

### SSO page

On the **SSO** page, select **Web** as the **Application type**. Select **POST** as the **Token endpoint**.

Save the **Client ID** and **Client secret** to use in Retool.

## 2. Update Retool settings

Configure SSO settings in Retool.

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";
import SettingsOverride from "../_settings-override.mdx";

On Retool Cloud and self-hosted Retool versions 3.16 and later, enter settings on **Settings > Single Sign-On (SSO)**. Retrieve the values for **Client ID** and **Client secret** from the **SSO** page in OneLogin.

| Setting        | Example                                            |
| -------------- | -------------------------------------------------- |
| Client ID      | `CLIENTID`                                         |
| Client secret  | `CLIENTSECRET`                                     |
| Scopes         | `openid email profile groups params`               |
| Auth URL       | `https://ONELOGINDOMAIN.onelogin.com/oidc/2/auth`  |
| Token URL      | `https://ONELOGINDOMAIN.onelogin.com/oidc/2/token` |
| Email key      | `idToken.email`                                    |
| First name key | `idToken.given_name`                               |
| Last name key  | `idToken.family_name`                              |
| Roles key      | `idToken.groups`                                   |
| Role mapping   | `devops -> admin, support -> viewer`               |

On self-hosted Retool versions earlier than 3.16, configure the following [environment variables](../../self-hosted/reference/environment-variables/index.mdx#authentication) on your Retool instance.

Retrieve the values for `CUSTOM_OAUTH2_SSO_CLIENT_ID` and `CUSTOM_OAUTH2_SSO_CLIENT_SECRET` from the **SSO** page in OneLogin. Replace `ONELOGINDOMAIN` with your OneLogin domain.

| Environment variable                   | Example                                            |
| -------------------------------------- | -------------------------------------------------- |
| `CUSTOM_OAUTH2_SSO_CLIENT_ID`          | `CLIENTID`                                         |
| `CUSTOM_OAUTH2_SSO_CLIENT_SECRET`      | `CLIENTSECRET`                                     |
| `CUSTOM_OAUTH2_SSO_SCOPES`             | `openid email profile groups params`               |
| `CUSTOM_OAUTH2_SSO_AUTH_URL`           | `https://ONELOGINDOMAIN.onelogin.com/oidc/2/auth`  |
| `CUSTOM_OAUTH2_SSO_TOKEN_URL`          | `https://ONELOGINDOMAIN.onelogin.com/oidc/2/token` |
| `CUSTOM_OAUTH2_SSO_JWT_EMAIL_KEY`      | `idToken.email`                                    |
| `CUSTOM_OAUTH2_SSO_JWT_FIRST_NAME_KEY` | `idToken.given_name`                               |
| `CUSTOM_OAUTH2_SSO_JWT_LAST_NAME_KEY`  | `idToken.family_name`                              |
| `CUSTOM_OAUTH2_SSO_JWT_ROLES_KEY`      | `idToken.groups`                                    |
| `CUSTOM_OAUTH2_SSO_ROLE_MAPPING`       | `devops -> admin, support -> viewer`               |

You should also set the `BASE_DOMAIN` environment variable to ensure links using your domain are correct. Retool's backend tries to determine the `BASE_DOMAIN` if it is not set, but it can be incorrect if your website uses a proxy.

```
BASE_DOMAIN=https://retool.yourcompany.com
```

## 3. Test the connection

---

## SSO tutorials

import Tutorial from '/docs/_partials/_doctypes/_tutorial.mdx';

:::tip Learn more about SSO

Read the [quickstart](./quickstart.mdx) guide to learn more about using single-sign on (SSO) with your Retool organization.

:::

---

## Versioned documentation

Retool provides multiple versions of product documentation. The version you use depends on whether your Retool organization is cloud-hosted or self-hosted, and the release channel for your self-hosted deployment.

## Available versions

import Stable from './releases/_partials/_stable.mdx';
import Edge from './releases/_partials/_edge.mdx';

## Cloud and Edge release documentation

**Cloud and Edge** is the default version of Retool's documentation. It reflects the current product state for Retool Cloud and the latest Edge channel release of self-hosted Retool.

## Stable release documentation

---

## Agentic workflows

import Shared from '/docs/_shared/_agentic-workflows.mdx';

---

## Workflow performance best practices

Workflow performance issues can be difficult to replicate, especially when they involve external factors. The following best practices can help avoid problems and reduce the need to debug workflows. Refer to documentation on [limits](./limits.mdx) for details on configurations and limitations around timeout, memory, and cpu at the block and Workflow level.

:::info Query best practices

Refer to the [best practices guide for queries](../../queries/concepts/best-practices.mdx) to learn more about writing performant queries for your data.

:::

Queries that retrieve large amounts of data impact workflow performance. The more data a workflow query requests, the longer it can take for the resource to complete the operation.

Where possible, write queries that return only the results that the workflow needs (e.g., use API pagination and `WHERE` SQL query clauses).

To avoid timeouts, cpu throttling and out-of-memory errors, build separate workflows to perform specific tasks instead of a single workflow that performs them together.

Workflows are [rate-limited](limits.mdx) and have a minimum interval of one minute if triggered via a [schedule trigger](../guides/triggers/schedule.mdx). Additionally, a workflow must also complete its current run—regardless of success or failure—before it can be triggered again using a schedule trigger.

Whether you [schedule](../guides/triggers/schedule.mdx) a workflow to run automatically or trigger it using a [webhook](../guides/triggers/webhooks.mdx) event, make sure that the workflow triggers only when it should run. This also helps reduce the amount of interactions a workflow has with your data source, which can reduce operational costs.

Workflows that work with large datasets can run into [memory or time limits](./limits.mdx). Retool recommends performing load tests on production workflows to ensure that you don't encounter any issues. This is especially important with variably-sized datasets, in which looping or returning data from a code block might fluctuate in load size. Test out your workflow with a larger dataset than expected to ensure that you won't encounter any resource or time constraints.

Make sure you're aggregating metrics across all services, especially `workflows-backend`, `worker`, and `code-executor` pods. If you notice high memory or CPU usage, scale the service vertically or horizontally. To reduce congestion in queued workflows, consider scaling up workers and/or increasing the `WORKFLOW_TEMPORAL_CONCURRENT_TASKS_LIMIT` and `WORKFLOW_TEMPORAL_CONCURRENT_ACTIVITIES_LIMIT` [environment variables](../reference/environment-variables.mdx) to higher values (such as 100). Also, ensure your Temporal deployment is scaled appropriately to support increased traffic.

---

## Workflow IDE

The workflow IDE is the interface you use to build workflows. It is made up of four primary areas:

- [Canvas](#canvas) - Arrange blocks and build your workflow.
- [Left panel](#left-panel) - Add blocks and functions, configure triggers, and configure code libraries.
- [Status bar](#status-bar) - Select an environment and debug workflow runs.
- [Toolbar](#toolbar) - Rename, run, and publish a workflow.

## Canvas

The _canvas_ is the main area of the IDE and is where you build workflows. You add [blocks](../quickstart.mdx) to the canvas that perform specific actions. You can move blocks around using drag-and-drop to position them anywhere on the canvas.

### Navigation controls

The canvas has no size constraints and you can add as many blocks as you need. There are a number of controls to navigate the workflow:

#### Pan

import Minimapsvg from "/img/icons/workflows/minimap.svg";
import Minimapclosesvg from "/img/icons/workflows/minimap-close.svg";

Click and drag anywhere on the canvas to move (pan) around the workflow. You can also pan around the canvas using the minimap at the lower-right corner. The visible area is highlighted—click and drag to move it around.

The  **Minimap** also adjusts when changing the zoom level. Click  to collapse the minimap.

#### Zoom

You can zoom in and out of the canvas to show more content in the visible area. There are multiple ways to adjust the zoom:

- Use the **Zoom** controls in the lower-right of the canvas.
- Mod + or Mod -.
- Mod + Mouse scroll.
- Pinch-to-zoom with a trackpad.

You can quickly return the zoom level to 100% by double-clicking on any area of the canvas or clicking on the zoom level.

#### Fit to screen

import Fitviewsvg from "/img/icons/workflows/fit-view.svg";

Click  **Fit view** to automatically adjust the zoom level to view the entirety of the canvas.

### Layout controls

import Autolayoutsvg from "/img/icons/workflows/autolayout.svg";
import Commentssvg from "/img/icons/workflows/comment.svg";
import Expandsvg from "/img/icons/workflows/expand-all.svg";
import Collapsesvg from "/img/icons/workflows/collapse-all.svg";

You can adjust the layout of blocks on the canvas using the available layout controls:

| Layout control                                             | Description                                                                   |
| ---------------------------------------------------------- | ----------------------------------------------------------------------------- |
|  **Autolayout**    | Automatically arrange blocks on the canvas.                                   |
|  **Expand all**        | Expand all blocks.                                                            |
|  **Collapse all**    | Collapse all blocks.                                                          |
|  **Toggle comments** | Show or hide [block comments](../quickstart.mdx#comments). |

### View controls

import Graphsvg from "/img/icons/workflows/graph-view.svg";
import Treesvg from "/img/icons/workflows/tree-view.svg";

Use the  **Graph view** and  **Tree view** controls at the top-center of the canvas to switch between the standard or vertical view of the workflow.

## Left panel

import Addblocksvg from "/img/icons/workflows/add-block.svg";
import Outlinesvg from "/img/icons/workflows/outline.svg";
import Functionssvg from "/img/icons/workflows/functions.svg";
import Triggerssvg from "/img/icons/workflows/triggers.svg";
import Librariessvg from "/img/icons/workflows/libraries.svg";

The left panel contains a set of tabs to configure functionality of a workflow.

### Blocks

The  **Blocks** tab is one method for [adding blocks](../guides/blocks/blocks.mdx#add-blocks-from-the-toolbar-or-contextual-menu) to the canvas. 

### Outline

import Tabsvg from "/img/icons/workflows/tab.svg";
import Splitviewsvg from "/img/icons/workflows/splitview.svg";

The  **Outline** tab displays a list of all blocks in the workflow. You can click on each block to show it in the canvas. As with blocks on the canvas, blocks in the **Outline** section include an icon that reflects their function.

#### Split view

You can select a block to open it in _split view_, making it easier to edit blocks with complex code. Click  **Split view** in the toolbar to show or hide the split view pane.

:::tip Open blocks in split view

Click  **Open in tab** in the toolbar of any block to open it in split view.

:::

### Functions

The  **Functions** tab is where you configure [Function blocks](../guides/functions.mdx)—reusable blocks that can be called by other blocks. Function blocks don't appear on the canvas due to their purpose and are managed in this area of the panel.

### Triggers

The  **Triggers** tab is where you configure [webhook](../guides/triggers/webhooks.mdx) and [schedule](../guides/triggers/schedule.mdx) triggers to automatically run workflows.

### Libraries

The  **Libraries** tab enables you to use custom [JavaScript](../guides/blocks/code/javascript.mdx) or [Python](../guides/blocks/code/python.mdx) libraries within a workflow. You can select which libraries to use, customize their configuration, and view documentation.

### Releases

The  **Releases** tab allows you to view and create releases. You can also view your workflow's release history, and revert to previous versions.

## Status bar

The status bar contains environment options, run history, and debugging tools. Workflows can run using data sources set up in different [environments](../../org-users/guides/configuration/environments.mdx), such as production and staging. You can select the environment in which a workflow runs from the environment select input on the right of the status bar area.

:::note Configure multiple environments

Multiple environments are available to Retool organizations on [paid plans](https://retool.com/pricing/) and you must configure environment settings for each resource.

:::

import Historysvg from "/img/icons/workflows/runhistory.svg";

The  **Run History** section displays [run logs](workflows/quickstart.mdx#review-and-debug-runs) for a workflow and contains debugging tools, should you need to [troubleshoot a workflow](workflows/quickstart.mdx).

export function Status() {
  return (
    
      
    
  )
}

## Toolbar

import Runsvg from "/img/icons/workflows/run-with-previous.svg";

The _toolbar_ at the top of the IDE contains controls to rename, create or edit the [README](../guides/create-workflows-readme), run, and publish a workflow release.

Use the  **Run** and **Publish release** buttons to run a workflow or [release and publish it](../quickstart.mdx#publishing) it to run automatically.

export function Publish() {
  return (
    
      
    
  )
}

---

## Workflow limits

Retool imposes certain limits on workflow runs and workflow block runs.

## Workflow run limits
A workflow run is a complete execution of a workflow through a trigger (e.g. app, schedule, webhook, Retool Event) or by manually running the workflow in the workflow IDE.

### Timeout

Retool limits the length of time a workflow run can remain in execution before it is automatically terminated.

For asynchronous workflow runs, the timeout is 30 hours. For asynchronous workflows with User Task or Wait blocks, the timeout is unlimited. Individual Wait blocks are limited to a maximum of 60 days.

For synchronous workflows runs, the timeout is 15 minutes up to executing the first webhook Response block. The remainder of the workflow follows the *asynchronous* timeout.

### Memory and CPU

A workflow run can use up to 2.5GB in memory, and 1 vCPU.

A workflow run has no memory and CPU limits by default. To configure limits, set environment variable [`WORKFLOW_MONITOR_PROCESS_ENABLED`](https://docs.retool.com/self-hosted/reference/environment-variables/code-executor#variable-WORKFLOW_MONITOR_PROCESS_ENABLED) to `true` and constrain the memory and CPU usage of the workflow via [`WORKFLOW_MEMORY_LIMIT_MBS`](https://docs.retool.com/self-hosted/reference/environment-variables/code-executor#variable-WORKFLOW_MEMORY_LIMIT_MBS) and [`WORKFLOW_CPU_LIMIT`](https://docs.retool.com/self-hosted/reference/environment-variables/code-executor#variable-WORKFLOW_CPU_LIMIT) respectively.

### Logs

A workflow run has no limits on log generation or storage.

A workflow run can generate and store a maximum of 10MB of logs, after which logs will be truncated.

### Rate limits

*Workflow concurrency* is how many of the same workflow can be actively running within a 15 minute period. If a workflow reaches the concurrency limit, one of the runs must complete (whether successful or not) before another can start.

*Workflow burst limit* is how many runs of the same workflow can be triggered within a 10 second period.

Workflow concurrency is 100, and burst limit is 200.

Workflow concurrency and burst limit are not limited by default and can be configured via environment variables.

### Concurrent external requests

Retool limits the number of in-flight outbound requests to 50 at a time from a single workflow. Self-hosted customers can change this value by changing the `WORKFLOW_REQUEST_CONCURRENCY_LIMIT` [environment variable](../reference/environment-variables.mdx).

### Triggers

#### App triggers
Workflows triggered via an app are not limited by the 2 minute query timeout on cloud or [`DBCONNECTOR_QUERY_TIMEOUT_MS`](../../self-hosted/reference/environment-variables#variable-DBCONNECTOR_QUERY_TIMEOUT_MS) on self-hosted. If there is no webhook response block, the workflow is enqueued for asynchronous execution and responds immediately to the Retool frontend. If there is a webhook response block, the workflow waits to execute all the blocks until the first webhook Response block, respond back to the Retool frontend, and then enqueue the rest of the execution to happen asynchronously.

:::note

On self-hosted instances, any network middlewares (load balancers, proxies, etc.) outside of Retool's control that surround the `MAIN_BACKEND` and `CODE_EXECUTOR` may interfere with long running network requests.

:::

#### Schedule triggers

Workflows triggered via schedule support a minimum interval of one minute. Each workflow must complete its run (whether successful or not) before another run can start. For example, a workflow that takes two minutes to complete can only be triggered every two minutes.

Workflows triggered at intervals via schedule have additional jitter to reduce spikes in queue time and smooth out the load on the system. Jitter is calculated with the following equation: `random(0, min(6 minutes, 10% of the interval))`. For example, a workflow with a 1 hour interval will have a jitter of `random(0, 6 minutes)` and a workflow with a 10 minute interval will have a jitter of `random(0, 1 minute)`. On self-hosted instances, this behavior can be disabled in the Beta settings, by turning off `Enable jitter on cron Workflows.`.

#### Workflow block triggers

Synchronous workflows triggered from another workflow have a timeout of 3 minutes.

## Block limits

Limits covered in this section apply to individual blocks that make up a workflow. These limits apply when blocks are run individually via the workflows editor and when they are run as part of a workflow run.

### Resource blocks

The default timeout is 10 seconds and is configurable with block settings.

For asynchronous workflow runs, resource blocks can run up to 10 minutes. For synchronous workflows runs, resource blocks can run up to 2 minutes.

The default timeout is 10 seconds and is configurable with block settings.

For asynchronous workflows runs, Resource query blocks can run up to 24 days. For synchronous workflows runs, Resource query blocks can run up to 2 minutes.

:::note

**AI Action** blocks have a maximum timeout of 120 seconds. If your call takes longer than 120 seconds, you can instead call the AI provider directly via REST API, which doesn’t have this restriction.

:::

### Code blocks

The default timeout is 10 seconds and is configurable with block settings up to 10 minutes.

The default timeout is 10 seconds and is configurable with block settings up to 24 days.

### Loop blocks

[Loop blocks](../guides/blocks/logic/loop.mdx) can run indefinitely (up to the workflow run timeout), but each iteration has a default of 10 seconds and can run up to 2 minutes.

[Legacy Loop blocks](../guides/blocks/logic/loop-legacy.mdx) have a default of 10 seconds and can run up to 15 minutes, including all iterations.

### Workflow blocks

The default timeout is 10 seconds and is configurable with block settings up to 3 minutes.

---

## Workflow run logs

:::tip

You can use [Ask AI](../../queries/concepts/ask.mdx) to debug JavaScript, SQL, or GraphQL queries using AI-powered debugging.

:::

Retool logs every successful and failed run of a workflow. Click **Run history** in the status bar to debug workflows. 

## Run history

The **Run history** panel contains a list of recent runs, each with their date, time, their status. You can then view the status of each block to see where a failure may have occurred.

Run logs can help troubleshoot problems with queries, such as malformed SQL statements or invalid JavaScript. You can filter log entries by start or end time, by block name, and by **error**, **success**, or **info**. You can also  download JSON data from the **JSON** tab for use in other tool

Keep in mind that queries are considered successful if they didn't return an error. A workflow cannot determine if its actions produced the results you expected.

## Retention period

Admins can enable and update the retention period for **Run History** data from **Settings** > **Advanced** > **Workflows: Enable Data Retention**. The default retention period is 30 days. The maximum retention period is 90 days for Retool Cloud and 365 days for Self-hosted Retool.

---

## Workflows concept guides

import Concept from '/docs/_partials/_doctypes/_concept.mdx';

:::tip Get started with workflows

If you are unfamiliar with Retool Workflows and want to get started: 

- Read the [quickstart](./quickstart) to learn about the fundamental concepts.
- Follow a [tutorial](./tutorial/) to build your first workflow.

:::

---

## Use Invoke Agent blocks in Retool Workflows



---

## Create agentic workflows with the Invoke Agent block

import Connectsvg from "/img/icon-packs/bold/interface-geometric-circle-alternate.svg";

Use the **Invoke Agent** block to trigger a Retool Agent from a workflow. Invoking an agent from a workflow creates an agentic workflow. Refer to the [conceptual guide on agentic workflows](../../../concepts/agentic-workflows.mdx) for more information.

:::note

Some tools require user approval or authentication to execute. These tools are not available in agentic workflows.

:::

This guide provides instructions on how to invoke an agent and retrieve its results. Click the following link to download a sample workflow in JSON form and [import it as a new workflow](../../import-and-export.mdx). Configure the workflow to call your own agent.

  Sample Workflow

## Prerequisites

Before you create an agentic workflow, you must first create an agent. Refer to the [Retool Agents tutorial](../../../../agents/tutorials/agents-tutorial.mdx) for more information.

## Invoke agent

Complete the following steps to add and configure an **Invoke Agent** block:

1. Click and drag  to create a new, connected block.
2. Select **Invoke Agent** as the type of block.
3. Set **Agent** to the agent you want to call.

## Select the return type
The time period it takes to receive a result from an agent can vary widely based on a variety of factors, including the model used, the number of tools called, and the type of reasoning performed. If you need to use the outcome of your agent call in the workflow, you can get the results either synchronously, or asynchronously, by selecting the **Return type**. 

* The **Result (sync)** type is the default setting. This returns the direct result of the agent's output. 
* The **Run state (async)** type returns the `agentRunId`, `agentId`, and `status` only. It does not include the output of the agent. 

:::note
To enable **Result (sync)** for Agents in self-hosted Retool version 3.284.0 or 3.284.1, reach out to your account manager. 

In version 3.284.2, **Result (sync)** for Agents is available by default, and admins can disable it in **Settings > Beta** by toggling the `Agent blocks in workflows sync mode` feature flag. 
:::

export function ArcadeEmbed() {
  return (
    
      
    
  )
}

You can then transform the data returned with a [Code](../../blocks/code/) block, a [Filter](../../blocks/filter) block, or any other action you may want to take within the workflow.  

## Handle responses

Finally, set your workflow up to gracefully handle success and failure.

To return a successful response:

1. Click and drag  to create a new, connected block.
2. Chose the  type.
3. Set the **Status code** to **200**.

To handle a failure:

1. Right-click on the canvas to create a block that is not connected to the rest of the control flow. 
2. Chose the **Response** type.
3. Click the **•••** menu and select **Add global error handler**. This setting configures the block as an error handler that applies to the full workflow.
4. Set the **Status code** to `400`.
5. Paste the following code (or something similar) in the **Return body** field:
    ```json
    workflowContext.currentRun.error
    ```

Test out your workflow by clicking the  button before you publish it.

## Add triggers

Like any other workflow, you can add [triggers](../../../guides/triggers/index.mdx) to run your agentic workflow. Note that if you create a webhook trigger, anyone with access to the webhook will be able to trigger your agent.

## Configure block settings

Refer to the [Wait block reference](../../../reference/objects/block/agent.mdx) for information about the block's settings.

---

## Poll an agent's logs from the Invoke Agent block

import Connectsvg from "/img/icon-packs/bold/interface-geometric-circle-alternate.svg";

You can view the results of the agent's output with the [Result (sync)](../../blocks/agent/invoke-agent#select-the-return-type) type. Or, you can poll an agent from an **Invoke Agent** block to retrieve log information or an agent's response with the **Run state (async)** type.  

## Invoke agent

Complete the following steps to add and configure an **Invoke Agent** block:

1. Click and drag  to create a new, connected block.
2. Select **Invoke Agent** as the type of block.
3. Set **Agent** to the agent you want to call.

## Multi-step functions in agentic workflows

You can write a [multi-step function](../../functions.mdx) to retrieve the agent's response. 

1. Create a new function, and name it `getAgentLogs`.
2. Add two parameters: `agentRunId` and `agentId`.
3. Set the **Type** to **Multi-step**. Click the **Edit function** button to navigate to the multi-step function control flow.
4. Click and drag  to create a new, connected block.
5. Select **Invoke Agent** as the type of block. Name the block `requestAgentLogs`.
6. Set the **Agent** to the agent you want to call.
7. Set the **Return type** to **Run state (async)**. 
7. Choose **JSON** as the **Input Mode** and set the **Agent Inputs** to the following:

  ```json
  { 
    "action": "getLogs", 
    "agentRunId": {{ params.agentRunId }} 
  }
  ```
9. Add a  block with a **Status code** of `200` and return the following: `requestAgentLogs.data`.

The multi-step function
 

## Poll the agent for results

Once you've created a multi-step function, you can poll the agent logs for results. There's no need to do this if you are simply using the workflow to trigger the agent invocation.

1. Click and drag  to create a new, connected block.
2. Select the  block type, and name the block `pollForAgentLogs`.
3. Paste in the following JavaScript code snippet to populate your block:

  ```javascript
  // Defines the polling interval (500ms), the max number of attempts (30), and the statuses that indicate the agent is still running (PENDING, IN_PROGRESS).
  const SLEEP_DURATION_IN_MS = 500;
  const MAX_ITERATIONS = 30;
  const AGENT_PENDING_STATUSES = ["PENDING", "IN_PROGRESS"];

  // A utility function that returns a promise which resolves after 500ms (used for pausing between polls).
  const sleep = () => new Promise(resolve => setTimeout(resolve, SLEEP_DURATION_IN_MS));

  // Extracts the agentId and agentRunId from the invokeAgent block. Initializes latestAgentStatus as "PENDING". latestAgentLogs will store the last retrieved logs, and iterations tracks the number of attempts.
  const { agentId, agentRunId } = invokeAgent.data;
  let latestAgentStatus = "PENDING"; // Initial assumption
  let latestAgentLogs;
  let iterations = 1;

  // Keeps polling as long as the agent is in a pending state and hasn’t exceeded the maximum attempts.
  while (AGENT_PENDING_STATUSES.includes(latestAgentStatus) && iterations 
Polling for agent results

---

## Interact with AI models with the AI action block

The AI action block enables you to leverage Retool AI in workflows. You can write queries that instruct [AI models](../../../data-sources/concepts/models.mdx) to perform different actions, such as generating text or chat responses.

import Aiactionsvg from "/img/icons/workflows/ai-action.svg";

To interact with AI in a workflow, add an  **AI Action** block to the canvas.

## Available actions

Workflows supports the following AI actions:

- [Generate text](../../../queries/guides/ai/text.mdx)
- [Generate chat response](../../../queries/guides/ai/chat.mdx)
- [Generate image](../../../queries/guides/ai/image.mdx)

Refer to these guides to learn how you can write AI action queries.

## Configure block settings

Refer to the [AI Action block reference](../../reference/objects/block/ai-action.mdx) for information about the block's settings.

---

## Assemble workflow blocks

import Addsvg from "/img/icons/workflows/add-block.svg";
import Connectsvg from "/img/icon-packs/bold/interface-geometric-circle-alternate.svg";
import Removesvg from "/img/icons/workflows/remove.svg";
import Runsvg from "/img/icons/workflows/run.svg";
import Startsvg from "/img/icons/workflows/blocks/start.svg";

A workflow contains [blocks](../../quickstart.mdx) that execute actions, such as [resource queries](resource-query.mdx) or [AI actions](ai-action.mdx). Each block connects to another along the workflow's control flow—the connecting line that begins at the  **startTrigger** block—and executes sequentially. Each block must be part of the control flow to execute during a run.

## Connect and disconnect blocks

Click and drag  to create a new, connected block. The **Add block** contextual menu appears from which you select the type of block to add. If you already have two blocks you wish to connect, click and drag from one block to the other.

To disconnect blocks, hover the cursor over the connecting line and click . Any blocks that are not in the control flow (i.e., no connecting line) do not run automatically and display a warning if they are not connected.

## Add blocks from the toolbar or contextual menu

:::info Blocks not connected by default

Blocks you add to the canvas from the  **Blocks** tab or contextual menu are not automatically connected. You must add them to the control flow by connecting them to another block.

:::

Click  in the left panel to open the **Blocks** tab, then drag a block type to the canvas. You can also right-click anywhere on the canvas to display the **Add block** contextual menu.

## Duplicate a block

import Duplicatesvg from "/img/icons/workflows/duplicate.svg";

Click ••• on an existing block and select  **Duplicate**. This duplicates the block's state and any connections to or from it.

## Execute blocks

import Runprevioussvg from "/img/icons/workflows/run-with-previous.svg";

Click  in a block's toolbar to run it individually. The block uses results passed from the previous block in the control flow. The block returns an error if there is no data received from a previous block.

You can also run all previous blocks in the control flow when running a block. Click ••• and select  **Run with previous blocks**. This effectively runs the workflow but stops once it reaches the current block.

---

## Assemble code blocks in Retool Workflows



---

## Execute JavaScript with the Code block

import Codesvg from "/img/icons/workflows/blocks/code.svg";

[JavaScript](../../../../queries/quickstart.mdx) is the primary method for manipulating and transforming data in Retool. Use  **Code** blocks to write custom JavaScript code that can transform data and perform complex logic. You can also use popular [JavaScript libraries](#use-javascript-libraries) to further extend the functionality of workflows.

## Add a JavaScript Code block

To use JavaScript in a workflow, add a  Code block to the canvas and select **JavaScript**.

## Write and execute JavaScript code

import Branchsvg from "/img/icons/workflows/blocks/branch.svg";
import Filtersvg from "/img/icons/workflows/blocks/filter.svg";

You can build complex logic or manipulate data using JavaScript methods like `map()`. For example, you could transform an array of customer records using `map()`:

```javascript title="Map"
const data = query1.data;

return data.map((customer) => ({
  fullName: customer.name,
  emailAddress: customer.email,
}));
```

In general, Retool recommends you visually construct conditional statements with  [Branch](../logic/branch.mdx) blocks or filter query results using  [Filter](../logic/filter.mdx) blocks.

JavaScript Code blocks can also call [functions](../../functions.mdx)—reusable blocks that operate outside the workflow control flow. Functions allow your workflow to perform actions only when required and can receive parameters to use.

## Use JavaScript libraries

Retool includes support for a selection of popular JavaScript libraries which you can use in a workflow. You can browse and add libraries, configure their imports, and use them in your workflow.

### Preloaded libraries

import PreloadedLibraries from "/docs/_shared/_libraries/_preloaded-libraries.mdx";

### Built-in libraries

import BuiltIn from "/docs/_shared/_libraries/_built-in-libraries.mdx";

### Add custom libraries

If the library that you want to use is not available as a preloaded or built-in library, you can import packages from `npm`.

:::caution

Retool automatically tries to use [NsJail](https://github.com/google/nsjail) to sandbox the creation of execution environments for custom libraries. NsJail requires [privileged](https://docs.docker.com/reference/cli/docker/container/run/#privileged) container access. While NsJail [is not required](/changelog/workflow-libaries-without-nsjail) to use custom libraries, it is strongly recommended. 

Privileged mode is not supported by ECS/Fargate deployments. To make use of custom libraries, you must either:

- Migrate to an ECS/EC2 deployment or another supported deployment framework.
- Host the [code-executor](../../../../self-hosted/concepts/architecture#code-executor) service in a separate "jailed" cluster that can run containers in privileged mode on ECS/EC2 or another supported deployment framework. You then configure the [`CODE_EXECUTOR_INGRESS_DOMAIN`](../../../../self-hosted/reference/environment-variables/workflows.mdx#variable-code_executor_ingress_domain) environment variable to communicate with the `code-executor` service over `http` or `https`.

:::

Both Cloud and self-hosted organizations can import *public* packages from `npm`. Self-hosted organizations can also import *private* packages from `npm`. 

### Use libraries in a workflow

You can reference libraries using [JavaScript Code blocks](javascript.mdx) or as inline JavaScript in other blocks using `{{ }}`. You can also use `require()` to include libraries directly within JavaScript Code blocks.

For example, you could use [marked](https://marked.js.org/) to convert Markdown into HTML. This can be useful to generate reports and send them as HTML-formatted rich emails.

```javascript
return marked.parse(query1.data.message);
```

#### Add a public npm package

As with typical Node.js development, you provide a set of libraries to use in a `package.json` file. Retool makes this file available through the Workflow editor. To populate it, navigate to the **Libraries** tab, click **+**, then select **Modify package.json**.

```txt title="package.json"
{
    "dependencies": { 
        "lodash": "4.17.21", 
        "numbro": "2.1.0", 
        "papaparse": "5.3.2", 
        "moment-timezone": "0.5.23",
        "uuid": "3.4.0"
    }
}
```

#### Add a private npm registry

:::caution Requirements

Private npm registries are only supported for organizations using self-hosted Retool on versions 3.36+. You must also configure the [code-executor](../../../../self-hosted/concepts/architecture.mdx#code-executor) service in order to use private npm registries.

:::

Configuring a private npm registry requires setting some environment variables in the code executor service: `NPM_REGISTRIES` and `NPM_REGISTRY_AUTH_LINES`.

`NPM_REGISTRIES` is a comma-separated list of domains, and each entry uses either the `@scope:url` or `url` format.

```