# Example output: 10.1.2.3
```

Save the internal IP -- you will point the A record at it.

#### 2. Create a Cloud DNS private zone

1. Open _Network Services → Cloud DNS → Create zone_.
2. Select zone type **Private**.
3. Enter a **Zone name** (e.g., `temporal-cloud`).
4. Enter a **DNS name** based on the table above (e.g., `payments.abcde.tmprl.cloud` or `aws-us-east-1.region.tmprl.cloud`).
5. Select **Add networks** and choose the Project and Network that contains your PSC endpoint.
6. Click **Create**.

#### 3. Add an A record

Inside the new zone, add a _standard A record_:

| Field                | Value                                                          |
| -------------------- | -------------------------------------------------------------- |
| DNS name             | the namespace endpoint (e.g. `payments.abcde.tmprl.cloud`)     |
| Resource record type | A                                                              |
| TTL                  | 60s is typical, but you can adjust as needed.                  |
| IPv4 Address         | the internal IP address of your PSC endpoint (e.g. `10.1.2.3`) |

#### 4. Verify DNS resolution from inside the Network

```shell
dig payments.abcde.tmprl.cloud
```

If the hostname resolves to the PSC endpoint IP address from a VM in the bound network, the override is working.

### Updating your workers/clients

With private DNS in place, configure your SDKs exactly as the public-internet examples show (filling in your own namespace):

```go
clientOptions := client.Options{
    HostPort: "payments.abcde.tmprl.cloud:7233",
    Namespace: "payments",
    // No TLS SNI override needed
}
```

The DNS resolver inside your network returns the private endpoint IP address, while TLS still validates the original hostname—simplifying both code and certificate management.

## Available GCP regions, PSC endpoints, and DNS record overrides

The following table lists the available Temporal regions, PrivateLink endpoints, and regional endpoints used for DNS record overrides:

<GCPRegions />

---

## Connectivity

## Private network connectivity for namespaces

Temporal Cloud supports private connectivity to namespaces via AWS PrivateLink or GCP Private Services Connect in addition to the default internet endpoints.

Namespace access is always securely authenticated via [API keys](/cloud/api-keys#overview) or [mTLS](/cloud/certificates), regardless of how you choose to connect.

For information about IP address stability and allowlisting, see [IP addresses](/cloud/connectivity/ip-addresses).

### Required steps

To use private connectivity with Temporal Cloud:

1. Set up the private connection from your VPC to the region where your Temporal namespace is located.
1. Update your private DNS and/or worker configuration to use the private connection.
1. (Required to complete Google PSC setup, optional if using AWS PrivateLink): create a connectivity rule for the private connection and attach it to the target namespace(s). This will block all access to the namespace that is not over the private connection, but you can also add a public rule to also allow internet connectivity.

For steps 1 and 2, follow our guides for the target namespace's cloud provider:
- [AWS PrivateLink](/cloud/connectivity/aws-connectivity) creation and private DNS setup
- [Google Cloud Private Service Connect](/cloud/connectivity/gcp-connectivity) creation and private DNS setup

:::caution Finish client setup (complete step 2)

After creating a private connection, you must set up private DNS or update the configuration of all clients you want to use the private connection.

We recommend using private DNS.

Without this step, your clients may connect to the namespace over the internet if they were previously using public connectivity, or they will not be able to connect at all.

If that's not an option for you, refer to [our guide for updating the server and TLS settings on your clients](/cloud/connectivity#update-dns-or-clients-to-use-private-connectivity).

:::

For step 3, keep reading for details on [connectivity rules](/cloud/connectivity#connectivity-rules).

## Connectivity rules

:::tip Support, stability, and dependency info

Connectivity rules are currently in [public preview](/evaluate/development-production-features/release-stages#public-preview).

:::

:::info Web UI Connectivity

The Temporal Cloud Web UI is not currently subject to connectivity rule enforcement.
Even if a namespace is configured with private connectivity rules, the Web UI for that namespace remains accessible over the public internet.

:::

### Definition

Connectivity rules are Temporal Cloud's mechanism for limiting the network access paths that can be used to access a namespace.

By default, a namespace has zero connectivity rules, and is accessible from 1. the public internet and 2. all private connections you've configured to the region containing the namespace. Namespace access is always securely authenticated via [API keys](/cloud/api-keys#overview) or [mTLS](/cloud/certificates), regardless of connectivity rules.

When you attach one or more connectivity rules to a namespace, Temporal Cloud will immediately block all traffic that does not have a corresponding connectivity rule from accessing the namespace. One namespace can have multiple connectivity rules, and may mix both public and private rules.

Each connectivity rule specifies either generic public (i.e. internet) access or a specific private connection.

A public connectivity rule takes no parameters.

An AWS PrivateLink (PL) private connectivity rule requires the following parameters:

- `connection-id`: The VPC endpoint ID of the PL connection (ex: `vpce-00939a7ed9EXAMPLE`)
- `region`: The region of the PL connection, prefixed with aws (ex: `aws-us-east-1`). Must be the same region as the namespace. Refer to the [Temporal Cloud region list](/cloud/regions) for supported regions.

A GCP Private Service Connect (PSC) private connectivity rule requires the following parameters:

- `connection-id`: The ID of the PSC connection (ex: `1234567890123456789`)
- `region`: The region of the PSC connection, prefixed with gcp (ex: `gcp-us-east1`). Must be the same region as the namespace. Refer to the [Temporal Cloud region list](/cloud/regions) for supported regions.
- `gcp-project-id`: The ID of the GCP project where you created the PSC connection (ex: `my-example-project-123`)

Connectivity rules can be created and managed with [tcld](https://docs.temporal.io/cloud/tcld/), [Terraform](https://github.com/temporalio/terraform-provider-temporalcloud/), or the [Cloud Ops API](/ops)

### Permissions and limits

Only [Account Admins and Account Owners](/cloud/users#account-level-roles) can create and manage connectivity rules. Connectivity rules are visible to Account Developers, Account Admins, and Account Owners.

By default each namespace is limited to 5 private connectivity rules, and each account is limited to 50 private connectivity rules. You can [contact support](/cloud/support#support-ticket) to request a higher limit.

There is only one public rule allowed per account, because it's generic and can be reused for all namespaces that you want to be available on the internet. Trying to create more than one public rule will throw an error.

## Creating a connectivity rule

### Temporal Cloud CLI (tcld)

Create private connectivity rule (AWS):

```bash
tcld connectivity-rule create --connectivity-type private --connection-id "vpce-abcde" --region "aws-us-east-1"
```

Create private connectivity rule (GCP):

```bash
tcld connectivity-rule create --connectivity-type private --connection-id "1234567890" --region "gcp-us-central1" --gcp-project-id "my-project-123"
```

Create public connectivity rule (you only need to do this once ever in your account):

```bash
tcld connectivity-rule create --connectivity-type public
```

The `cr` alias works the same way:

Private connectivity rule:

```bash
tcld cr create --connectivity-type private --connection-id "vpce-abcde" --region "aws-us-east-1"
```

```bash
tcld cr create --connectivity-type public
```

### Terraform

[Examples in the Terraform repo](https://github.com/temporalio/terraform-provider-temporalcloud/blob/main/examples/resources/temporalcloud_connectivity_rule/resource.tf)

## Attach connectivity rules to a namespace

Be careful! When any connectivity rules are set on a namespace, that namespace is ONLY accessible via the connections defined in those rules. If you remove a connectivity rule that your workers are using, your traffic will be interrupted.

If you already have workers using a namespace, adding both a public rule and any private rules simultaneously can help you avoid unintended loss of access. You can then ensure all workers are using private connections, and then remove the public rule.

### Temporal Cloud CLI (tcld)

Setting the connectivity rules on a namespace:

```bash
tcld namespace set-connectivity-rules --namespace "my-namespace.abc123" --connectivity-rule-ids "rule-id-1" --connectivity-rule-ids "rule-id-2"
```

Or using aliases:

```bash
tcld n scrs -n "my-namespace.abc123" --ids "rule-id-1" --ids "rule-id-2"
```

Connectivity rules are attached as a set, so if rules `rule-a`, `rule-b`, and `rule-c` were attached to a namespace and you wanted to detach `rule-c` only, you'd make one call attaching both `rule-a` and `rule-b`:

```bash
tcld namespace set-connectivity-rules --namespace "my-namespace.abc123 --ids rule-a --ids rule-b
```

Remove all connectivity rules (this will make the namespace public):

```bash
tcld namespace set-connectivity-rules --namespace "my-namespace.abc123" --remove-all
```

### Terraform

[Example in the Terraform repo](https://github.com/temporalio/terraform-provider-temporalcloud/tree/main/examples/resources/temporalcloud_namespace/resource.tf#L113-L128)

## View the connectivity rules for a namespace

You have two ways to view the connectivity rules attached to a particular namespace.

### Get namespace

Connectivity rules are included in the namespace details returned by the `namespace get` command. 

```bash
tcld namespace get -n "my-namespace.abc123"
```

### List connectivity rules by namespace

To see only the connectivity rules for a specific namespace (without other namespace details), use the `connectivity-rule list` command with a namespace argument.

```bash
tcld connectivity-rule list -n "my-namespace.abc123"
```

## Update DNS or clients to use private connectivity

We strongly recommend using private DNS instead of updating client server and TLS settings: 

- [How to set up private DNS in AWS](/cloud/connectivity/aws-connectivity#configuring-private-dns-for-aws-privatelink) 
- [How to set up private DNS in GCP](/cloud/connectivity/gcp-connectivity#configuring-private-dns-for-gcp-private-service-connect)

If you are unable to configure private DNS, you must update two settings in your Temporal clients:

1. Set the endpoint server address to the PrivateLink or Private Services Connect endpoint (e.g. `vpce-0123456789abcdef-abc.us-east-1.vpce.amazonaws.com:7233` or `<GCP PSC IP address>:7233`)
2. Set TLS configuration to override the TLS server name (e.g., my-namespace.my-account.tmprl.cloud)

Updating these settings depends on the client you're using.

#### temporal CLI
```bash
TEMPORAL_ADDRESS=vpce-0123456789abcdef-abc.us-east-1.vpce.amazonaws.com:7233
TEMPORAL_NAMESPACE=my-namespace.my-account
TEMPORAL_TLS_CERT=<path/to/cert.pem>
TEMPORAL_TLS_KEY=<path/to/cert.key>
TEMPORAL_TLS_SERVER_NAME=my-namespace.my-account.tmprl.cloud

temporal workflow count -n $TEMPORAL_NAMESPACE
```

#### grcpurl
```bash
grpcurl \
    -servername my-namespace.my-account.tmprl.cloud \
    -cert path/to/cert.pem \
    -key path/to/cert.key \
    vpce-0123456789abcdef-abc.us-east-1.vpce.amazonaws.com:7233 \
    temporal.api.workflowservice.v1.WorkflowService/GetSystemInfo
```

#### Temporal SDKs
<SdkTabs>
<SdkTabs.Go>
```go
c, err := client.Dial(client.Options{
	HostPort:  "vpce-0123456789abcdef-abc.us-east-1.vpce.amazonaws.com:7233",
	Namespace: "namespace-name.accId",
	ConnectionOptions: client.ConnectionOptions{
		TLS: &tls.Config{
			Certificates: []tls.Certificate{cert},
			ServerName:   "my-namespace.my-account.tmprl.cloud",
		},
	},
})
```
</SdkTabs.Go>
<SdkTabs.Java>
```java
WorkflowServiceStubs service =
        WorkflowServiceStubs.newServiceStubs(
            WorkflowServiceStubsOptions.newBuilder()
                .setSslContext(sslContext)
                .setTarget("vpce-0123456789abcdef-abc.us-east-1.vpce.amazonaws.com:7233")
                .setChannelInitializer(
                    c -> c.overrideAuthority("my-namespace.my-account.tmprl.cloud"))
                .build());
```
</SdkTabs.Java>
<SdkTabs.TypeScript>
```ts
const connection = await NativeConnection.connect({
  address: "vpce-0123456789abcdef-abc.us-east-1.vpce.amazonaws.com:7233",
  tls: {
    serverNameOverride: "my-namespace.my-account.tmprl.cloud",
    //serverRootCACertificate,
    // See docs for other TLS options
    clientCertPair: {
      crt: fs.readFileSync(clientCertPath),
      key: fs.readFileSync(clientKeyPath),
    },
  },
});
```
</SdkTabs.TypeScript>
<SdkTabs.Python>
```python
client_config["tls"] = TLSConfig(
    client_cert=bytes(crt, "utf-8"),
    client_private_key=bytes(key, "utf-8"),
    domain="my-namespace.my-account.tmprl.cloud",
)

client = await Client.connect("vpce-0123456789abcdef-abc.us-east-1.vpce.amazonaws.com:7233")
```
</SdkTabs.Python>
<SdkTabs.DotNet>
```dotnet
// Create client 
var client = await TemporalClient.ConnectAsync(
  new(ctx.ParseResult.GetValueForOption(targetHostOption)!)
  {
    Namespace = ctx.ParseResult.GetValueForOption (namespaceOption)!,
    // Set TLS options with client certs. Note, more options could 
    // be added here for server CA (i.e. "ServerRootCACert") or SNI 
    // override (i.e. "Domain") for self-hosted environments with 
    // self-signed certificates. 
    Tls = new() 
    {
      ClientCert = 
        await File.ReadAllBytesAsync(ctx.ParseResult.GetValueForOption(clientCertOption) !.FullName), 
      ClientPrivateKey = 
        await File.ReadAllBytesAsync(ctx.ParseResult.GetValueFor0ption(clientKey0ption)!.FullName), Domain = "my-namespace.my-account.tmprl.cloud",
  },
});

// dotnet run --target-host "vpce-0123456789abcdef-abc.us-east-1.vpce.amazonaws.com:7233"
```
</SdkTabs.DotNet>
</SdkTabs>

To check whether your client has network connectivity to the private endpoint in question, run:
```bash
nc -zv vpce-0123456789abcdef-abc.us-east-1.vpce.amazonaws.com 7233
```

## Control plane connectivity

Using the Temporal Cloud [web UI](/web-ui), [Terraform provider](/cloud/terraform-provider), [`tcld` CLI](/cloud/tcld), or [Cloud Ops APIs](/ops) requires network access to the Temporal Cloud control plane. 

### Control plane hostnames

Different hostnames are used for different parts of the service.

- `saas-api.tmprl.cloud` (required for Terraform, tcld, and Cloud Ops APIs) 
- `web.onboarding.tmprl.cloud` (required for Web UI)
- `web.saas-api.tmprl.cloud` (required for Web UI)

### AWS PrivateLink connectivity to Temporal Cloud control plane

Temporal Cloud supports [AWS PrivateLink](https://aws.amazon.com/privatelink/) connections to the control plane, which allows access from applications running in VPCs that cannot egress to the public internet. Temporal Cloud does **not** support restricting an account so that private connectivity is the sole connectivity method to the control plane; the control plane is always accessible via public internet.

Control plane access is always securely authenticated via [API keys](/cloud/api-keys#overview) or JWT tokens, regardless of how you choose to connect.

To set up a PrivateLink connection to the Temporal Cloud control plane, follow [these instructions](/cloud/connectivity/aws-connectivity), but use the control plane endpoint information below:

| Hostname               | Region      | Control Plane PrivateLink Service Name                    |
| ---------------------- | ----------- | --------------------------------------------------------- |
| `saas-api.tmprl.cloud` | `us-west-2` | `com.amazonaws.vpce.us-west-2.vpce-svc-0c57a5930b6f6be0e` |

The control plane PrivateLink endpoint includes a [private DNS name](https://docs.aws.amazon.com/vpc/latest/privatelink/manage-dns-names.html), which lets your clients use the PrivateLink connection without having to set up private DNS or having to override client configuration. To use the DNS name, make sure your VPC has the `Enable DNS hostnames` and `Enable DNS support` options enabled. If you cannot use the DNS name, you can also manually [set up private DNS](/cloud/connectivity/aws-connectivity#configuring-private-dns-for-aws-privatelink) or [override the server and TLS settings on your clients](/cloud/connectivity#update-dns-or-clients-to-use-private-connectivity).

:::caution Finish client setup

After creating a private connection, you must use the provided DNS name, set up private DNS, or update the configuration of all clients you want to use the private connection.

Without this step, your clients may connect to the control plane over the internet if they were previously using public connectivity, or they will not be able to connect at all.
:::

The control plane is also accessible via the PrivateLink endpoint in AWS us-west-2 that can be used for namespace traffic, but we strongly recommend using the control-plane specific endpoint for control plane traffic.

---

## Temporal Cloud IP Addresses

The specific IP addresses for Temporal Cloud resources are subject to change at any time. Temporal Cloud resources may use any IPs within the IP ranges published by the relevant cloud provider.

If you need to limit outbound access from your client network, we recommend using [AWS PrivateLink or GCP Private Services Connect](/cloud/connectivity#private-network-connectivity-for-namespaces) instead of IP allowlisting.

:::warning Do not allowlist specific IP addresses

**Temporal Cloud IPs are not static and may change without notice.** 

Do not allowlist specific IP addresses you see Temporal Cloud services using at a point in time, as this **will cause an outage** when those IPs change. Your clients will not be able to connect to Temporal Cloud.

If you have to allowlist IP ranges, you must allowlist the entire cloud provider IP range:
- [AWS IP address ranges](https://ip-ranges.amazonaws.com/ip-ranges.json)
- [GCP IP address ranges](https://www.gstatic.com/ipranges/cloud.json)

:::

---

## Workflow History Export

Workflow History Export allows users to export closed Workflow Histories from Temporal Cloud to cloud object storage (AWS S3 or GCP GCS), enabling:

- Compliance and audit trails of complete Workflow History data in [proto format](https://github.com/temporalio/api/blob/master/temporal/api/export/v1/message.proto)
- Analytics on Workflow History when ingested to the data platform of your choice

Workflow History Export in Temporal Cloud provides similar functionality as [Archival](/self-hosted-guide/archival) in a Self-Hosted Temporal Server. 
Archival is not supported in Temporal Cloud.

Exports run hourly, beginning 10 minutes after the hour. 
Allow up to 24 hours for a closed Workflow to appear in the exported file.
Delivery is guaranteed at least once.

## Prerequisites {#prerequisites}

To use Workflow History Export, you must have:

1. A cloud account in the cloud provider where your Namespace is hosted.
2. An object storage bucket available to receive the exported History.

## Configure Workflow History Export {#configure}

### AWS

[AWS S3 Export Configuration](/cloud/export/aws-export-s3)

### GCP

[GCP GCS Export Configuration](/cloud/export/gcp-export-gcs)

## Verify export setup {#verify}

From the Export configuration page, select **Verify**.
This validates that Temporal can successfully write a test file to your object storage.

If everything is configured correctly, you will see a `Success` status indicating Temporal has written to the object store.

## Monitor export progress {#monitor}

After Export has been configured, you can check that it's still working in several ways:

1. **Object Storage**:

   - File Delivery: After the initial hour of setting up, inspect your object storage.
     You should see the exported Workflow History files.
   - Directory Structure: Your exported files will adhere to the following naming convention and path:

   ```bash
   //[bucket-name]/temporal-workflow-history/export/[Namespace]/[Year]/[Month]/[Day]/[Hour]/[Minute]/
   ```
The exported file name will include a randomly generated ID. The time recorded in the directory structure
is the time the export uploads to object storage, not the Workflow completion time.

2. **Temporal Cloud Web UI**:

   - Export UI:

      - Last Successful Export: This displays the timestamp of the most recent successful export.
      - Last Status Check: This reflects the timestamp of the latest internal Workflow healthcheck.

   - Usage Dashboard:
      - Actions from the Export Job are included in the [Usage Dashboard](/cloud/billing-and-cost).

3. **Metrics**:
   - Export-related metrics are available from the [Cloud metrics endpoint](/cloud/metrics/), specifically the metric `temporal_cloud_v1_total_action_count` with the label `is_background="true"`.

4. **Email**:
   - Emails are sent to `Namespace Administrator`, `Account Owner`, and `Global Administrator` roles when a Workflow History Export job fails due to a user related error (such as Object Store permissions issue).

## Working with exported files

Use the proto schema defined [here](https://github.com/temporalio/api/blob/master/temporal/api/export/v1/message.proto) to deserialize exported files.

### Using exported files in analytics

It can be useful to convert protos to another format to perform analytics on the data. To convert protos to parquet, follow 
[the example Python Workflow](https://github.com/temporalio/samples-python/tree/main/cloud_export_to_parquet). Note that this example Workflow:
* Transforms the nested proto structure into a flat, tabular format.
* Each row in the table represents a single history event from a Workflow. To preserve their relationship post-conversion, the `workflowID` and `runID` is included in every row.
* If you have enabled the codec server, the payload field is encrypted. This field may contain characters that are not recognized when loaded into a database so the payload field is excluded in this example.

## Export and High Availability Namespaces {#export-ha}

### Export Region Persistence

When Export is configured for a [High Availability](/cloud/high-availability) Namespace, the export is tied to the specific region where it was initially set up. The export configuration does not automatically failover with the Namespace.

- If Export is configured in Region A, it will continue to export from Region A's storage even after a Namespace failover to Region B
- Exports always read from and write to the same region where they were originally configured
- The export process is independent of Namespace failover events
- Export does not fail over automatically because we prioritize data completeness and consistency over real-time availability for exports. HA data replication has inherent latency, which could result in incomplete or inconsistent exports during a failover.

### Failover Scenarios

**Namespace Failover with Healthy Primary Region**: When a Namespace fails over to a secondary region but the primary region remains healthy (including its blob storage), the export job continues to operate from the primary region. It does not automatically switch to export data from the secondary region.

**Primary Region Outage**: If the primary region (where Export was configured) experiences a complete outage including S3/GCS storage: Exports will be unavailable until the primary region recovers. Once the primary region recovers, export will resume and include any Workflow histories that occurred during the outage. There may be delays in export processing, but the complete dataset will eventually be available. It does not automatically switch to export data from the secondary region.

---

## Exporting Workflow Event History to GCS

## Prerequisites {#prerequisites}

Before configuring the Export sink, complete the following steps in Google Cloud.

1. Create a GCS bucket and take note of its bucket name, for example, "test-export"

- Enable customer-managed encryption keys (CMEK) if you need additional security for your GCS bucket.
- Currently, only single region buckets are supported (choose "Region" option when creating the bucket in GCS, not "Multi-region" or "Same-region")
- The region of the bucket must be the same as the region of your Temporal Cloud Namespace.

2. Record the GCP Project ID that owns the bucket.
3. Create a service account in the same project that grants Temporal permission to write to your GCS bucket.
4. Follow the instructions in the Temporal Cloud UI. There are two ways to set up this service account:
   - Manual Setup:
     - Input the service account ID, GCP project ID and GCS bucket name.
     - Follow the instructions, manually set up a new service account.
   - Automated Setup:
     - Use the [Terraform template](https://github.com/temporalio/terraform-modules/tree/main/modules/export-sa) to create the service account.

## Configure Workflow History Export

There are multiple ways to configure export: through the [Temporal Cloud UI](#using-temporal-cloud-ui), [`tcld`](#using-tcld), or [`terraform`](#using-terraform).

:::note Why does Temporal Cloud provision multiple service accounts for Export?

Temporal Cloud creates multiple intermediary service accounts for export operations primarily for security purposes. The system randomly selects from these accounts when writing to your storage sink, which provides several benefits:

- **Security isolation**: If one service account is compromised or needs to be decommissioned, other accounts remain available
- **Load distribution**: Prevents exclusively using a single account, reducing security risk
- **Warm standby**: Keeps multiple accounts active to avoid potential throttling when switching between accounts
- **Reliability**: Provides resilience against cloud provider account-level issues that could affect a single service account

This approach prioritizes security and availability, ensuring robust export operations even if individual service accounts encounter issues.
:::

### Using Temporal Cloud UI

The following steps guide you through setting up Workflow History Export using the Temporal Cloud UI.

![](/img/cloud/gcp/export-sink-ui-gcp.png)

1. In the Cloud UI, navigate to the Namespaces section. Confirm that the Export feature is visible and properly displayed.
2. Configure the Export sink for a Namespace:
   1. Choose GCS as the sink type.
   2. Provide the following information:
      1. Name
      2. Service account ID
      3. GCP Project ID
      4. GCS bucket name
3. After inputting the necessary values, click on **Verify**.
   You should be able to write to the sink successfully.
   If not, please fix any errors or reach out to support for help.
   - If you just created the GCS bucket and granted permission for your service account, it may take some time for the permission to propagate. You may need to wait up to 5 minutes before clicking the **Verify** button to verify the connection.
4. Clicking **Create** will complete the Export sink setup.
5. The page will auto-refresh and you should see the status “Enabled” on the Export screen.
   You are now ready to export Workflow histories.
6. You can toggle the enable button if you want to stop export and resume in the future.
   **Note**: when you re-enable the feature, it will start from the current point in time, and not from the time when you disabled export.
7. You can also delete export by clicking **Delete**.

:::tip

Don't forget to click Create at the end of your setup to confirm your export.

:::

### Using tcld

To access export-related commands in tcld, please follow these steps:

1. Download the latest version of tcld from following instructions (here)[https://docs.temporal.io/cloud/tcld/#install-tcld].
2. Make sure your tcld version is v0.35.0 or above.
3. Run the command: `tcld n export gcs`:
   ```bash
   NAME:
      tcld namespace export gcs - Manage GCS export sink

   USAGE:
      tcld namespace export gcs command [command options] [arguments...]

   COMMANDS:
      create, c    Create export sink
      update, u    Update export sink
      validate, v  Validate export sink
      get, g       Get export sink
      delete, d    Delete export sink
      list, l      List export sinks
      help, h      Shows a list of commands or help for one command

   OPTIONS:
      --help, -h  show help
   ```

4. Run the `tcld n export gcs create` command and provide the following information:
   - `--namespace`: The Namespace to configure export for.
   - `--sink-name`: The name of the export sink.
   - `--service-account-email`: The service account that has access to the sink.
   - `--gcs-bucket`: The name of the GCP GCS bucket.
   
   For example:

   ```bash
   tcld n export gcs create -n test.ns --sink-name test-sink --service-account-email test-sink@test-export-sink.iam.gserviceaccount.com --gcs-bucket test-export-validation
   ```
5. Check the status of this command by either viewing the Namespace Export status in the Temporal Cloud UI or using the following command and looking for the state of "Active":

```bash
tcld n export gcs g -n test.ns --sink-name test-sink
{
	"name": "test.ns",
	"resourceVersion": "b954de0c-c6ae-4dcc-90bd-3918b52c3f28",
	"state": "Active",
	"spec": {
		"name": "test-sink",
		"enabled": true,
		"destinationType": "Gcs",
		"s3Sink": null,
		"gcsSink": {
			"saId": "test-sink",
			"bucketName": "test-export-validation",
			"gcpProjectId": "test-export-sink",
		}
	},
	"health": "Ok",
	"errorMessage": "",
	"latestDataExportTime": "0001-01-01T00:00:00Z",
	"lastHealthCheckTime": "2024-01-23T06:40:02Z"
}
```

### Using `terraform`

See the [Terraform export support](https://registry.terraform.io/providers/temporalio/temporalcloud/latest/docs/resources/namespace_export_sink) for setup instructions.

### Next Steps

- [Verify export setup](/cloud/export#verify)
- [Monitor export progress](/cloud/export#monitor)
- [Work with exported files](/cloud/export#working-with-exported-files)

---

## Manage API keys

Temporal Cloud API keys offer industry-standard identity-based authentication for Temporal users and
[Service Accounts](/cloud/service-accounts). This document introduces Temporal Cloud's API key features:

- [API key overview](#overview)
- [API key best practices](#best-practices)
- [Global Administrator and Account Owner API key management](#manage-api-keys)
- [User API key management](#user-api-keys)
- [Manage API keys for Service Accounts](#serviceaccount-api-keys)
- [API keys for Namespace authentication](#namespace-authentication)
- [Use API keys to authenticate](#using-apikeys)
- [Troubleshoot your API key use](#troubleshooting)
- [API keys: Frequently Asked Questions](#faqs)

## API key overview {#overview}

Each Temporal Cloud API key is a unique identity linked to role-based access control (RBAC) settings to ensure secure
and appropriate access.

The authentication process follows this pathway:

<CaptionedImage
  src="/img/cloud/apikeys/apikeyrbac.png"
  title="API key (authentication) → Identity (user or Service Account) → RBAC (authorization)"
/>

## API key best practices {#best-practices}

- **Keep it secret; keep it safe**: Treat your API key like a password. Do not expose it in client-side code, public
  repositories, or other easily accessible locations.
- **Rotate keys regularly**: Change your API keys periodically to reduce risks from potential leaks.
- **Design your code for key updates**: Use key management practices that retrieve your API keys without hard-coding
  them into your apps. This lets you restart your Workers to refresh your rotated keys without recompiling your code.
- **Monitor API key usage**: Check usage metrics and logs regularly. Revoke the key immediately if you detect any
  unexpected or unauthorized activity.
- **Use a Key Management System (KMS)**: Employ a Key Management System to minimize the risk of key leaks.

### API key use cases

API keys are used for the following scenarios:

- _**Cloud operations automation**_: API keys work with Temporal Cloud operational tools, including
  [`tcld`](/cloud/tcld), [Cloud Ops APIs](/ops), and
  [the Terraform provider](/cloud/terraform-provider). Use them to manage your Temporal Cloud
  account, Namespaces, certificates, and user identities.
- _**Namespace authentication**_: API keys serve as an authentication mechanism for executing and managing Workflows via
  the SDK and Temporal CLI, offering an alternative to mTLS-based authentication.

### API key supported tooling

Use API keys to authenticate with:

- [The Temporal CLI](/cli)
- [Temporal SDKs](/develop)
- [`tcld`](/cloud/tcld/index.mdx)
- [The Cloud Operations API](/cloud/operation-api.mdx)
- [Temporalʼs Terraform provider](/cloud/terraform-provider)

### API key permissions

API keys support both users and Service Accounts. Here are the differences in their permissions:

- Any user can create, delete, and update their _own_ API key using the Cloud UI or `tcld`.
- Only Global Administrators and Account Owners can create, delete, and update access to API keys for all types of
  Service Accounts.
- Namespace Admins can create, delete, and update access to API keys for the Namespace-scoped Service Accounts they
  administer.

### API key prerequisites

Check these setup details before using API keys:

- The Global Administrator or Account Owner may need to [enable API keys access](#manage-api-keys) for your Temporal
  Account.
- Have access to the [Temporal Cloud UI](https://cloud.temporal.io/) or Temporal Cloud CLI
  ([tcld](https://docs.temporal.io/cloud/tcld/)) to create an API key.

## Global Administrator and Account Owner API key management {#manage-api-keys}

Global Administrators and Account Owners can monitor, manage, disable, and delete API keys for any user or Service
Account within their account. To manage your account’s API keys:

1. [Log in](https://cloud.temporal.io/) to the Temporal Cloud UI.
1. Go to [Settings → API Keys](https://cloud.temporal.io/settings/api-keys)

Administrators can disable the creation of new API keys using the **Disable Create API Keys** button on the **API Keys**
Settings page. Existing API keys can still be used to authenticate into Temporal Cloud normally until they are either
disabled, deleted, or expired.

To disable or delete an individual API key use the vertical ellipsis menu in the API key table row.

To find an API key, you can filter by API key state and identity type (Global Administrators and Account Owners only).

:::caution DISABLED API KEYS

Deleting or disabling a key removes its ability to authenticate into Temporal Cloud. If you delete or disable an API key
being used by Workers to run a Workflow, those Workers will be unable to connect to Temporal until a new API key secret
is created and configured.

:::

## User API key management {#user-api-keys}

Manage your personal API keys with the Temporal Cloud UI or `tcld`. These sections show you how to generate, manage, and
remove API keys for a user.

### Generate an API key

Create API keys using one of the following methods:

:::caution

- Once generated, copy and securely save the API key. It will be displayed only once for security purposes.

:::

#### Generate API keys with the Temporal Cloud UI

[Log in](https://cloud.temporal.io/) to the Temporal Cloud UI and navigate to your
[Profile Page → API Keys](https://cloud.temporal.io/profile/api-keys). Then select **Create API key** and provide the
following information:

- **API key name**: A short, identifiable name for the key
- **API key description**: A longer description of the key's use
- **Expiration date**: The end date for the API key

Finish by selecting **Generate API Key**.

#### Generate API keys with tcld

To generate an API key, log into your account and issue the following command:

```command
tcld login
tcld apikey create \
    --name <api-key-name> \
    --description "<api-key-description>" \
    --duration <api-key-duration>
```

Duration specifies the time until the API key expires, for example: "30d", "4d12h", etc.

### Enable or Disable an API key

You can enable or disable API keys. When disabled, an API key cannot authenticate with Temporal Cloud.

#### Manage API key state with the Temporal Cloud UI

Follow these steps:

1. [Log in](https://cloud.temporal.io/) to the Temporal Cloud UI.
1. Go to your [Profile Page → API Keys](https://cloud.temporal.io/profile/api-keys).
1. Select the vertical ellipsis menu in the API key table row.
1. Choose **Enable** or **Disable**.

#### Manage API Key State with tcld

To manage an API key, log into your account and use one of the following commands to enable or disable it:

```command
tcld login
tcld apikey disable --id <api-key-id>
tcld apikey enable --id <api-key-id>
```

### Delete an API key

Deleting an API key stops it from authenticating with Temporal Cloud.

:::caution

Deleting an API key used by Workers to run a Workflow will cause it to fail unless you rotate the key with a new one.
This can affect long-running Workflows that outlast the API key's lifetime.

:::

#### Delete API keys with the Temporal Cloud UI

Follow these steps to remove API keys:

1. [Log in](https://cloud.temporal.io/) to the Temporal Cloud UI.
1. Navigate to your [Profile Page → API Keys](https://cloud.temporal.io/profile/api-keys).
1. Select the vertical ellipsis menu in the API key table row.
1. Choose **Delete**.

#### Delete API keys with tcld

To delete an API key, log into your account and issue the following:

```command
tcld login
tcld apikey delete --id <api-key-id>
```

### Rotate an API key

Temporal API keys automatically expire based on the specified expiration time. Follow these steps to rotate API keys:

1. Create a new key. You may reuse key names if that helps.
1. Ensure that both the original key and new key function properly before moving to the next step.
1. Switch clients to load the new key and start using it.
1. Delete the old key after it is no longer in use.

## Manage API keys for Service Accounts {#serviceaccount-api-keys}

Global Administrators and Account Owners can manage and generate API keys for _all_ Service Accounts in their account.
Namespace Admins can manage and generate API keys for the Namespace-scoped Service Accounts they administer.

This is different for non-admin users, who manage and generate their own API keys.

### Generate an API Key for a Service Account

Create API keys for Service Accounts using one of the following methods:

:::caution

- Once generated, copy and securely save the API key. It will be displayed only once for security purposes.

:::

#### Generate API Keys with the Temporal Cloud UI

[Log in](https://cloud.temporal.io/) to the Temporal Cloud UI. Global Administrators or Account Owners can go to
[Settings → API Keys](https://cloud.temporal.io/settings/api-keys). Namespace Admins can go to
[Profile Page → API Keys](https://cloud.temporal.io/profile/api-keys). Select **Create API Key**, then choose **Service
Account** from the "Create an API key for" dropdown. In the "Mapped to identity" input box, select a Service Account and
provide the following information:

- **API key name**: A short, identifiable name for the key
- **API key description**: A longer description of the key's use
- **Expiration date**: The end date for the API key

Finish by selecting **Generate API Key**.

#### Generate API keys with tcld

To create an API key for a Service Account, use `tcld apikey create` with the `--service-account-id` flag:

```
tcld apikey create \
    --name <api-key-name> \
    --description "<api-key-description>" \
    --duration <api-key-duration> \
    --service-account-id <service-account-id>
```

### Enable or Disable an API key

Global Administrators and Account Owners can manage API key access for any user in their account using the Temporal
Cloud UI or `tcld`.

#### Manage keys with Temporal Cloud UI

Follow these steps:

1. [Log in](https://cloud.temporal.io/) to the Temporal Cloud UI.
1. Global Administrators or Account Owners can go to [Settings → API Keys](https://cloud.temporal.io/settings/api-keys).
   Namespace Admins can go to [Profile Page → API Keys](https://cloud.temporal.io/profile/api-keys).
1. Find the API key. Use the vertical ellipsis menu in the table row and select the Disable/Enable option to perform the
   action. There may be a delay after changing the status. Once successful, the updated API key status will be shown in
   the row.

#### Manage keys with tcld

Use the `tcld apikey disable` or `tcld apikey enable` command to disable or enable an API key:

```
tcld login
tcld apikey disable --id <api-key-id>
tcld apikey enable --id <api-key-id>
```

This command is the same for users and Service Accounts.

### Delete an API key for a Service Account

Global Administrators and Account Owners can delete API keys for any user or Service Account in their account using the
Temporal Cloud UI or `tcld`. Deleting a key removes its ability to authenticate with Temporal Cloud. If you delete an
API key used by a Worker to run a Workflow, that Worker will fail to connect to Temporal server unless you rotate the
API key with a new one.

#### Delete a Service Account API key with Temporal Cloud UI

Follow these steps:

1. Go to [Settings → API Keys](https://cloud.temporal.io/settings/api-keys).
1. Locate the API key. Use the vertical ellipsis menu in the table row and select the Delete option. There may be a
   delay after deleting the API key.
1. Once successful, the updated API key status will be reflected in the row.

#### Delete a Service Account API key with tcld

Use the `tcld apikey delete` command to delete an API key. The process for deleting an API key is the same for a user or
Service Account.

```
tcld login
tcld apikey delete --id <api-key-id>
```

### Rotate a Service Account API key

Temporal API keys automatically expire based on the specified expiration time. Follow these steps to rotate API keys:

1. Create a new key. You may reuse key names if that helps.
1. Ensure that both the original key and new key function properly before moving to the next step.
1. Switch clients to load the new key and start using it.
1. Delete the old key after it is no longer in use.

:::tip

Service Accounts can rotate their own API keys irrespective of their configured permissions. To use this feature, have
your Service Account create a new API key using the [Cloud Ops APIs](/ops) or [`tcld`](/cloud/tcld) before the current
one expires. Service Accounts cannot delete their own API keys without the requisite permissions, which helps keep
Workflow access secure.

:::

## API keys for Namespace authentication {#namespace-authentication}

Create a Namespace with API key authentication as an alternative to mTLS-based authentication by selecting "Allow API
key authentication" during setup. The gRPC endpoint format for the Namespace depends on the authentication method and
whether or not High Availability features are enabled.

See the following documentation for [accessing Namespaces](/cloud/namespaces#access-namespaces) for more information.

:::info

When switching on or off High Availability features for a Namespace, you may need to update the gRPC endpoint used by
your Workers and Clients, because the Namespace endpoint changes based on whether High Availability features are
enabled. See [Disable High Availability](/cloud/high-availability/enable#disable) for more information.

:::

### Without High Availability features

Use the gRPC regional endpoint `<region>.<cloud_provider>.api.temporal.io:7233`.

### With High Availability features

Use the gRPC Namespace endpoint: `<namespace>.<account>.tmprl.cloud:7233`. This allows Workers and Clients to always
connect to the active region of the Namespace. In a failover event, Temporal Cloud changes the Namespace's active region
and points this Namespace endpoint to the new active region.

## Use API keys to authenticate {#using-apikeys}

Authenticate with Temporal Cloud using API keys with the following clients:

- [Temporal CLI](/cli)
- [SDKs](/develop)
- [Temporal Cloud CLI `tcld`](/cloud/tcld/index.mdx)
- [The Cloud Operations API](/cloud/operation-api.mdx)
- [Temporal’s Terraform Provider](/cloud/terraform-provider)

### Temporal CLI

To use your API key with the Temporal CLI, either pass it with the `--api-key` flag or set an environment variable in
your shell (recommended). The CLI automatically picks up the `TEMPORAL_API_KEY` environment variable from your shell.

In addition to the API key, the following client options are required:

- `--address`: Provide the Namespace's gRPC endpoint from the Namespace UI's gRPC endpoint box.
  - For API key connections, use the format `<region>.<cloud_provider>.api.temporal.io:7233`.
  - You can set the address using an environment variable.
- `--namespace`: Provide the `namespace.accountId` from the top of the Namespace page in the UI.
  - Use the format `<namespace_id>.<account_id>`.
  - This can be set using an environment variable.

For example, to connect to Temporal Cloud from the CLI using an environment variable for the API key:

```bash
export TEMPORAL_API_KEY=<key-secret>
temporal workflow list \
    --address <endpoint> \
    --namespace <namespace_id>.<account_id>
```

:::tip ENVIRONMENT VARIABLES

Do not confuse environment variables, set with your shell, with temporal env options.

:::

### SDKs

To use your API key with a Temporal SDK, see the instructions in each SDK section.

[How to connect to Temporal Cloud using an API Key with the Go SDK](/develop/go/temporal-client#connect-to-temporal-cloud)

[How to connect to Temporal Cloud using an API Key with the Java SDK](/develop/java/temporal-client#connect-to-temporal-cloud)

[How to connect to Temporal Cloud using an API Key with the Python SDK](/develop/python/temporal-client#connect-to-temporal-cloud)

[How to connect to Temporal Cloud using an API Key with the TypeScript SDK](/develop/typescript/temporal-client#connect-to-temporal-cloud)

[How to connect to Temporal Cloud using an API Key with the .NET SDK](/develop/dotnet/temporal-client#connect-to-temporal-cloud)

### tcld

To use an API key with `tcld`, choose one of these methods:

- Use the `--api-key` flag.
- Set the `TEMPORAL_API_KEY` environment variable in your shell.

:::tip ENVIRONMENT VARIABLES

Do not confuse environment variables, set with your shell, with temporal env options.

:::

### Cloud Ops API

To use an API key with the [Cloud Ops API](/ops), securely pass the API key in your API client. For a complete example,
see
[Cloud Samples in Go](https://github.com/temporalio/cloud-samples-go/blob/1dd4254b6ed1937e361005c0144410e72b8a5542/client/api/apikey.go).

### Terraform Provider

To use an API key with the [Temporal Terraform Provider](/cloud/terraform-provider), pass the API
key as a provider argument.

## Troubleshoot your API key use {#troubleshooting}

**Invalid API key errors**: Check that you copied the key correctly and that it hasn't been revoked or expired.

## API keys: Frequently Asked Questions {#faqs}

**Q: Can I issue and use multiple API keys for the same account?**

A: Yes, you can generate multiple API keys for different services or team members.

**Q: How many API keys can be issued at once?**

A: Up to 10 non-expired keys per user and 20 non-expired keys per Service Account.

**Q: Do API keys expire?**

A: Yes, API keys expire based on the specified expiration date. Temporal recommends rotating API keys periodically.

**Q: Whats the maximum allowed expiration for an API key?**

A: The maximum expiration time for an API key is 2 years.

**Q: What happens if I misplace or lose my API bearer token/secret key?**

A: The full key is displayed only once upon creation for security reasons. If you lose it, generate a new one.

**Q: What is the `Generate API Key` button on the Namespace page?**

A: The `Generate API Key` button on a Namespace page generates an API key with `Admin` permissions for the given
Namespace and the maximum expiration time, which is 2 years. For additional details, refer to
[Namespace-scoped Service Accounts](/cloud/service-accounts#scoped).

---

## Usage and Billing Management

Temporal strives to provide full transparency over billing and costs.
Account Owners and Finance Admins can view their [detailed billing information](https://cloud.temporal.io/billing) at any time.
Use this information to assess your spending patterns, inspect your credit ledger, check your invoice histories, update payment details,
and manage your current plan as needed.
You can see namespace-level cost estimates on the [usage dashboard](https://cloud.temporal.io/usage).

For more information on current Temporal Cloud pricing for Actions, storage, and services/support, please visit our [Pricing](/cloud/pricing) page.

The [billing](https://cloud.temporal.io/billing) page includes the following information.
If you're not on a standard plan, your billing page may show a subset of this list:

- [Current balance](#current-balance): Your balance to date for this billing cycle
- [Recent bill](#recent-bill): The amount of your most recent bill
- [Invoice history](#invoice): Access to all past invoices
- [Credit ledger](#credit-table): A record of all credit related transactions including details on credit grants, purchases, usage, and remaining credits, if applicable
- [Plan](#plans): Your current plan, consumption pricing, and entitlements, with the ability to manage upgrades and downgrades, payment method, and account deletion

The [Usage](https://cloud.temporal.io/usage) page shows the cost breakdown by Namespace.
If your organization separates projects by Namespace -- for architectural reasons, for development/production differentiation, for different products, etc -- you can view individual costs for each Namespace.

## Current balance {#current-balance}

Your current balance card shows the balance for your current billing cycle and the date it was last updated.
This balance adjusts with use and appears on the first line of your Invoices table.

:::note Billing Cycles

Billing cycles normally begin on the first of the month (UTC).
The minimum plan fee for your first month is prorated based on your sign-up date.

:::

## Recent bill {#recent-bill}

The "Recent Bill" card displays the previous bill amount.

![Recent bill card showing a balance of $0.00](/img/cloud/billing/billing-card.png)

- If you pay your invoices through Stripe, you'll see a **Pay Now** button.
  It takes you to the Stripe portal to complete your payment
- If your account is set up for auto-payment, you don’t need manually pay bills.
  However, you can choose to make manual payments whenever you wish

## Invoices {#invoice}

To review your invoices, follow these steps:

1. Click **Billing** on your left-side vertical navigation.
2. Under the **Invoices** section, select and download the invoice(s) you want to review.

The Invoices table shows the following information:

- Date (UTC): The date range covered by the invoice
- Type: The type of invoice, such as credit purchase or cloud usage
- Status: The current status of the invoice, such as paid or pending
- Credit Granted: The total credits added to your account
- Credit Purchase Amount: The amount paid for purchasing credits
- Credit Usage: The credits used during the billing cycle
- Subtotal: The total amount of the invoice before any adjustments
- Balance Due: The amount to pay after applying credits

![Billing page showing Invoices tab](/img/cloud/billing/billing-invoices.png)

You may download your Invoices prior to this calendar month by clicking the download icon by the date.

:::note Current Month Invoice

During the current billing period, your invoice will not be finalized and the download option will not be available.

:::

## Credits {#credit-table}

The following information appears under the credits table:

- Effective At (UTC): The date when the credit grant became effective
- Type: Indicates whether the transaction was a deduction, expiry, or grant
- Amount: The credit amount that was granted, deducted, or expired
- Credits Remaining: The remaining credit available in the account

![Billing page showing Credits tab](/img/cloud/billing/billing-credits.png)

## Cost by Namespace {#cost-by-namespace}

Account Owners and Finance Admins can access a cost column on the Usage page.
This allows you to monitor your cost on a per Namespace basis.
If your organization separates work by Namespace—for development, production, or different products—you can view costs for each.

![Billing page showing Usage](/img/cloud/billing/billing-usage.png)

:::note Cost Breakdown Limitations

Namespace cost details are not available for "last 90 days" or "last 120 days".

Cost breakdowns distribute the total usage cost to namespaces proportionally based on their metered usage. The proration
reflects your effective price, factoring in included Actions/Storage and tiered pricing rates in your Temporal plan.

:::

## Plans {#plans}

Account Owners and Finance Admins can access their Temporal Plan information on the plans page.
For customers on a standard agreement you will be able to:

- View current plan information, pricing details and entitlements
- View other available plans, pricing details and entitlements
- View Pay-as-You-Go pricing rates applicable to your plan
- Upgrade and Downgrade between plans available on a standard agreement

![Billing page showing Plans tab](/img/cloud/billing/billing-plans.png)

Requests to upgrade your plan are processed immediately and you will be billed on a pro-rated basis for that billing period.
Your monthly entitlements will reflect the full volume of included Actions and Storage of the upgrade plan for that billing month.
After an upgrade, a downgrade cannot be processed until the following billing period.

Requests to downgrade will be processed immediately. Billing and entitlements will be backdated to the beginning of the billing period.

## Account Cancellation

The way you created your Temporal account determines how you can cancel your subscription and remove the account.

- **For accounts managed by our sales team**. 
  Please submit a support ticket so we can help you.

- **For accounts created through our self-signup portal**. 
  Account owners can delete their accounts on the Temporal Cloud Billing page, under the **Plan** tab.
  If you're no longer using Temporal Cloud, use the Delete Account button to begin the process.

  - Permanently deleted accounts will immediately cease billing and be scheduled for full deletion within 72 hours.

  - Account Data and Active Storage will be permanently deleted. Retained Storage will be deleted in accordance with its configured retention period.

![Billing page showing the Plan tab. The contents on the tab include "Manage Payment Method" and "Delete Account" buttons. The "Delete Account" button is placed below text asking "No longer using Temporal Cloud?"](/img/cloud/billing/billing-cancel.png)

---

## Authenticate with mTLS certificates

[Temporal Cloud](https://temporal.io/cloud) supports both mTLS and [API key](/cloud/api-keys) authentication for
namespace access.

When using mTLS authentication, each [Worker Process](/workers#worker-process) uses a CA certificate and private key to
connect to Temporal Cloud. Temporal Cloud does not require an exchange of secrets; only the certificates produced by
private keys are used for verification.

:::caution Don't let your certificates expire

An expired root CA certificate invalidates all downstream certificates.

An expired end-entity certificate prevents a [Temporal Client](/encyclopedia/temporal-sdks#temporal-client) from
connecting to a Namespace or starting a Workflow Execution. If the client is on a Worker, any current Workflow
Executions that are processed by that Worker either run indefinitely without making progress until the Worker resumes or
fail because of timeouts.

Temporal Cloud sends [courtesy emails](/cloud/notifications#admin-notifications) prior to certificate expiry.

To update certificates, see
[How to add, update, and remove certificates in a Temporal Cloud Namespace](#manage-certificates).

:::

All certificates used by Temporal Cloud must meet the following requirements.

## Requirements for CA certificates in Temporal Cloud {#certificate-requirements}

Certificates provided to Temporal for your [Namespaces](/namespaces) _must_ meet the following requirements.

### CA certificates

A CA certificate is a type of X.509v3 certificate used for secure communication and authentication. In Temporal Cloud,
CA certificates are required for configuring mTLS.

CA certificates _must_ meet the following criteria:

- The certificates must be X.509v3.
- Each certificate in the bundle must be either a root certificate or issued by another certificate in the bundle.
- Each certificate in the bundle must include `CA: true`.
- A certificate cannot be a well-known CA (such as DigiCert or Let's Encrypt) _unless_ the user also specifies
  certificate filters.
- The signing algorithm must be either RSA or ECDSA and must include SHA-256 or stronger message authentication. SHA-1
  and MD5 cannot be used.
- The certificates cannot be generated with a passphrase.

:::info

A certificate bundle can contain up to 16 CA certificates. A certificate bundle can have a maximum payload size of 32 KB
before base64 encoding.

:::

### End-entity certificates

An end-entity certificate is a type of X.509v3 certificate used by clients to authenticate themselves. Temporal Cloud
lets you limit access to specific end-entity certificates by using [certificate filters](#manage-certificate-filters).

An end-entity (leaf) certificate _must_ meet the following criteria:

- The certificate must be X.509v3.
- Basic constraints must include `CA: false`.
- The key usage must include Digital Signature.
- The signing algorithm must be either RSA or ECDSA and must include SHA-256 or stronger message authentication. SHA-1
  and MD5 cannot be used.

When a client presents an end-entity certificate, and the whole certificate chain is constructed, each certificate in
the chain (from end-entity to the root) must have a unique Distinguished Name.

:::caution

Distinguished Names are _not_ case sensitive; that is, uppercase letters (such as ABC) and lowercase letters (such as
abc) are equivalent.

:::

## How to issue root CA and end-entity certificates {#issue-certificates}

Temporal Cloud authenticates a client connection by validating the client certificate against one or more CA
certificates that are configured for the specified Namespace.

Choose one of the following options to generate and manage the certificates:

### Option 1: You already have certificate management infrastructure

If you have existing certificate management infrastructure that supports issuing CA and end-entity certificates, export
the CA and generate an end-entity certificates using your existing tools.

Ensure that the CA certificate is long-lived and that the end-entity certificate expires before the CA certificate.

Follow the instructions to [upload the CA certificate](/cloud/certificates#update-certificates-using-temporal-cloud-ui)
and [configure your client](/cloud/certificates#configure-clients-to-use-client-certificates) with the end-entity
certificate.

### Option 2: You don't have certificate management infrastructure

If you don't have an existing certificate management infrastructure, we recommend using API keys for authentication
instead. API keys are generally easier to manage than mTLS certs if you're not using certificate management
infrastructure otherwise.

If you still want to use mTLS, issue the CA and end-entity certificates using [tcld](#use-tcld-to-generate-certificates)
or open source tools like OpenSSL or [step CLI](#use-step-cli-to-generate-certificates).

#### Use tcld to generate certificates

You can generate CA and end-entity certificates by using [tcld](/cloud/tcld).

Although Temporal Cloud supports long-lived CA certificates, a CA certificate generated by [tcld](/cloud/tcld) has a
maximum duration of 1 year (`-d 1y`). You must set an end-entity certificate to expire before its root CA certificate,
so specify its duration appropriately.

To create a new CA certificate, use `tcld gen ca`.

```sh
mkdir temporal-certs
cd temporal-certs
tcld gen ca --org temporal -d 1y --ca-cert ca.pem --ca-key ca.key
```

The contents of the generated `ca.pem` should be pasted into the "Authentication" section of your Namespace settings
page.

To create a new end-entity certificate, use `tcld gen leaf`.

```sh
tcld gen leaf --org temporal -d 364d --ca-cert ca.pem --ca-key ca.key --cert client.pem --key client.key
```

You can now use the generated CA certificate (`ca.pem`) with Temporal Cloud and configure your client with these certs
(`client.pem`, `client.key`).

Upload the contents of the `ca.pem` file to the **Authentication** section of your **Namespace** settings.

Follow the instructions to [upload the CA certificate](/cloud/certificates#update-certificates-using-temporal-cloud-ui)
and [configure your client](/cloud/certificates#configure-clients-to-use-client-certificates) with the end-entity
certificate.

#### Use step CLI to generate certificates

Temporal Cloud requires client certificates for authentication and secure communication.
[The step CLI](https://github.com/smallstep/cli) is a popular and easy-to-use tool for issuing certificates.

Before you begin, ensure you have installed smallstep/cli by following the instructions in the
[installation guide](https://github.com/smallstep/cli#installation).

A Certificate Authority (CA) is a trusted entity that issues digital certificates. These certificates certify the
ownership of a public key by the named subject of the certificate. End-entity certificates are issued and signed by a
CA, and they are used by clients to authenticate themselves to Temporal Cloud.

Create a self-signed CA certificate and use it to issue an end-entity certificate for your Temporal Cloud namespace.

##### 1. Create a Certificate Authority (CA)

Create a new Certificate Authority (CA) using step CLI:

```command
step certificate create "CertAuth" CertAuth.crt CertAuth.key --profile root-ca --no-password --insecure
```

This command creates a self-signed CA certificate named `CertAuth.crt` and private key `CertAuth.key`. This CA certificate will be used to sign and issue end-entity certificates.

##### 2. Set the Namespace Name

Set the Namespace Name as the common name for the end-entity certificate:

<Tabs>
  <TabItem value="macos" label="macOS" default>

For Linux or macOS:

```command
export NAMESPACE_NAME=your-namespace
```

</TabItem>
    <TabItem value="windows" label="Windows" default>

For Windows:

```command
set NAMESPACE_NAME=your-namespace
```

</TabItem>
</Tabs>

Replace `your-namespace` with the name of your Temporal Cloud namespace.

##### 3. Create and Sign an End-Entity Certificate

Create and sign an end-entity certificate with a common name equal to the Namespace Name:

```command
step certificate create ${NAMESPACE_NAME} ${NAMESPACE_NAME}.crt ${NAMESPACE_NAME}.key --ca CertAuth.crt --ca-key CertAuth.key --no-password --insecure --not-after 8760h
```

This command creates an end-entity certificate (`your-namespace.crt`) and private key (`your-namespace.key`) that is signed by your CA (`CertAuth`).

##### 4. (optional) Convert to PKCS8 Format for Java SDK

If you are using the Temporal Java SDK, you will need to convert the PKCS1 file format to PKCS8 file format. Export the
end-entity's private key to a PKCS8 file:

```command
openssl pkcs8 -topk8 -inform PEM -outform PEM -in ${NAMESPACE_NAME}.key -out ${NAMESPACE_NAME}.pkcs8.key -nocrypt
```

##### 5. Use the Certificates with Temporal Cloud

You can now use the generated client certificate (`your-namespace.crt`) and the CA certificate (`CertAuth.crt`) with
Temporal Cloud.

Upload the contents of the `CertAuth.crt` file to the **CA Certificates** section of your **Namespace** settings.

Follow the instructions to [upload the CA certificate](/cloud/certificates#update-certificates-using-temporal-cloud-ui)
and [configure your client](/cloud/certificates#configure-clients-to-use-client-certificates) with the end-entity
certificate.

## How to control authorization for Temporal Cloud Namespaces {#control-authorization}

We recommend that an end-entity certificate be scoped to a specific Namespace to enforce the principle of least
privilege. Temporal Cloud requires full CA chains, so you can achieve authorization in two ways.

### Option 1: Issue a separate root certificate for each Namespace

Each certificate must belong to a chain up to the root CA certificate. Temporal uses the root CA certificate as the
trusted authority for access to your Namespaces.

1. Ensure that your certificates meet the [certificate requirements](#certificate-requirements).
1. [Add client CA certificates to a Cloud Namespace](/cloud/tcld/namespace/#add).

### Option 2: Use the same root certificate for all Namespaces but create a separate certificate filter for each Namespace

[How to manage certificate filters in Temporal Cloud](#manage-certificate-filters)

## How to receive notifications about certificate expiration {#expiration-notifications}

To keep your Namespace secure and online, you must update the CA certificate for the Namespace _before_ the certificate
expires.

To help you remember to do so, Temporal Cloud sends [email notifications](/cloud/notifications#admin-notifications). 

## How to handle a compromised end-entity certificate

:::warning

Temporal does not support or check certificate revocation lists (CRLs). Customers are expected to keep their
certificates up to date.

:::

The recommended approach to avoiding compromised certificates is to have short-lived end-entity certificates. A
short-lived compromised certificate can be left to expire on its own. Seek guidance from your infosec team to determine
an appropriate value of "short-lived" for your business.

If you suspect or confirm that an end-entity certificate has been compromised, and leaving it to expire is not an
option, take immediate action to secure your Temporal Cloud Namespace and prevent unauthorized access.

If you're using certificate filters, you can set the filters to block a compromised certificate. Follow the instructions
in [How to manage certificate filters in Temporal Cloud](#manage-certificate-filters).

If you need to replace a compromised certificate manually, follow these steps:

### 1. Generate a new CA certificate

Follow the instructions in [How to issue CA and end-entity certificates](#issue-certificates). All end-entity
certificates that can be reached by the previous CA must be regenerated. Ensure the new CA certificate meets
[the certificate requirements](#certificate-requirements).

### 2. Deploy the new CA certificate to the Namespace

Follow the instructions for issuing a
[new CA certificate to a Namespace](#option-1-issue-a-separate-root-certificate-for-each-namespace). Deploy the new CA
certificate alongside the existing one, so you don’t lose connectivity with old end-entity certificates before the new
ones are generated and deployed.

### 3. Regenerate end-entity certificates with the new CA certificate

[Configure all clients](https://docs.temporal.io/cloud/certificates#configure-clients-to-use-client-certificates)
(Temporal CLI, SDKs, or Workers) to use the new certificate and private key alongside the compromised key. Update the
client configuration as described in
[Configure clients to use client certificates](#configure-clients-to-use-client-certificates).

Test the new certificate to confirm clients can connect to the Namespace without issues.

### 4. Remove the compromised CA certificate

Follow the instructions in
[How to add, update, and remove certificates in a Temporal Cloud Namespace](#manage-certificates) to remove the
compromised CA certificate from the Namespace.

### 5. Monitor and audit

After implementing the changes, monitor your Namespace for unauthorized access attempts or unusual activity in audit
logs. Review your certificate management practices to identify how the compromise occurred. Consider implementing
stricter controls, such as:

- Limiting end-entity certificates to specific Namespaces
- Rotating certificates regularly as a preventive measure
- [Audit logs on a regular schedule](https://docs.temporal.io/cloud/audit-logs)

## How to add, update, and remove certificates in a Temporal Cloud Namespace {#manage-certificates}

:::note

To manage certificates for a Namespace, a user must have [Namespace Admin](/cloud/users#namespace-level-permissions)
permission for that Namespace.

:::

To manage certificates for Temporal Cloud Namespaces, use the **Namespaces** page in Temporal Cloud UI or the
[tcld namespace accepted-client-ca](/cloud/tcld/namespace/#accepted-client-ca) commands.

Don't let your certificates expire! Add reminders to your calendar to issue new CA certificates well before the
expiration dates of the existing ones. Temporal Cloud begins sending notifications 15 days before expiration. For
details, see the previous section
([How to receive notifications about certificate expiration](#expiration-notifications)).

When updating CA certificates, it's important to follow a rollover process (sometimes referred to as "certificate
rotation"). Doing so enables your Namespace to serve both CA certificates for a period of time until traffic to your old
CA certificate ceases. This prevents any service disruption during the rollover process.

{/* How to update certificates in Temporal Cloud using Temporal Cloud UI */}

### Update certificates using Temporal Cloud UI

Updating certificates using the following strategy allows for a zero-downtime rotation of certificates.

1. On the left side of the window, select **Namespaces**.

2. Select the name of the Namespace to update.

3. In the top-right portion of the page for the Namespace, select **Edit**.

4. On the **Edit** page, select the **Authentication** card to expand it.

5. In the certificates box, scroll to the end of the existing certificate (that is, past `-----END CERTIFICATE-----`).

6. On the following new line, paste the entire PEM block of the new certificate.

7. Select **Save**.

8. Wait until all Workers are using the new certificate.

9. Return to the **Edit** page of the Namespace and select the **Authentication** card.

10. In the certificates box, delete the old certificate, leaving the new one in place.

11. Select **Save**.

{/* How to update certificates in Temporal Cloud using tcld */}

### Update certificates using tcld

Updating certificates using the following strategy allows for a zero-downtime rotation of certificates.

1. Create a single file that contains both your old and new CA certificate PEM blocks. Just concatenate the PEM blocks
   on adjacent lines.

   ```
   -----BEGIN CERTIFICATE-----
   ... old CA cert ...
   -----END CERTIFICATE-----
   -----BEGIN CERTIFICATE-----
   ... new CA cert ...
   -----END CERTIFICATE-----
   ```

1. Run the `tcld namespace accepted-client-ca set` command with the CA certificate bundle file.

   ```bash
   tcld namespace accepted-client-ca set --ca-certificate-file <path>
   ```

1. Monitor traffic to your old certificate until it ceases.

1. Create another file that contains only the new CA certificate.

1. Run the `tcld namespace accepted-client-ca set` command again with the updated CA certificate bundle file.

## How to manage certificate filters in Temporal Cloud {#manage-certificate-filters}

To limit access to specific [end-entity certificates](#end-entity-certificates), create certificate filters. Each filter
contains values for one or more of the following fields:

- commonName (CN)
- organization (O)
- organizationalUnit (OU)
- subjectAlternativeName (SAN)

Corresponding fields in the client certificate must match every specified value in the filter.

The values for the fields are case-insensitive. If no wildcard is used, each specified value must match its field
exactly.

To match a substring, place a single `*` wildcard at the beginning or end (but not both) of a value. You cannot use a
`*` wildcard by itself.

You can create a maximum of 25 certificate filters in a Namespace.

If you provide a well-known CA certificate, you cannot clear a certificate filter. A well-known CA certificate is one
that is typically included in the certificate store of an operating system.

**Examples**

In the following example, only the CN field of the certificate's subject is checked, and it must be exactly
`code.example.com`. The other fields are not checked.

```json
AuthorizedClientCertificate {
  CN : "code.example.com"
}
```

In the following example, the CN field must be `stage.example.com` and the O field must be `Example Code Inc.`

```json
AuthorizedClientCertificate {
  CN : "stage.example.com"
  O : "Example Code Inc."
}
```

When using a `*` wildcard, the following values are valid:

- `*.example.com` matches `code.example.com` and `text.example.com`.
- `Example Code*` matches `Example Code` and `Example Code Inc`.

The following values are not valid:

- `.example.*`
- `code.*.com`
- `*`

{/* How to manage certificate filters in Temporal Cloud using Temporal Cloud UI */}

### Manage certificate filters using Temporal Cloud UI

To add or remove a certificate filter, follow these steps:

1. On the left side of the window, click **Namespaces**.
1. On the **Namespaces** page, click the name of the Namespace to manage.
1. On the right side of the page for the selected Namespace, click **Edit**.
1. On the **Edit** page, click the **Authentication** card.
   - To add a certificate filter, click **Add a Certificate Filter** and enter values in one or more fields.
   - To remove a certificate filter, click the **×** in the upper-right corner of the filter details.
1. To cancel your changes, click **Back to Namespace**. To save your changes, click **Save**.

{/* How to manage certificate filters in Temporal Cloud using tcld */}

### Manage certificate filters using tcld

To set or clear certificate filters, use the following [tcld](/cloud/tcld) commands:

- [tcld namespace certificate-filters import](/cloud/tcld/namespace/#import)
- [tcld namespace certificate-filters clear](/cloud/tcld/namespace/#clear)

To view the current certificate filters, use the
[tcld namespace certificate-filters export](/cloud/tcld/namespace/#export) command.

## Configure clients to use client certificates {#configure-clients-to-use-client-certificates}

- [Go SDK](/develop/go/temporal-client#connect-to-temporal-cloud)
- [Java SDK](/develop/java/temporal-client#connect-to-temporal-cloud)
- [PHP SDK](/develop/php/temporal-client#connect-to-a-dev-cluster)
- [Python SDK](/develop/python/temporal-client#connect-to-temporal-cloud)
- [TypeScript SDK](/develop/typescript/temporal-client#connect-to-temporal-cloud)
- [.NET SDK](/develop/dotnet/temporal-client#connect-to-temporal-cloud)

### Configure Temporal CLI {#configure-temporal-cli}

To connect to a Temporal Namespace using the Temporal CLI and certificate authentication, specify your credentials and
the TLS server name:

```sh
temporal <command> <subcommand> \
    --tls-ca-path <Path to server CA certificate> \
    --tls-cert-path <Path to x509 certificate> \
    --tls-key-path <Path to private certificate key> \
    --tls-server-name <Override for target TLS server name>
```

For more information on Temporal CLI environment variables, see [Environment variables](/cli#environment-variables).

---

## Get started with Temporal Cloud

Getting started with Temporal Cloud involves a few key steps:

1. [Sign up for Temporal Cloud](#sign-up-for-temporal-cloud)
1. [Create a Namespace](#create-a-namespace)
1. [Set up your Clients and Workers](#set-up-your-clients-and-workers)
1. [Run your first Workflow](#run-your-first-workflow)
1. [Invite your team](#invite-your-team)

## Sign up for Temporal Cloud

To create a Temporal Cloud account, you can:

- Sign up [directly](https://temporal.io/get-cloud); or
- Subscribe at the AWS Marketplace for
  [Temporal Cloud Pay-As-You-Go](https://aws.amazon.com/marketplace/pp/prodview-xx2x66m6fp2lo). Signing up through the
  AWS Marketplace is similar to signing up directly on the Temporal Cloud site, but billing goes through your AWS
  account.
- To purchase Temporal Cloud on the Google Cloud Marketplace, please contact our team at sales@temporal.io.

For information about Temporal Cloud Pricing, see our [Pricing Page](/cloud/pricing).

## Create a Namespace

See [Managing Namespaces](/cloud/namespaces#create-a-namespace) to create your first namespace.

Temporal Cloud supports either [API key](/cloud/api-keys) or [mTLS](/cloud/certificates) authentication for each
namespace. If you're not sure which to use, we recommend using [API keys](/cloud/api-keys) because they're easier to
manage and rotate for most teams. If your organization already has private key infrastructure (PKI) and is familiar with
cert management, then [mTLS](/cloud/certificates) is an excellent choice.

## Set up your Clients and Workers

See our guides for connecting each SDK to your Temporal Cloud Namespace:

- [Connect to Temporal Cloud in Go](/develop/go/temporal-client#connect-to-temporal-cloud)
- [Connect to Temporal Cloud in Java](/develop/java/temporal-client#connect-to-temporal-cloud)
- [Connect to Temporal Cloud in Python](/develop/python/temporal-client#connect-to-temporal-cloud)
- [Connect to Temporal Cloud in TypeScript](/develop/typescript/core-application#connect-to-temporal-cloud)
- [Connect to Temporal Cloud in .NET](/develop/dotnet/temporal-client#connect-to-temporal-cloud)
- [Connect to Temporal Cloud in PHP](/develop/php/temporal-client#connect-to-temporal-cloud)
- [Connect to Temporal Cloud in Ruby](/develop/ruby/temporal-client#connect-to-temporal-cloud)

## Run your first Workflow

See our guides for starting a workflow using each SDK:

- [Start a workflow in Go](/develop/go/temporal-client#start-workflow-execution)
- [Start a workflow in Java](/develop/java/temporal-client#start-workflow-execution)
- [Start a workflow in Python](/develop/python/temporal-client#start-workflow-execution)
- [Start a workflow in TypeScript](/develop/typescript/core-application#start-workflow-execution)
- [Start a workflow in .NET](/develop/dotnet/temporal-client#start-workflow)
- [Start a workflow in PHP](/develop/php/temporal-client#start-workflow-execution)
- [Start a workflow in Ruby](/develop/ruby/temporal-client#start-workflow)

## Invite your team

See [Managing users](/cloud/users) to add other users and assign them roles.

You can also use [Service Accounts](/cloud/service-accounts) to represent machine identities.

Since you created the account when you signed up, your email address is the first
[Account Owner](/cloud/users#account-level-roles) for your account.

---

## Namespaces

:::info Temporal Cloud
This page covers namespace operations in **Temporal Cloud**.
For core namespace concepts, see [Temporal Namespace](/namespaces).
For open source Temporal, see [Managing Namespaces](/self-hosted-guide/namespaces).
:::

A Namespace is a unit of isolation within Temporal Cloud, providing security boundaries, Workflow management, unique
identifiers, and gRPC endpoints in Temporal Cloud.

- [Create a Namespace](#create-a-namespace)
- [Access a Namespace](#access-namespaces)
- [Manage Namespaces](#manage-namespaces)
- [Delete a Namespace](#delete-a-namespace)
- [Tag a Namespace](#tag-a-namespace)

## What is a Cloud Namespace Name? {#temporal-cloud-namespace-name}

A Cloud Namespace Name is a customer-supplied name for a [Namespace](/namespaces) in Temporal Cloud. Each Namespace
Name, such as `accounting-production`, is unique within the scope of a customer's account. It cannot be changed after
the Namespace is provisioned.

Each Namespace Name must conform to the following rules:

- A Namespace Name must contain at least 2 characters and no more than 39 characters.
- A Namespace Name must begin with a letter, end with a letter or number, and contain only letters, numbers, and the
  hyphen (-) character.
- Each hyphen (-) character must be immediately preceded _and_ followed by a letter or number; consecutive hyphens are
  not permitted.
- All letters in a Namespace Name must be lowercase.

## What is a Temporal Cloud Account ID? {#temporal-cloud-account-id}

A Temporal Cloud Account ID is a unique customer identifier assigned by Temporal Technologies. Each Id is a short string
of numbers and letters like `f45a2`, at least five characters long. This account identifier is retained throughout the
time each customer uses Temporal Cloud.

At times you may need to know your customer Account ID. Accessing the account's Namespaces provides an easy way to
capture this information. Each Temporal Namespace use an Account ID suffix. This is the alphanumeric character string
found after the period in any Temporal Cloud Namespace name.

You can retrieve an Account ID from the [Temporal Cloud](https://cloud.temporal.io) Web UI or by using the `tcld`
utility at a command line interface (CLI). Follow these steps.

<Tabs>

    <TabItem value="webui" label="Web UI">
    Follow these steps to retrieve your Account ID:

    1. Log into Temporal Cloud.
    1. Select your account avatar at the top right of the page.
       A profile dropdown menu appears.
    1. Copy the Cloud Account ID from the menu.
       <ZoomingImage
           src="/img/cloud/cloud-guide/cloud-account-id.png"
           alt="Temporal Cloud user profile dropdown menu. The dropdown includes a Cloud Account ID and copy button, with role, profile and logout options, and links to community resources."
       />

    In this example, the Account ID is `123de`.

    </TabItem>

    <TabItem value="tcldcli" label="tcld">
    1. Use the `tcld` utility to log into an account.

       ```
       tcld login
       ```

       The `tcld` output presents a URL with an activation code at the end. Take note of this code. The utility blocks until the login/activation process completes.

       ```
       Login via this url: https://login.tmprl.cloud/activate?user_code=KTGC-ZPWQ
       ```

       A Web page automatically opens for authentication in your default browser.

    1. Visit the browser.
       Ensure the user code shown by the CLI utility matches the code shown in the Web browser.
       Then, click Confirm in the browser to continue.
       After confirmation, Web feedback lets you know that the CLI "device" is now connected.

    1. Return to the command line.
       Issue the following command.

       ```
       tcld namespace list
       ```

       The CLI tool returns a short JSON packet with your namespace information.
       This is the same list found in the Temporal Cloud Web UI Namespaces list.
       Like the browser version, each Namespace uses an Account ID suffix.

       ```
       {
         "namespaces": [
           "your-namespace.123de",
           "another-namespace.123de"
         ],
         "nextPageToken": ""
       }
       ```

    Each Namespace automatically appends an Account ID suffix to its customer-supplied identifier.
    This five-character-or-longer string appears after the name, separated by a period.
    In this Namespace listing sample, the Account ID is 123de.

    </TabItem>

</Tabs>

## What is a Cloud Namespace Id? {#temporal-cloud-namespace-id}

A Cloud Namespace Id is a globally unique identifier for a [Namespace](/namespaces) in Temporal Cloud. A Namespace Id is
formed by concatenating the following:

1. A [Namespace Name](#temporal-cloud-namespace-name)
1. A period (.)
1. The [Account ID](#temporal-cloud-account-id) to which the Namespace belongs

For example, for the Account ID `123de` and Namespace Name `accounting-production`, the Namespace Id is
`accounting-production.123de`.

## What is a Cloud gRPC Endpoint? {#temporal-cloud-grpc-endpoint}

Temporal Clients communicate between application code and a Temporal Server by sending and receiving messages via the
gRPC protocol. gRPC is a Remote Procedure Call framework featuring low latency and high performance. gRPC provides
Temporal with an efficient, language-agnostic communication framework.

Every Temporal Namespace uses a gRPC endpoint for communication. When migrating to Temporal Cloud, you'll need to switch
the gRPC endpoint in your code from your current hosting, whether self-hosted or locally-hosted, to Temporal Cloud.

A gRPC endpoint appears on the detail page for each Cloud Namespace. Follow these steps to find it:

1. Log into your account on [cloud.temporal.io](https://cloud.temporal.io/namespaces).
2. Navigate to the Namespace list page from the left-side vertical navigation.
3. Tap or click on the Namespace Name to select and open the page for the Namespace whose endpoint you want to retrieve.
4. On the Namespace detail page, click on the "Connect" button in the top right corner of the page.
5. Click the copy icon next to the gRPC address to copy it to your clipboard.

See [How to access a Namespace in Temporal Cloud](/cloud/namespaces/#access-namespaces) for more information on
different gRPC endpoint types and how to access them.

## How to create a Namespace in Temporal Cloud {#create-a-namespace}

:::info

The user who creates a [Namespace](/namespaces) is automatically granted
[Namespace Admin](/cloud/users#namespace-level-permissions) permission for that Namespace.

To create a Namespace, a user must have the Developer, Account Owner, or Global Admin account-level
[Role](/cloud/users#account-level-roles).

:::

:::tip

By default, each account is allocated with a limit of ten Namespaces. As you start using Namespaces by scheduling
Workflows, Temporal Cloud automatically raises your allowance. This automatic adjustment happens whenever all your
Namespaces are in use, up to a maximum of 100 Namespaces. You can request further increases beyond the 100 Namespace
limit by opening a [support ticket](/cloud/support#support-ticket).

:::

### Information needed to create a Namespace

To create a Namespace in Temporal Cloud, gather the following information:

- [Namespace Name](/cloud/namespaces#temporal-cloud-namespace-name), region, and Cloud Provider
- [Retention Period](/temporal-service/temporal-server#retention-period) for the
  [Event History](/workflow-execution/event#event-history) of closed [Workflow Executions](/workflow-execution).
- [CA certificate](/cloud/certificates#certificate-requirements) for the Namespace, if you are using mTLS
  authentication.
- [Codec Server endpoint](/production-deployment/data-encryption#set-your-codec-server-endpoints-with-web-ui-and-cli) to
  show decoded payloads to users in the Event History for Workflow Executions in the Namespace. For details, see
  [Securing your data](/production-deployment/data-encryption).
- [Permissions](/cloud/users#namespace-level-permissions) for each user.

<Tabs>

<TabItem value="namespace-webui" label="Web UI">

### Create a Namespace using Temporal Cloud UI

1. Gather the information listed earlier in
   [Information needed to create a Namespace](#information-needed-to-create-a-namespace).
1. Go to the Temporal Cloud UI and log in.
1. On the left side of the window, click **Namespaces**.
1. On the **Namespaces** page, click **Create Namespace** in the upper-right portion of the window.
1. On the **Create Namespace** page in **Name**, enter the Namespace Name.
1. In **Cloud Provider**, select the cloud provider in which to host this Namespace.
1. In **Region**, select the region in which to host this Namespace.
1. In **Retention Period**, specify a value from 1 to 90 days. When choosing this value, consider your needs for Event
   History versus the cost of maintaining that Event History. Typically, a development Namespace has a short retention
   period and a production Namespace has a longer retention period. (If you need to change this value later, contact
   [Temporal Support](/cloud/support#support-ticket).)
1. Select your authentication method: [API keys](/cloud/api-keys) or [mTLS](/cloud/certificates).
1. If using mTLS authentication, paste the CA certificate for this Namespace.
1. Optional: In **Codec Server**, enter the HTTPS URL (including the port number) of your Codec Server endpoint. You may
   also enable "Pass the user access token with your endpoint" and "Include cross-origin credentials." For details, see
   [Hosting your Codec Server](/production-deployment/data-encryption#set-your-codec-server-endpoints-with-web-ui-and-cli).
1. Click **Create Namespace**.

</TabItem>

<TabItem value="namespace-tcld" label="tcld">

See the [`tcld` namespace create](/cloud/tcld/namespace/#create) command reference for details.

</TabItem>

</Tabs>

## What are some Namespace best practices? {#best-practices}

This section provides general guidance for organizing [Namespaces](/namespaces) across use cases, services,
applications, or domains. Temporal Cloud provides Namespace–as-a-service, so the Namespace is the endpoint. Customers
should consider not only a Namespace naming convention but also how to group or isolate workloads using the Namespace as
a boundary.

Each team can have their own Namespace for improved modularity, security, debugging, and fault isolation. Namespaces
contain the blast radius of misbehaving Workers that may exhaust rate limits. Sensitive Workflow state (PCI data) can be
secured with per-Namespace permissions and encrypted with a separate encryption key.

Temporal Applications in different Namespaces may be connected with [Nexus](/cloud/nexus) by exposing a clean service
contract for others to use with built-in [Nexus access controls](/cloud/nexus/security). Nexus supports cross-team,
cross-domain, multi-region, and multi-cloud use cases.

### Constraints and limitations

Before considering an appropriate Namespace configuration, you should be aware of the following constraints:

- By default, each account is allocated with a limit of ten Namespaces. As you create and use your Namespaces, for
  example by scheduling Workflows, Temporal Cloud automatically raises your limit. Our service identifies your usage
  patterns. It responds by slowly increasing your allowance, up to 100 Namespaces. You can request further increases
  beyond the 100 Namespace limit by opening a [support ticket](/cloud/support#support-ticket).
- Each Namespace has a rate limit, which is measured in Actions per second (APS). A namespace may be throttled when its
  throughput becomes too high. Throttling means limiting the rate at which actions are performed to prevent overloading
  the system. A Namespace's default limit is set at 400 APS and automatically adjusts based on recent usage (over the
  prior 7 days). Your APS limit will never fall below this default value.
- Each Namespace has a default service-level agreement (SLA) of 99.9% uptime.
- You can opt-in to using [High Availability features](https://docs.temporal.io/cloud/high-availability) with a 99.99%
  contractual SLA.
- A Namespace is a security isolation boundary. Access to Temporal by [Worker Processes](/workers#worker-process) is
  permitted at the Namespace level. Isolating applications or environments (development, test, staging, production)
  should take this into consideration.
- A Namespace is provisioned with an endpoint for executing your Workflows. Accessing a Namespace from a Temporal Client
  requires [API keys](/cloud/api-keys) or [mTLS](/cloud/certificates) authentication.
- [Workflow Id](/workflow-execution/workflowid-runid#workflow-id)uniqueness is per Namespace.
- [Task Queue](/task-queue) names are unique per Namespace.
- Closed Workflow retention is per Namespace.
- RBAC [permissions](/cloud/users#namespace-level-permissions) are implemented at the Namespace level.

### General guidance

Namespace configuration requires some consideration. Following are some general guidelines to consider.

- Namespaces are usually defined per use case. A use case can encompass a broad range of Workflow types and a nearly
  unlimited scale of concurrent [Workflow Executions](/workflow-execution).
- Namespaces can be split along additional boundaries such as service, application, domain or even sub-domain.
- Environments such as production and development usually have requirements for isolation. We recommend that each
  environment has its own Namespace.
- Namespaces should be used to reduce the "blast radius" for mission-critical applications.
- Workflows that need to communicate with each other should (for now) be in the same Namespace.
- If you need to share Namespaces across team or domain boundaries, be sure to ensure the uniqueness of Workflow Ids.

### Examples

Following are some ideas about how to organize Namespaces.

#### Example 1: Namespace per use case and environment

We recommend using one Namespace for each use case and environment combination for simple configurations in which
multiple services and team or domain boundaries don't exist.

Sample naming convention:

<pre>&lt;use-case>_&lt;environment></pre>

#### Example 2: Namespace per use case, service, and environment

We recommend using one Namespace for each use case, service, and environment combination when multiple services that are
part of same use case communicate externally to Temporal via API (HTTP/gRPC).

Sample naming convention:

<pre>&lt;use-case>_&lt;service>_&lt;environment></pre>

#### Example 3: Namespace per use case, domain, and environment

We recommend using one namespace per use case, domain, and environment combination when multiple services that are part
of the same use case need to communicate with each another via [Signals](/sending-messages#sending-signals) or by
starting [Child Workflows](/child-workflows). In this case, though, you must be mindful about Workflow Id uniqueness by
prefixing each Workflow Id with a service-specific string. The name of each Task Queue must also be unique. If multiple
teams are involved, the domain could also represent a team boundary.

Sample naming convention:

<pre>&lt;use-case>_&lt;domain>_&lt;environment></pre>

Sample workflowId convention:

<pre>&lt;service-string>_&lt;workflow-id></pre>

## How to access a Namespace in Temporal Cloud {#access-namespaces}

{/* How to access a Namespace in Temporal Cloud */}

Temporal Cloud normally supports authentication to Namespaces using [API keys](/cloud/api-keys) _or_
[mTLS](/cloud/certificates). If you need to migrate from one authentication method to another, or you require both API
key and mTLS authentication to be enabled on your Namespace, please contact
[Support](https://docs.temporal.io/cloud/support#support-ticket).

:::info

Namespace authentication requiring both API key and mTLS is in
[pre-release](/evaluate/development-production-features/release-stages), and doesn't support
[High Availability features](/cloud/high-availability).

:::

See the documentation for [API keys](/cloud/api-keys) and [mTLS certificates](/cloud/certificates) for more information
on how to create and manage your credentials.

Programmatically accessing your Namespace requires specific endpoints based on your authentication method. There are two
types of gRPC endpoints for accessing a Namespace in Temporal Cloud:

- A namespace endpoint (`<namespace>.<account>.tmprl.cloud:7233`)
- A regional endpoint (`<region>.<cloud_provider>.api.temporal.io:7233`).

Which one to use depends on your authentication method and whether your Namespace has
[High Availability features](/cloud/high-availability) enabled, as shown in the table below.

|                        | Not High Availability | High Availability                                                                                                            |
| ---------------------- | --------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| mTLS Authentication    | Namespace             | Namespace                                                                                                                    |
| API Key Authentication | Regional              | Both work, but we reommend using the Namespace endpoint because it reduces the unavailability window during a failover event |

:::info

When switching on or off High Availability features for a Namespace, you may need to update the gRPC endpoint used by
your clients and Workers, because the Namespace endpoint changes based on whether High Availability features are
enabled. See [Disable High Availability](/cloud/high-availability/enable#disable) for more information.

:::

For information on how to connect to Clients using a specific authentication method see the following documentation.

- To use API keys to connect with the [Temporal CLI](/cli), [Client SDK](/develop), [tcld](/cloud/tcld),
  [Cloud Ops API](/ops), and [Terraform](/cloud/terraform-provider), see
  [Use API keys to authenticate](/cloud/api-keys#using-apikeys).
- To use mTLS to connect with the [Temporal CLI](/cli) and [Client SDK](/develop), see
  [Configure Clients to use Client certificates](/cloud/certificates#configure-clients-to-use-client-certificates).

For accessing the Temporal Web UI, use the HTTPS endpoint in the form:
`https://cloud.temporal.io/namespaces/<namespace>.<account>`. For example:
`https://cloud.temporal.io/namespaces/accounting-production.f45a2`.

To ensure the security of your data, all traffic to and from your Namespace is encrypted. However, for enhanced
protection, you have additional options:

- (Recommended) Set up private connectivity by [creating a ticket for Temporal Support](/cloud/support#support-ticket).
- Set up your allow list for outgoing network requests from your Clients and Workers with the IP address ranges of the
  Cloud Provider region in which your Namespace is located:
  - [AWS IP address ranges](https://docs.aws.amazon.com/vpc/latest/userguide/aws-ip-ranges.html)
  - [GCP IP address ranges](https://cloud.google.com/compute/docs/faq#find_ip_range)

## How to manage Namespaces in Temporal Cloud {#manage-namespaces}

{/* How to manage Namespaces in Temporal Cloud using Temporal Cloud UI */}

### Manage Namespaces in Temporal Cloud using Temporal Cloud UI

To list Namespaces:

- On the left side of the window, select **Namespaces**.

To edit a Namespace (including custom Search Attributes, certificates, certificate filters, Codec Server endpoint,
permissions, and users), find the Namespace and do either of the following:

- On the right end of the Namespace row, select the three vertical dots (⋮). Click **Edit**.
- Select the Namespace name. In the top-right portion of the page, select **Edit**.

On the **Edit** page, you can do the following:

- Add a [custom Search Attribute](/search-attribute#custom-search-attribute).
- [Manage CA certificates](/cloud/certificates).
- [Manage certificate filters](/cloud/certificates#manage-certificate-filters-using-temporal-cloud-ui).
- Set
  [Codec Server endpoint](/production-deployment/data-encryption#set-your-codec-server-endpoints-with-web-ui-and-cli)
  for all users on the Namespace. Each user on the Namespace has the option to
  [override this setting](/production-deployment/data-encryption#web-ui) in their browser.
- Manage [Namespace-level permissions](/cloud/users#namespace-level-permissions).
- Add users.

To add a user to a Namespace, scroll to the bottom of the page and select **Add User**.

After you make changes, select **Save** in the top-right or bottom-left portion of the page.

{/* How to manage Namespaces in Temporal Cloud using tcld */}

### Manage Namespaces in Temporal Cloud using tcld

To list Namespaces and get information about them, use the following [tcld](/cloud/tcld/) commands:

- [tcld namespace list](/cloud/tcld/namespace/#list)
- [tcld namespace get](/cloud/tcld/namespace/#get)

To manage certificates, use the [tcld namespace accepted-client-ca](/cloud/tcld/namespace/#accepted-client-ca) commands.
For more information, see [How to manage certificates in Temporal Cloud](/cloud/certificates).

To manage certificate filters, use the [tcld namespace certificate-filters](/cloud/tcld/namespace/#certificate-filters)
commands. For more information, see
[How to manage certificate filters in Temporal Cloud](/cloud/certificates#manage-certificate-filters).

## How to delete a Namespace in Temporal Cloud {#delete-a-namespace}

:::info

To delete a Namespace, a user must have Namespace Admin [permission](/cloud/users#namespace-level-permissions) for that
Namespace.

:::

### Delete a Namespace using Temporal Cloud UI

1. Go to the Temporal Cloud UI and log in.
1. On the left side of the window, select **Namespaces**.
1. On the **Namespaces** page, select a Namespace Name.
1. On the Namespace page, select **Edit** in the upper-right portion of the window.
1. On the **Edit** Namespace page, select **Delete Namespace** in the upper-right portion of the window.
1. In the **Delete Namespace** dialog, type `DELETE` to confirm the deletion of that Namespace.
1. Select **Delete**.

After deleting a Temporal Cloud Namespace, the Temporal Service immediately removes the Namespace's Workflow Executions
and Task Queues. Make sure all Workflows have been completed, canceled, or terminated before removing a Namespace. The
Namespace removal is permanent.

Closed Workflow Histories remain in Temporal storage until the user-defined retention period expires. This period
reflects the policy in effect when the Workflow Execution was closed.

For further questions or concerns, contact [Support](https://docs.temporal.io/cloud/support#support-ticket).

### Delete a Namespace using tcld

See the [tcld namespace delete](/cloud/tcld/namespace/#delete) command reference for details.

### Namespace deletion protection {#delete-protection}

To prevent accidental Namespace deletion, Temporal Cloud provides a protection feature. When you enable Deletion
Protection for your production environment Namespace, you ensure that critical data won't be deleted unintentionally.

Follow these steps:

- Visit the [Namespaces page](https://cloud.temporal.io/namespaces) on Temporal Cloud.
- Open your Namespace details page.
- Select the Edit button.
- Scroll down to Security and click the disclosure button (downward-facing caret).
- Enable **Deletion Protection**

<CaptionedImage
  src="/img/cloud/namespace/deletion-protection.png"
  title="Deletion Protection is enabled by toggling the switch"
/>

To enable or disable this feature using [`tcld`](/cloud/tcld), use the following command. Set the value to `true` to
enable or `false` to disable:

```
tcld namespace lifecycle set \
    --namespace <namespace_id.account_id> \
    --enable-delete-protection <Boolean>
```

## How to tag a Namespace in Temporal Cloud {#tag-a-namespace}

Tags are key-value metadata pairs that can be attached to namespaces in Temporal Cloud to help operators organize,
track, and manage namespaces more easily.

### Tag Structure and Limits

- Each namespace can have a maximum of 10 tags
- Each key must be unique for a given namespace (e.g., a namespace cannot have both `team:foo` and `team:bar` tags)
- Keys and values must be 1-63 characters in length
- Allowed characters: lowercase letters (`a-z`), numbers (`0-9`), periods (`.`), underscores (`_`), hyphens (`-`), and
  at signs (`@`)
- Tags are not a secure storage mechanism and should not store PII or PHI
- Tags will not change the behavior of the tagged resource
- There is a soft limit of 1000 unique tag keys per account

### Permissions

- Only [**Account Admins** and **Account Owners**](/cloud/users#account-level-roles) can create and edit tags
- All users with access to a namespace can view its tags

### tcld

See the [tcld namespace tags](/cloud/tcld/namespace/#tags) command reference for details.

### Terraform

See the
[Terraform provider](https://github.com/temporalio/terraform-provider-temporalcloud/blob/main/docs/resources/namespace_tags.md)
for details.

### Web UI

Tags can be viewed and managed through the Temporal Cloud web interface. When viewing a namespace, you'll see tags
displayed and can add, edit, or remove them if you have the appropriate permissions.

<CaptionedImage src="/img/cloud/tags/Namespace-DetailsWithTags.png" title="Tags appear in namespace details" />

<CaptionedImage src="/img/cloud/tags/Namespaces-IndexWithTags.png" title="Tags appear on the list of namespaces" />

<CaptionedImage
  src="/img/cloud/tags/CreateNamespace-AddNewTag.png"
  title="Where to add tags during namespace creation"
/>

<CaptionedImage
  src="/img/cloud/tags/CreateNamespace-AddedTag.png"
  title="After adding a tag during namespace creation"
/>

---

## Manage service accounts

Temporal Cloud provides Account Owner and Global Admin [roles](/cloud/users#account-level-roles) with the option to create machine identities named Service Accounts.

Service Accounts are a type of identity in Temporal Cloud.
Temporal Cloud supports User identities as a representation of a human user who uses Temporal Cloud.
Service Accounts afford Temporal Cloud Account Owner and Global Admin [roles](/cloud/users#account-level-roles) the ability to create an identity for machine authentication, an identity not associated with a human user.

With the addition of Service Accounts, Temporal Cloud now supports 2 identity types:

- Users (tied to a human, identified by email address or ID)
- Service Accounts (not tied to a human, email address optional, identified by name or ID)

Service Accounts use API Keys as the authentication mechanism to connect to Temporal Cloud.
You should use Service Accounts to represent a non-human identity when authenticating to Temporal Cloud for operations automation or the Temporal SDKs and the Temporal CLI for Workflow Execution and management.

:::tip

Namespace Admins can now manage and create [Namespace-scoped Service Accounts](/cloud/service-accounts#scoped), regardless of their Account Role.

:::

## Manage Service Accounts

Account Owner and Global Admin [roles](/cloud/users#account-level-roles) can manage Service Accounts by creating, viewing, updating, deleting Service Accounts using the following tools:

- Temporal Cloud UI
- Temporal Cloud CLI (tcld)
  - Use `tcld service-account --help` for a list of all service-account commands

Account Owner and Global Admin [roles](/cloud/users#account-level-roles) also have the ability to manage API Keys for Service Accounts.

### Prerequisites

- A Cloud user account with Account Owner or Global Admin [role](/cloud/users#account-level-roles) permissions
- Access to the Temporal Cloud UI or Temporal Cloud CLI (tcld)
- Enable access to API Keys for your Account
- To manage Service Accounts using the Temporal Cloud CLI (tcld), upgrade to the latest version of tcld (v0.18.0 or higher) using `brew upgrade tcld`.
  - If using a version of tcld less than v0.31.0, enable Service Account commands with `tcld feature toggle-service-account`.

### Create a Service Account

Create a Service Account using the Temporal Cloud UI or tcld.
While User identities are invited to Temporal Cloud, Service Accounts are created in Temporal Cloud.

<Tabs groupId="service-account">
  <TabItem value="cloud-ui" label="Using the Cloud UI">

1. Go to [Settings → Identities](https://cloud.temporal.io/settings/identities)
2. Click the `Create Service Account` button located near the top of the `Identities` page
3. Provide the following information:
   - **Name** (required)
   - **Description** (optional)
   - **Account Level Role** (required)
   - **Namespace Permissions** (optional)
     - Use this section of the Create Service Account page to grant the Service Account access to individual Namespaces
4. Click `Create Service Account` at the bottom of the page
   - A status message is displayed at the bottom right corner of the screen and on the next screen
   - You will be prompted to create an API Key for the Service Account (optional)
5. (Optional) Create API Key
   - It is recommended to create an API Key for the Service Account right after you create the Service Account, though you can create/manage API Keys for Service Accounts at any time
   - See the API Key [documentation](/cloud/api-keys) for more information on creating and managing API Keys

  </TabItem>
  <TabItem value="tcld" label="Using tcld">

To create a Service Account using tcld, use the `tcld service-account create` command:

```
tcld service-account create -n "sa_test" -d "this is a test SA" --ar "Read"
```

This example creates a Service Account with the name `"sa_test"`, description `"this is a test SA"`, and a `Read` Account Role.

Creating a Service Account requires the following attributes: `name` and `account-role` (as above).
You can also provide the Namespace Permissions for the Service Account using the `—-np` flag.
Creating a Service Account returns the `ServiceAccountId` which is used to retrieve, update, or delete a Service Account.

  </TabItem>
</Tabs>

### View Service Accounts

View a single or all Service Account(s) using the Temporal Cloud UI or tcld.

<Tabs groupId="service-account">
  <TabItem value="cloud-ui" label="Using the Cloud UI">

Service Accounts are listed on the `Identities` section of the `Settings` page, along with Users.
To locate a Service Account:

1. Go to [Settings → Identities](https://cloud.temporal.io/settings/identities)
2. Select the `Service Accounts` filter

  </TabItem>
  <TabItem value="tcld" label="Using tcld">

To view all Service Accounts in your account using tcld, use the `tcld service-account list` command:

```
tcld service-account list
```

  </TabItem>
</Tabs>

### Delete a Service Account

Delete a Service Account using the Temporal Cloud UI or tcld. When you delete a Service Account, all associated API keys are automatically deleted as well.
Therefore, you don't need to manually remove API keys after deleting a Service Account.

<Tabs groupId="service-account">
  <TabItem value="cloud-ui" label="Using the Cloud UI">

1. Go to [Settings → Identities](https://cloud.temporal.io/settings/identities)
2. Find the relevant Service Account
3. Select the vertical ellipsis menu in the Service Account row
4. Select `Delete`
5. Confirm the delete action when prompted

  </TabItem>
  <TabItem value="tcld" label="Using tcld">

To delete a Service Account using tcld, use the `tcld service-account delete` command:

```
tcld service-account delete --service-account-id "e9d87418221548"
```

Use the tcld Service Account list command to validate the Service Account has been removed from the account.
The Service Account is deleted when it is no longer visible in the output of .

  </TabItem>
</Tabs>

### Update a Service Account {#update}

Update a Service Account's description using the Temporal Cloud UI or tcld.

<Tabs groupId="service-account">
  <TabItem value="cloud-ui" label="Using the Cloud UI">

1. Go to [Settings → Identities](https://cloud.temporal.io/settings/identities)
2. Find the relevant Service Account
3. Select the vertical ellipsis menu in the Service Account row
4. Select `Edit`
5. Make changes to the Service Account
   - You can change the Service Account's name, description, Account Level Role, and Namespace Permissions
6. Click the `Save` button located in the bottom left of the screen
   - A status message is displayed at the bottom right corner of the screen

  </TabItem>
  <TabItem value="tcld" label="Using tcld">

Three different commands exist to help users update a Service Account using tcld:

- `tcld service-account update`: to update a Service Account's name or description field
- `tcld service-account set-account-role`: to update a Service Account's Account Role
- `tcld service-account set-namespace-permissions`: to update a Service Account's Namespace Permissions

Example:

```
tcld service-account update --id "2f68507677904e09b9bcdbf93380bb95" -d "new description"
```

  </TabItem>
</Tabs>

## Namespace-scoped Service Accounts {#scoped}

There is a special type of Service Account, called a Namespace-scoped Service Account, which shares the
same functionality as the Service Accounts above, but is limited (or scoped) to a single namespace.

In particular, a Namespace-scoped Service Account must _always_ have:

- A `Read` Account Role
- A single Namespace Permission

Note that a Namespace-scoped Service Account cannot be reassigned to a different Namespace after creation, but its Namespace permission can be modified (e.g. from `Read` to `Write`).

Namespace-scoped Service Accounts are useful in situations when you need to restrict a client's access to a single Namespace.

You can retrieve, update, and delete a Namespace-scoped Service Account using the same process and commands as above, but creation is slightly different.

### Permissions
Unlike regular Service Accounts, which require a Global Admin or Account Owner role, Namespace-scoped Service Accounts can be created and managed by Namespace Admins.
For example, an Account Developer with Namespace Admin for `test_ns` can create a Service Account scoped to `test_ns`.

Global Admins and Account Owners can also create Namespace-scoped Service Accounts, as they implicitly have Namespace Admin rights for all Namespaces.

### Create a Namespace-scoped Service Account

As with regular Service Accounts, Namespace-scoped Service Accounts can be created using Temporal Cloud UI or tcld.

#### Using the Cloud UI {#scoped-ui}

Currently, creating a Namespace-scoped Service Account from the Temporal Cloud UI happens on an individual [Namespace](/cloud/namespaces#manage-namespaces) page.
If the current Namespace has API key authentication enabled, then there will be a `Generate API Key` button as a banner on the top of the Namespace page or in the `Authentication` section.

By clicking on the `Generate API Key` button, a Namespace-scoped Service Account will be automatically created for the given Namespace (if one does not already exist) and an associated API key will be displayed. This key will have the maximum expiration time, which is 2 years.

The resulting Namespace-scoped Service Account will be named `<namespace>-service-account` and will have an `Admin` Namespace permission by default.

#### Using tcld

To create a Namespace-scoped Service Account with tcld, use the `tcld service-accounted create-scoped` command:

```
tcld service-account created-scoped -n "test-scoped-sa" --np "test-ns=Admin"
```

This example creates a Namespace-scoped Service Account for the Namespace `test-ns`, named `test-scoped-sa`, with `Admin` Namespace Permission.
Note that the Account Role is omitted, since Namespace-scoped Service Accounts always have a `Read` Account Role.

### Lifecycle

When a Namespace is deleted, all associated Namespace-scoped Service Accounts and their associated API keys are automatically deleted as well.
Therefore, you do not need to manually remove Namespace-scoped Service Accounts and their API keys after deleting a Namespace.

---

## Manage user groups

## What are user groups?

User groups can be used to help manage sets of users that should have the same
access. Instead of separately assigning the same role to individual users, a user group can be
created, assigned the desired roles, and then users added to the user group. This
eases the toil of managing individual user permissions and can simplify access management. When
a new role is needed, it can be added to the group once and all users' access will reflect the
new role.

User groups can be assigned both [account-level roles](/cloud/users#account-level-roles) and [namespace-level permissions](/cloud/users#namespace-level-permissions).

One user can be assigned to many groups. In the event that a user's group memberships have multiple roles for the same resource, the user will have an effective role of the most permissive of the permissions. For example if `Group A` grants a read-only role to a namespace, but `Group B` grants a write role to a namespace then a user that belongs to both `Group A` and `Group B` would have the write role to the namespace.

[Service accounts](/cloud/service-accounts) cannot be assigned to user groups.

Only users with the Account Owner or Global Admin account-level [role](/cloud/users#account-level-roles) can manage user groups.

## How SCIM groups work with user groups {#scim-groups}

[SCIM groups](/cloud/scim) work similarly to user groups with respect to role assignment. Unlike a user group, the lifecycle of a SCIM group is fully managed by the SCIM integration which means:

1. SCIM groups cannot be created except through the SCIM integration
1. SCIM groups cannot be deleted except through the SCIM integration
1. SCIM group membership is managed through the SCIM integration

User groups and SCIM groups can be used simultaneously in a single Temporal Cloud account. One user may belong to multiple SCIM groups and to multiple user groups.

Using user group and SCIM groups together can be useful when the groups defined in the identity provider (IDP) don't map cleanly to the access you need to grant in Temporal Cloud. Instead of having to update the IDP (which is often sensitive and time-consuming), you can use Temporal Cloud user groups to manage access.

:::info

All user group administration requires an Account Owner or Global Admin account-level [role](/cloud/users#account-level-roles).

:::

## How to create a user group in your Temporal Cloud account {#create-group}

User group names must be 3-64 characters long and can only contain lowercase letters, numbers, hyphens, and underscores.

<Tabs>

<TabItem value="create-group-webui" label="Web UI">

1. Navigate to the [identities page](https://cloud.temporal.io/settings/identities)
1. Click the Create Group button
1. Name the group
1. Assign an account-level role to the group (you can assign namespace-level permissions after the group is created)
1. Click Save

</TabItem>

<TabItem value="create-group-tcld" label="tcld">
See the [`tcld` user-group create](/cloud/tcld/user-group/#create) command reference for details.
</TabItem>

<TabItem value="create-group-tf" label="Terraform">
See the [Terraform provider documentation](https://registry.terraform.io/providers/temporalio/temporalcloud/latest/docs/resources/group) for details.
</TabItem>

</Tabs>

## How to assign roles to a user group {#assign-group-roles}

<Tabs>
<TabItem value="assign-roles-webui" label="Web UI">

To edit the account role of a group:
  1. Navigate to the [identities page](https://cloud.temporal.io/settings/identities)
  1. Find the group to edit (You can filter the list of identities to only show groups to find the relevant group by clicking the Groups tab on the table)
  1. Click Edit Group
  1. Click the Account Role dropdown
  1. Select a new account role
  1. Click Save

To add namespace permissions to a group:
  1. Navigate to the [identities page](https://cloud.temporal.io/settings/identities)
  1. Find the group to edit (You can filter the list of identities to only show groups to find the relevant group by clicking the Groups tab on the table)
  1. Click Edit Group
  1. Click Add Namespaces
  1. Under Grant Access to a Namespace, search for the namespace you’d like to add permissions for
  1. Select the namespace
  1. Click the pencil to edit the permissions for the selected namespace
  1. Click Save

To edit or remove namespace permissions from a group:
  1. Click Edit Group
  1. Click the pencil on a permission to edit it, or the trash can to delete it
  1. Click Save

</TabItem>

<TabItem value="assign-roles-tcld" label="tcld">
See the [`tcld` user-group set-access](/cloud/tcld/user-group/#set-access) command reference for details.
</TabItem>

<TabItem value="assign-roles-tf" label="Terraform">
See the [Terraform provider documentation](https://registry.terraform.io/providers/temporalio/temporalcloud/latest/docs/resources/group) for details.
</TabItem>

</Tabs>

## How to manage users in a group {#assign-group-members}

<Tabs>
<TabItem value="assign-group-members-webui" label="Web UI">

To add users to the group:
  1. Navigate to the [identities page](https://cloud.temporal.io/settings/identities)
  1. Find the group to edit (You can filter the list of identities to only show groups to find the relevant group by clicking the Groups tab on the table)
  1. Click Edit Group
  1. Under Members, search for the user you’d like to add 
  1. Select the user
  1. Click Save
To remove a user from the group:
  1. Click Edit Group
  1. Under Members, click the X next to the user you’d like to remove
  1. Click Save

</TabItem>

<TabItem value="assign-group-members-tcld" label="tcld">
See the [`tcld` user-group add-users](/cloud/tcld/user-group/#add-users) and the [`tcld` user-group remove-users](/cloud/tcld/user-group/#remove-users) command reference for details.
</TabItem>

<TabItem value="assign-group-members-tf" label="Terraform">
See the [Terraform provider documentation](https://registry.terraform.io/providers/temporalio/temporalcloud/latest/docs/resources/group) for details.
</TabItem>

</Tabs>

## Delete a user group

<Tabs>
<TabItem value="delete-group-webui" label="Web UI">

  1. Navigate to the [identities page](https://cloud.temporal.io/settings/identities)
  1. Find the group to edit (You can filter the list of identities to only show groups to find the relevant group by clicking the Groups tab on the table)
  1. Click the dropdown next to the edit button
  1. Click Delete
  1. Confirm by clicking Delete

</TabItem>

<TabItem value="delete-group-tcld" label="tcld">
See the [`tcld` user-group delete](/cloud/tcld/user-group/#delete) command reference for details.
</TabItem>

<TabItem value="delete-group-tf" label="Terraform">
See the [Terraform provider documentation](https://registry.terraform.io/providers/temporalio/temporalcloud/latest/docs/resources/group) for details.
</TabItem>

</Tabs>

---

## User management

:::caution
Access to Temporal Cloud can be authorized through email and password, Google single sign-on, Microsoft single sign-on, or SAML, depending on your setup.

If you are using Google OAuth for single sign-on and an email address is not associated with a Google Account, the user must follow the instructions in the [Use an existing email address](https://support.google.com/accounts/answer/27441?hl=en#existingemail) section of [Create a Google Account](https://support.google.com/accounts/answer/27441).

**Important:** Do _not_ create a Gmail account when creating a Google Account.

If your organization uses Google Workspace or Microsoft Entra ID, and your IT administrator has enabled controls over single sign-on permissions, then you will need to work with your IT administrator to allow logins to Temporal Cloud.

:::

When a user is created in Temporal Cloud, they receive an invitation email with a link.
They must use this link to finalize their setup and access Temporal Cloud.
Accounts with SAML configurations can ignore this email.
However, those using Google/Microsoft SSO or email and password authentication need to accept the invitation link for their initial login to Temporal Cloud.
For future logins, they must use the same authentication method they originally signed up with.

:::info

To invite users, a user must have the Global Admin or Account Owner account-level [role](/cloud/users#account-level-roles).

:::

### Roles and permissions

Each user in Temporal Cloud is assigned a role.
Each user can be assigned permissions for individual Namespaces.

- [Account-level roles](/cloud/users#account-level-roles)
- [Namespace-level permissions](/cloud/users#namespace-level-permissions)

<Tabs>

<TabItem value="invite-webui" label="Web UI">

To invite users using the Temporal Cloud UI:

1. In Temporal Web UI, select **Settings** in the left portion of the window.
1. On the **Settings** page, select **Create Users** in the upper-right portion of the window.
1. On the **Create Users** page in the **Email Addresses** box, type or paste one or more email addresses.
1. In **Account-Level Role**, select a [Role](/cloud/users#account-level-roles).
   The Role applies to all users whose email addresses appear in **Email Addresses**.
1. If the account has any Namespaces, they are listed under **Grant access to Namespaces**.
   To add a permission, select the checkbox next to a Namespace, and then select a [permission](/cloud/users#namespace-level-permissions).
   Repeat as needed.
1. When all permissions are assigned, select **Send Invite**.

Temporal sends an email message to each user.
To join Temporal Cloud, a user must select **Accept Invite** in the message.

</TabItem>

<TabItem value="invite-tcld" label="tcld">

To invite users using tcld, see the [tcld user invite](/cloud/tcld/user/#invite) command.

Temporal sends an email message to the specified user.
To join Temporal Cloud, the user must select **Accept Invite** in the message.

</TabItem>

<TabItem value="invite-ops-api" label="Cloud Ops API">

You can invite users pragmatically using the Cloud Ops API.

1. Create a connection to your Temporal Service using the Cloud Operations API.
2. Use the [CreateUser service](https://github.com/temporalio/api-cloud/blob/main/temporal/api/cloud/cloudservice/v1/service.proto) to create a user.

</TabItem>

</Tabs>

### Frequently Asked Questions

#### Can the same email be used across different Temporal Cloud accounts?

No — each email address can only be associated with a single Temporal Cloud account.
If you need access to multiple accounts, you’ll need a separate invite for each one using a different email address.

#### Can I use Google or Microsoft SSO after signing up with email and password?

If you originally signed up for Temporal Cloud using an email and password, you won’t be able to log in using Google or Microsoft single sign-on.

If you prefer SSO, ask your Account Owner to delete your current user and send you a new invitation.
During re-invitation, be sure to sign up using your preferred authentication method.

#### How do I complete the `Secure Your Account` step?

If you signed up to Temporal Cloud using an email and password, you're required to set up multi-factor authentication (MFA) for added security.
Currently, only authenticator apps are supported as an additional factor (such as Google Authenticator, Microsoft Authenticator, and Authy).

To proceed:

1. Download a supported authenticator app on your mobile device.
2. Scan the QR code shown on the **Secure Your Account** screen.
3. Enter the verification code from your app to complete MFA setup.
4. Securely store your recovery code.
   This code allows you to access your account if you lose access to your authenticator app.

Once MFA is configured, you’ll be able to continue using Temporal Cloud.

#### What if I lose access to my authenticator app?

If you lose access to your authenticator app, you can still log in by clicking **Try another method** on the MFA screen.
From there, you can either:

- Enter your recovery code (provided when you first set up MFA)
- Receive a verification code through email

Once you're logged in, you can reset your authenticator app by navigating to **My Profile** > **Password and Authentication** and then clicking **Authenticator App** > **Remove method**.

#### How do I reset my password?

If you're currently logged in and would like to change your password, click your profile icon at the top right of the Temporal Cloud UI,
navigate to **My Profile** > **Password and Authentication**, and then click **Reset Password**.

If you're not currently logged in, navigate to the login page of the Temporal Cloud UI, enter your email address, click **Continue**, and then select **Forgot password**.
In both cases, you will receive an email with instructions on how to reset your password.

---

## Manage users

- [How to invite users to your Temporal Cloud account](#invite-users)
- [What are the account-level roles?](#account-level-roles)
- [What are the Namespace-level permissions?](#namespace-level-permissions)
- [How to update an account-level Role in Temporal Cloud](#update-roles)
- [How to update Namespace-level permissions in Temporal Cloud](#update-permissions)
- [How to delete a user from your Temporal Cloud account](#delete-users)
- [How to troubleshoot account access issues](#troubleshoot-access)

## How to invite users to your Temporal Cloud account {#invite-users}

<InvitationContent />

## What are the account-level roles for users in Temporal Cloud? {#account-level-roles}

When an Account Owner or Global Admin invites a user to join an account, they select one of the following roles for that user:

- **Global Admin**
  - Has full administrative permissions across the account, including users and usage
  - Can create and manage [Namespaces](/namespaces) and [Nexus Endpoints](/nexus/endpoints)
  - Has Namespace Admin [permissions](#namespace-level-permissions) on all Namespaces in the account.
    This permission cannot be revoked
- **Developer**
  - Can create Namespaces
  - Is granted [Namespace Admin](/cloud/users#namespace-level-permissions) permission for each Namespace they create.
    This permission can be revoked
  - Can create and manage Nexus Endpoints where they are a [Namespace Admin](/cloud/users#namespace-level-permissions) on the Endpoint's target Namespace
- **Read-Only**
  - Can read information
  - Can be granted Namespace [permissions](#namespace-level-permissions), for example to read or write Workflow state in a given Namespace
  - Can view all Nexus Endpoints in the account, which have separate [runtime access controls](/nexus/security#runtime-access-controls)

In addition, there are two roles that the Global Admin cannot assign:

- **Account Owner**
  - Has full administrative permissions across the account, including users, usage and [billing](/cloud/billing-and-cost)
  - Can create and manage Namespaces and Nexus Endpoints
  - Has Namespace Admin [permissions](#namespace-level-permissions) on all [Namespaces](/namespaces) in the account.
    This permission cannot be revoked
- **Finance Admin**
  - Has permissions to view [billing](/cloud/billing-and-cost) information and update payment information
  - Otherwise, has the same permissions as Account Read-only users
  - Can be assigned to Service Accounts by a Global Admin, but otherwise can only be assigned by an Account Owner

:::note Default Role

When the account is created, the initial user who logs in is automatically assigned the Account Owner role.
If your account does not have an Account Owner, please reach out to [Support](https://temporalsupport.zendesk.com/) to assign the appropriate individual to this role.

:::

## Using the Account Owner role

The Account Owner role (i.e., users with the Account Owner system role) holds the highest level of access in the system.
This role configures account-level parameters and manages Temporal billing and payment information.
It allows users to perform all actions within the Temporal Cloud account.

:::tip Best Practices

Temporal strongly recommends the following precautions when assigning the Account Owner role to users:

- Assign the role to at least two users in your organization.
  Otherwise, limit the number of users with this role.
- Associate a person’s direct email address to the Account Owner, rather than a shared or generic address, so Temporal Support can contact the right person in urgent situations.

This latter rule is useful for anyone on your team who may need to be contacted urgently, regardless of their Account role.

:::

## What are the Namespace-level permissions for users in Temporal Cloud? {#namespace-level-permissions}

An Account Owner or Global Admin can assign permissions for any [Namespace](/namespaces) in an account.
A Developer can assign permissions for a Namespace they create.

For a Namespace, a user can have one of the following permissions:

- **Namespace Admin:**
  - Can [manage the Namespace](/cloud/namespaces#manage-namespaces) including identities and permissions
  - Can create, rename, update, and delete [Workflows](/workflows) within the Namespace
- **Write:**
  - Can create, rename, update, and delete [Workflows](/workflows) within the Namespace
- **Read-Only:**
  - Can only read information from the Namespace

## How to update an account-level role in Temporal Cloud {#update-roles}

With Global Admin or Account Owner privileges, you can update any user's account-level [role](#account-level-roles) using either the Web UI or the tcld CLI utility.
The Account Owner role can only be granted by existing Account Owners.

For security reasons, changes to the Account Owner role must be made through Temporal Support.
To change or delete an Account Owner, you must submit a [support ticket](https://temporalsupport.zendesk.com/).

{/* How to update an account-level role in Temporal Cloud using Web UI */}

### How to update an account-level role using Web UI

1. In Temporal Web UI, select **Settings** in the left portion of the window.
1. On the **Settings** page, select the user.
1. On the user profile page, select **Edit User**.
1. On the **Edit User** page in **Account Level Role**, select the role.
1. Select **Save**.

{/* How to update an account-level role in Temporal Cloud using tcld */}

### How to update an account-level role using tcld

For details, see the [tcld user set-account-role](/cloud/tcld/user/#set-account-role) command.

## How to update Namespace-level permissions in Temporal Cloud {#update-permissions}

You can update Namespace-level [permissions](#namespace-level-permissions) by using either Web UI or tcld.

{/* How to update Namespace-level permissions for a Namespace in Temporal Cloud using Web UI */}

### How to use the Web UI to update a user's permissions across multiple Namespaces

1. In Temporal Web UI, select **Namespaces** in the left portion of the window.
1. On the **Namespaces** page, select the Namespace.
1. If necessary, scroll down to the list of permissions
1. On the user profile page in **Namespace permissions**, select the Namespace.
1. On the Namespace page in **Account Level Role**, select the role.
1. Select **Save**.

{/* How to update Namespace-level permissions for a user in Temporal Cloud using Web UI */}

### How to use the Web UI to update permissions for multiple users within a single Namespace

:::note

A user with the Account Owner or Global Admin account-level [role](#account-level-roles) has Namespace Admin permissions for all Namespaces.

:::

1. In Temporal Web UI, select **Settings** in the left portion of the window.
1. On the **Settings** page in the **Users** tab, select the user.
1. On the user profile page, select **Edit User**.
1. On the **Edit User** page in **Namespace permissions**, change the permissions for one or more Namespaces.
1. Select **Save**.

{/* How to update an account-level role in Temporal Cloud using tcld */}

### How to use tcld to update Namespace-level permissions

For details, see the [tcld user set-namespace-permissions](/cloud/tcld/user/#set-namespace-permissions) command.

## How to delete a user from your Temporal Cloud account {#delete-users}

You can delete a user from your Temporal Cloud Account by using either Web UI or tcld.

:::info

To delete a user, a user must have the Account Owner or Global Admin account-level [role](#account-level-roles).

:::

{/* How to delete a user from your Temporal Cloud account using Web UI */}

### How to update an account-level role using Web UI

1. In Temporal Web UI, select **Settings** in the left portion of the window.
1. On the **Settings** page, find the user and, on the right end of the row, select **Delete**.
1. In the **Delete User** dialog, select **Delete**.

You can delete a user in two other ways in Web UI:

- User profile page: Select the down arrow next to **Edit User** and then select **Delete**.
- **Edit User** page: Select **Delete User**.

{/* How to delete a user from your Temporal Cloud account using tcld */}

### How to update an account-level role using tcld

For details, see the [tcld user delete](/cloud/tcld/user/#delete) command.

## Account-level roles and Namespace-level permissions {#account-level-roles-and-namespace-level-permissions}

Temporal account-level roles and Namespace-level permissions provide access to specific Temporal Workflow and Temporal Cloud operational APIs.
The following table provides the API details associated with each account-level role and Namespace-level permission.

:::note

Account Owners and Global Admins have Namespace Admin permissions on all Namespaces.

:::

#### Account-level role details

This table provides API-level details for the permissions granted to a user through account-level roles. These permissions are configured per user.

| Permission                        | Read-only | Developer | Finance Admin | Global Admin | Account Owner |
| --------------------------------- | --------- | --------- | ------------- | ------------ | ------------- |
| CountIdentities                   | ✔         | ✔         | ✔             | ✔            | ✔             |
| CreateAccountAuditLogSink         |           |           |               | ✔            | ✔             |
| CreateAPIKey                      | ✔         | ✔         | ✔             | ✔            | ✔             |
| CreateNamespace                   |           | ✔         |               | ✔            | ✔             |
| CreateNexusEndpoint               |           | ✔         |               | ✔            | ✔             |
| CreateServiceAccount              |           |           |               | ✔            | ✔             |
| CreateServiceAccountAPIKey        |           |           |               | ✔            | ✔             |
| CreateStripeCustomerPortalSession |           |           | ✔             |              | ✔             |
| CreateUser                        |           |           |               | ✔            | ✔             |
| DeleteAccountAuditLogSink         |           |           |               | ✔            | ✔             |
| DeleteAPIKey                      | ✔         | ✔         | ✔             | ✔            | ✔             |
| DeleteNexusEndpoint               |           | ✔         |               | ✔            | ✔             |
| DeleteServiceAccount              |           |           |               | ✔            | ✔             |
| DeleteUser                        |           |           |               | ✔            | ✔             |
| GetAccount                        | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetAccountAuditLogSink            |           |           |               | ✔            | ✔             |
| GetAccountAuditLogSinks           |           |           |               | ✔            | ✔             |
| GetAccountFeatureFlags            | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetAccountLimits                  | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetAccountSettings                | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetAccountUsage                   |           |           |               | ✔            | ✔             |
| GetAPIKey                         | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetAPIKeys                        | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetAsyncOperation                 | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetAuditLogs                      |           |           |               | ✔            | ✔             |
| GetDecodedCertificate             | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetIdentities                     | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetIdentity                       | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetNamespaces                     | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetNamespacesUsage                |           |           |               | ✔            | ✔             |
| GetNexusEndpoint                  | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetNexusEndpoints                 | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetRegion                         | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetRegions                        | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetRequestStatus                  | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetRequestStatuses                |           |           |               | ✔            | ✔             |
| GetRequestStatusesForNamespace    | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetRequestStatusesForUser         | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetRoles                          | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetRolesByPermissions             | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetServiceAccount                 | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetServiceAccounts                | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetStripeInvoice                  |           |           | ✔             |              | ✔             |
| GetUser                           | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetUsers                          | ✔         | ✔         | ✔             | ✔            | ✔             |
| GetUsersWithAccountRoles          | ✔         | ✔         | ✔             | ✔            | ✔             |
| InviteUsers                       |           |           |               | ✔            | ✔             |
| ListCreditLedgerEntries           |           |           | ✔             |              | ✔             |
| ListGrants                        |           |           | ✔             |              | ✔             |
| ListMetronomeInvoices             |           |           | ✔             |              | ✔             |
| ListMetronomeInvoicesForNamespace |           |           | ✔             |              | ✔             |
| ListNamespaces                    | ✔         | ✔         | ✔             | ✔            | ✔             |
| ListPromotionGrantBalances        |           |           | ✔             |              | ✔             |
| ResendUserInvite                  |           |           |               | ✔            | ✔             |
| SetAccountSettings                |           |           |               | ✔            | ✔             |
| SyncCurrentUserInvite             | ✔         | ✔         | ✔             | ✔            | ✔             |
| UpdateAccount                     |           |           |               | ✔            | ✔             |
| UpdateAccountAuditLogSink         |           |           |               | ✔            | ✔             |
| UpdateAPIKey                      | ✔         | ✔         | ✔             | ✔            | ✔             |
| UpdateNexusEndpoint               |           | ✔         |               | ✔            | ✔             |
| UpdateServiceAccount              |           |           |               | ✔            | ✔             |
| UpdateUser                        |           |           |               | ✔            | ✔             |
| ValidateAccountAuditLogSink       |           |           |               | ✔            | ✔             |

#### Namespace-level permissions details

This table provides API-level details for the permissions granted to a user through Namespace-level permissions.
These permissions are configured per Namespace per user.

| Permission                         | Read | Write | Namespace Admin |
| ---------------------------------- | ---- | ----- | --------------- |
| CountWorkflowExecutions            | ✔    | ✔     | ✔               |
| CreateExportSink                   |      | ✔     | ✔               |
| CreateSchedule                     |      | ✔     | ✔               |
| DeleteExportSink                   |      | ✔     | ✔               |
| DeleteNamespace                    |      | ✔     | ✔               |
| DeleteSchedule                     |      | ✔     | ✔               |
| DescribeBatchOperation             | ✔    | ✔     | ✔               |
| DescribeNamespace                  | ✔    | ✔     | ✔               |
| DescribeSchedule                   | ✔    | ✔     | ✔               |
| DescribeTaskQueue                  | ✔    | ✔     | ✔               |
| DescribeWorkflowExecution          | ✔    | ✔     | ✔               |
| FailoverNamespace                  |      |       | ✔               |
| GetExportSink                      | ✔    | ✔     | ✔               |
| GetExportSinks                     | ✔    | ✔     | ✔               |
| GetNamespace                       | ✔    | ✔     | ✔               |
| GetNamespaceUsage                  | ✔    | ✔     | ✔               |
| GetReplicationStatus               | ✔    | ✔     | ✔               |
| GetSearchAttributes                | ✔    | ✔     | ✔               |
| GetUsersForNamespace               | ✔    | ✔     | ✔               |
| GetWorkerBuildIdCompatibility      | ✔    | ✔     | ✔               |
| GetWorkerTaskReachability          | ✔    | ✔     | ✔               |
| GetWorkflowExecutionHistory        | ✔    | ✔     | ✔               |
| GetWorkflowExecutionHistoryReverse | ✔    | ✔     | ✔               |
| GlobalizeNamespace                 |      |       | ✔               |
| ListBatchOperations                | ✔    | ✔     | ✔               |
| ListClosedWorkflowExecutions       | ✔    | ✔     | ✔               |
| ListExportSinks                    | ✔    | ✔     | ✔               |
| ListFailoverHistoryByNamespace     | ✔    | ✔     | ✔               |
| ListOpenWorkflowExecutions         | ✔    | ✔     | ✔               |
| ListReplicaStatus                  | ✔    | ✔     | ✔               |
| ListScheduleMatchingTimes          | ✔    | ✔     | ✔               |
| ListSchedules                      | ✔    | ✔     | ✔               |
| ListTaskQueuePartitions            | ✔    | ✔     | ✔               |
| ListWorkflowExecutions             | ✔    | ✔     | ✔               |
| PatchSchedule                      |      | ✔     | ✔               |
| PollActivityTaskQueue              |      | ✔     | ✔               |
| PollWorkflowTaskQueue              |      | ✔     | ✔               |
| QueryWorkflow                      | ✔    | ✔     | ✔               |
| RecordActivityTaskHeartbeat        |      | ✔     | ✔               |
| RecordActivityTaskHeartbeatById    |      | ✔     | ✔               |
| RenameCustomSearchAttribute        |      | ✔     | ✔               |
| RequestCancelWorkflowExecution     |      | ✔     | ✔               |
| ResetStickyTaskQueue               |      | ✔     | ✔               |
| ResetWorkflowExecution             |      | ✔     | ✔               |
| RespondActivityTaskCanceled        |      | ✔     | ✔               |
| RespondActivityTaskCanceledById    |      | ✔     | ✔               |
| RespondActivityTaskCompleted       |      | ✔     | ✔               |
| RespondActivityTaskCompletedById   |      | ✔     | ✔               |
| RespondActivityTaskFailed          |      | ✔     | ✔               |
| RespondActivityTaskFailedById      |      | ✔     | ✔               |
| RespondQueryTaskCompleted          |      | ✔     | ✔               |
| RespondWorkflowTaskCompleted       |      | ✔     | ✔               |
| RespondWorkflowTaskFailed          |      | ✔     | ✔               |
| SetUserNamespaceAccess             |      |       | ✔               |
| SignalWithStartWorkflowExecution   |      | ✔     | ✔               |
| SignalWorkflowExecution            |      | ✔     | ✔               |
| StartBatchOperation                |      | ✔     | ✔               |
| StartWorkflowExecution             |      | ✔     | ✔               |
| StopBatchOperation                 |      | ✔     | ✔               |
| TerminateWorkflowExecution         |      | ✔     | ✔               |
| UpdateExportSink                   |      | ✔     | ✔               |
| UpdateNamespace                    |      | ✔     | ✔               |
| UpdateSchedule                     |      | ✔     | ✔               |
| UpdateUserNamespacePermissions     |      |       | ✔               |
| ValidateExportSink                 |      | ✔     | ✔               |
| ValidateGlobalizeNamespace         |      |       | ✔               |

Account Owners and Global Admins will have Namespace Admin permissions on Namespaces.

## How to troubleshoot account access issues {#troubleshoot-access}

### Why can't I sign in after my email domain changed? {#email-domain-change}

If your organization changed its email domain (for example, from `@oldcompany.com` to `@newcompany.com`), you may be unable to sign in to Temporal Cloud with your existing account.

**Why this happens:**
When you sign in using "Continue with Google" or "Continue with Microsoft", Temporal Cloud identifies your account by your email address.
If your email address changes, Temporal Cloud sees this as a different identity and cannot match it to your existing account.

**How to resolve this:**
[Create a support ticket](/cloud/support#support-ticket) with the following information:

- Your previous email address (the one originally used to access Temporal Cloud)
- Your new email address
- Your Temporal Cloud Account Id (if known)

Temporal Support can update your account to use your new email address.

:::tip Use SAML for enterprise identity management

If your organization frequently changes email domains or wants centralized control over user authentication, consider using [SAML authentication](/cloud/saml).
With SAML, your identity provider (IdP) manages user identities, and email domain changes can be handled within your IdP without affecting Temporal Cloud access.

:::

---

## Enable High Availability

:::tip Support, stability, and dependency info

Same-region Replication and Multi-cloud Replication are in
[Public Preview](/evaluate/development-production-features/release-stages#public-preview).

Multi-region Replication is in
[General Availability](/evaluate/development-production-features/release-stages#general-availability)

:::

You can enable High Availability features
([Single-region Replication](/cloud/high-availability#same-region-replication),
[Multi-region Replication](/cloud/high-availability#multi-region-replication), or
[Multi-cloud Replication](/cloud/high-availability#multi-cloud-replication)) for a new or existing Namespace by adding a
replica. When you add a replica, Temporal Cloud begins asynchronously replicating ongoing and existing Workflow
Executions.

Not all replication options are available in all regions. See the [region documentation](/cloud/regions) for the
replication options available in each region.

Using private network connectivity with a HA namespace requires extra setup. See
[Connectivity for HA](/cloud/high-availability/ha-connectivity).

There are charges associated with Replication and enabling High Availability features. For pricing details, visit
Temporal Cloud's [Pricing](/cloud/pricing) page.

## Create a Namespace with High Availability features {#create}

To create a new Namespace with High Availability features, you can use the Temporal Cloud UI or the tcld command line
utility.

<Tabs>

<TabItem value="webui" label="Web UI">

    1. Visit Temporal Cloud in your Web browser.
    1. During Namespace creation, specify the primary [region](/cloud/regions) for the Namespace.
    1. Select "Add a replica".
    1. Choose the [region](/cloud/regions) for the replica.

    The web interface will present an estimated time for replication to complete.
    This time is based on your selection and the size and scale of the Workflows in your Namespace.

</TabItem>

<TabItem value="tcldcli" label="tcld">

At the command line, enter:

```
tcld namespace create \
   --namespace <namespace_id>.<account_id> \
   --region <primary_region> \
   --region <replica_region>
```

Specify the [region codes](/cloud/regions) as arguments to the two `--region` flags.

- Using the same region replicates to an isolation domain within that region.
- Using a different region replicates across regions.

If using API key authentication with the `--api-key` flag, you must add it directly after the tcld command and before
`namespace create`.

</TabItem>

</Tabs>

Temporal Cloud sends an email alert to all Namespace Admins once your Namespace replica is ready for use.

## Add High Availability to an existing Namespace {#upgrade}

A replica can be added after a namespace has already been created.

<Tabs>

<TabItem value="webui" label="Web UI">

1. Visit Temporal Cloud Namespaces in your Web browser.
1. Navigate to the Namespace details page.
1. Select the “Add a replica” button.
1. Choose the [region](/cloud/regions) for the replica.

The web interface will present an estimated time for replication to complete. This time is based on your selection and
the size and scale of the Workflows in your Namespace.

Temporal Cloud sends an email alert to all Namespace Admins once your Namespace replica is ready for use.

</TabItem>

<TabItem value="tcldcli" label="tcld">

At the command line, enter:

```
tcld namespace add-region \
   --namespace <namespace_id>.<account_id> \
   --region <replica_region>
```

Specify the [region code](/cloud/regions) of the region where you want to create the replica as an argument to the
`--region` flag.

If using API key authentication with the `--api-key` flag, you must add it directly after the tcld command and before
`namespace add-region`.

Temporal Cloud sends an email alert once your Namespace is ready for use.

</TabItem>

</Tabs>

## Change a replica location {#changing}

Temporal Cloud can't change replica locations directly. To change a replica's location, you need to remove the replica
and add a new one.

:::caution

We discourage changing the location of your replica for deployed applications, except under exceptional circumstances.
If you remove your replica, you lose the availability guarantees of the Namespace, and it can take time to add another
replica.

If you remove a replica from a region, you must wait seven days before you can re-enable High Availability (HA) in that
same location. During this period, you may add a replica to a different region, provided you have not had one active
there within the last seven days.

:::

Follow these steps to change the replica location:

1. [Remove your replica](#disable). This disables High Availability for your Namespace.
2. [Add a new replica](#upgrade) to your Namespace.

You will receive an email alert once your Namespace is ready for use.

## Disable High Availability (remove a replica) {#disable}

To disable High Availability features on a Namespace, remove the replica from that Namespace. Removing a replica
disables all High Availability features:

- Discontinues replication of the Workflows in the Namespace.
- Disables the Namespace's ability to trigger a failover to a different region or cloud.
- For Workers and Clients that use API keys, removing a replica requires connecting to the Namespace using the published
  [regional endpoint](/cloud/regions) for the Namespace's region.
  - Disables connecting to the Namespace with API keys and the Namespace's endpoint or the replica region's regional
    endpoint.
- Ends High Availability charges.

:::caution

After removing a Namespace's replica, you cannot re-enable replication on that same Temporal Cloud Namespace in the same
location for seven days.

:::

Follow these steps to remove a replica from a Namespace:

<Tabs>

<TabItem value="webui" label="Web UI">

1. If you are using API keys for authentication on this Namespace, configure your Workers and Clients that use API keys
   to [connect with the regional Temporal Cloud endpoint](/cloud/api-keys#namespace-authentication) for the Namespace's
   primary region.
1. Navigate to the Namespace details page in Temporal Cloud
1. Select the option to "Remove Replica" on the "Region" card.

</TabItem>

<TabItem value="tcldcli" label="tcld">

First, if you are using API keys for authentication on this Namespace, configure your Workers and Clients that use API
keys to [connect with the regional Temporal Cloud endpoint](/cloud/api-keys#namespace-authentication) for the
Namespace's primary region.

Then, run the following command to remove the replica:

```
tcld namespace delete-region \
    --api-key <api_key> \
    --namespace <namespace_id>.<account_id> \
    --region <replica_region>
```

</TabItem>

</Tabs>

:::important

To remove a replica from a Namespace with API keys enabled, you need assistance from Temporal Support. Please
[contact support](/cloud/support#support-ticket) with the Namespace ID of the Namespace where you want to remove the
replica. You must confirm that Workers and Clients with API keys have been configured to connect to the Namespace using
the published [regional endpoint](/cloud/regions).

This safeguard ensures that Workers and Clients continue running uninterrupted once Temporal Support removes the
replica. After the replica is removed, if Workers and Clients with API keys attempt to use the Namespace endpoint or the
former replica's regional endpoint, their requests will fail.

:::

---

## Configure and Trigger Failovers

In case of an incident or an outage, Temporal will automatically <ToolTipTerm term="fail over" src="failover" /> your Namespace from the primary to the replica.
This lets Workflow Executions continue with minimal interruptions or data loss.
You can also [manually initiate failovers](/cloud/high-availability/failovers) based on your situational monitoring or for testing.

Returning control from the replica to the primary is called a <ToolTipTerm term="failback" />.
The replica is active for a brief duration during an incident.
After the incident, Temporal fails back to the primary.

## Failovers

Occasionally, a Namespace may become temporarily unavailable due to an unexpected incident.
Temporal Cloud detects these issues using regular health checks.

### Health checks

Temporal Cloud monitors error rates, latencies, and infrastructure problems, such as request timeouts.
If it finds unhealthy conditions where indicators exceed the allowed thresholds, Temporal automatically switches the primary to the replica.
In most cases, the replica is unaffected by the issue.
This process is known as failover.

### Automatic failovers

Failovers prevent data loss and application interruptions.
Existing Workflows continue, and new Workflows start as the incident is addressed.
Once the incident is resolved, Temporal Cloud performs a "failback," shifting Workflow Execution processing back to the original Namespace.

Temporal Cloud handles failovers automatically, ensuring continuity without manual intervention.

<CaptionedImage src="/img/cloud/high-availability/failover.png" title="On failover, the replica becomes active and the Namespace endpoint directs access to it." />

For more control over the failover process, you can [disable automated failovers](/cloud/high-availability/failovers#disabling-temporal-initiated).

:::tip

You can test the failover of Namespace with High Availability features by manually triggering a failover using the UI or the 'tcld' CLI utility.
In most scenarios, we recommend you let Temporal handle failovers for you.

After failover, be aware of the following points:

- When working with Multi-region Namespaces, your CNAME may change.
  For example, it may switch from aws-us-west-1.region.tmprl.cloud to aws-us-east-1.region.tmprl.cloud.
  This change doesn't affect same-region Namespaces.

- Your Namespace endpoint _will not change_.
  If it is `my_namespace.my_account.tmprl.cloud:7233` before failover, it will be `my_namespace.my_account.tmprl.cloud:7233` after failover.

:::

### The failover process {#failover-process}

Temporal's automated failover process works as follows:

- During normal operation, the primary asynchronously copies operations and metadata to its replica, keeping them in sync.
- If the primary becomes unavailable, Temporal detects the issue through health checks.
  It automatically switches to the replica, using one of its available [failover scenarios](#scenarios).
- The replica takes over the active role and becomes the primary.
  Operations continue with minimal disruption.
- When the original primary recovers, the roles can either switch back (failback, by default) or remain as they are, based on your Namespace settings.
  Automatic role switching with failover and failback minimizes downtime for consistent availability.

:::info

A Namespace failover, which updates the "active region" field in the Namespace record, is a metadata update.
This update is replicated through the Namespace metadata mechanism.

:::

## Failover scenarios {#scenarios}

The Temporal Cloud failover mechanism supports several modes for executing Namespace failovers.
These modes include graceful failover ("handover"), forced failover, and a hybrid mode.
The hybrid mode is Temporal Cloud’s default Namespace behavior.

### Graceful failover (handover) {#graceful-failover}

In this mode, Temporal Cloud fully processes and drains replication Tasks.
Temporal Cloud pauses traffic to the Namespace before the failover.
Graceful failover prevents the loss of progress and avoids data conflicts.

During graceful failover, the Namespace may experience a brief period of unavailability. 
This duration can be limited by the caller and defaults to 10 seconds. 
If the system is unable to reach a consistent state within this timeout, the failover attempt is aborted and the Namespace reverts to its previous state.
During this unavailable period:

- Existing Workflows stop progress.
- Temporal Cloud returns a "Service unavailable error".
  This error is retryable by the Temporal SDKs.
- State transitions will not happen and tasks are not dispatched.
- User requests like start/signal Workflow are rejected.
- Operations are paused during handover.

This mode favors _consistency_ over availability.

### Forced failover {#forced-failover}

In this mode, a Namespace immediately activates in the replica.
Events not replicated due to replication lag undergo conflict resolution upon reaching the new active Namespace.

This mode prioritizes _availability_ over consistency.

### Hybrid failover mode {#hybrid-failover}

While graceful failovers are preferred for consistency, they aren’t always practical.
Temporal Cloud’s hybrid failover mode (the default mode) limits the initial graceful failover attempt to 10 seconds or less.

During this period:

- Existing Workflows stop progress.
- Temporal Cloud returns a "Service unavailable error", which is retried by SDKs.

If the graceful approach doesn’t resolve the issue, Temporal Cloud automatically switches to a forced failover.

This strategy balances _consistency_ and _availability_ requirements.

### Scenario summary

| Failover Scenario            | Characteristics                                         |
| ---------------------------- | ------------------------------------------------------- |
| Graceful failover (handover) | Favors _consistency_ over availability.                 |
| Forced failover              | Prioritizes _availability_ over consistency.            |
| Hybrid failover mode         | Balances _consistency_ and _availability_ requirements. |

## Network partitions

At any time only the primary or the replica is active.
The only exception occurs in the event of a [network partition](https://en.wikipedia.org/wiki/Network_partition), when a Network splits into separate subnetworks.
Should this occur, you can promote a replica to active status.
**Caution:** This temporarily makes both regions active.
After the network partition is resolved and communication between the isolation domains/regions is restored, a conflict resolution algorithm determines whether the primary or replica remains active.

:::tip

In traditional active/active replication, multiple nodes serve requests and accept writes simultaneously, ensuring strong synchronous data consistency.
In contrast, with a Temporal Cloud Namespace with High Availability Features, only the primary accepts requests and writes at any given time.
Workflow History Events are written to the primary first and then asynchronously replicated to the replica, ensuring that the replica remains in sync.

:::

## Conflict resolution {#conflict-resolution}

Namespaces with replicas rely on asynchronous event replication.
Updates made to the primary may not immediately be reflected in the replica due to <ToolTipTerm term="replication lag" />, particularly during failovers.
In the event of a non-graceful failover, replication lag may cause a temporary setback in Workflow progress.

Namespaces that aren't replicated can be configured to provide _at-most-once_ semantics for Activities execution when a retry policy's [maximum attempts](https://docs.temporal.io/retry-policies#maximum-attempts) is set to 0.
High Availability Namespaces provide _at-least-once_ semantics for execution of Activities.
Completed Activities _may_ be re-dispatched in a newly active Namespace, leading to repeated executions.

When a Workflow Execution is updated in a newly active replica following a failover, events from the previously active Namespace that arrive after the failover can't be directly applied.
At this point, Temporal Cloud has forked the Workflow History.

After failover, Temporal Cloud creates a new branch history for execution, and begins its <ToolTipTerm term="conflict resolution"/> process.
The Temporal Service ensures that Workflow Histories remain valid and are replayable by SDKs post-failover or after conflict resolution.
This capability is crucial for ensuring Workflow Executions continue forward without losing progress, and for maintaining consistency across replication, even during incidents that cause disruptions in replication.

## Perform a manual failover {#triggering-failovers}

For some users, Temporal's automated health checks and failovers don't provide sufficient nuance and control.
For this reason, you can manually trigger failovers based on your own custom alerts and for testing purposes.
This section explains how and what to expect afterward.

:::warning Check Your Replication Lag

Always check the <ToolTipTerm term="replication lag" /> before initiating a failover.
A forced failover when there is a significant replication lag has a higher likelihood of rolling back Workflow progress.

:::

### Trigger the failover {#manual-failovers}

You can trigger a failover manually using the Temporal&nbsp;CloudWeb&nbsp;UI or the tcld CLI, depending on your preference and setup.
The following instructions outline the steps for each method:

<Tabs>

<TabItem value="webui" label="Web UI">

1. Visit the [Namespace page](https://cloud.temporal.io/namespaces) on the Temporal Cloud Web UI.
1. Navigate to your Namespace details page and select the **Trigger a failover** option from the menu.
1. Confirm your action.
   After confirmation, Temporal initiates the failover.

</TabItem>

<TabItem value="tcldcli" label="tcld">

To manually trigger a failover, run the following command in your terminal:

```
tcld namespace failover \
    --namespace <namespace_id>.<account_id> \
    --region <target_region>
```

If using API key authentication with the `--api-key` flag, you must add it directly after the tcld command and before `namespace failover`.

</TabItem>

</Tabs>

Temporal fails over the primary to the replica.
When you're ready to fail back, follow these failover instructions to move the primary back to the original.

### Post-failover event information {#info}

After any failover, whether triggered by you or by Temporal, an event appears in both the [Temporal Cloud Web UI](https://cloud.temporal.io/namespaces) (on the Namespace detail page) and in your audit logs.
The audit log entry for Failover uses the `"operation": "FailoverNamespace"` event.
After failover, the replica becomes active, taking over in the isolation domain or region.

You don't need to monitor Temporal Cloud's failover response in real time.
Whenever there is a failover event, Temporal Cloud [notifies you via email](/cloud/notifications#admin-notifications)

### Returning to the primary with failbacks

After Temporal-initiated failovers, Temporal Cloud shifts Workflow Execution processing back to the original region or isolation domain that was active before the incident once the incident is resolved.
This is called a "failback".

:::note

To failback a manually-initiated failover, follow the [Manual Failover](#manual-failovers) directions to failover back to the original primary.

:::

## Disabling Temporal-initiated failovers {#disabling-temporal-initiated}

When you add a replica to a Namespace, in the event of an incident or an outage Temporal Cloud automatically fails over the Namespace to its replica.
_This is the recommended and default option._

However if you prefer to disable Temporal-initiated failovers and handle your own failovers, you can do so by following these instructions:

<Tabs>

<TabItem value="webui" label="Web UI">

1. Navigate to the Namespace detail page in Temporal Cloud.
1. Choose the "Disable Temporal-initiated failovers" option.

</TabItem>

<TabItem value="tcldcli" label="tcld">

To disable Temporal-initiated failovers, run the following command in your terminal:

```
tcld namespace update-high-availability \
    --namespace <namespace_id>.<account_id> \
    --disable-auto-failover=true
```

If using API key authentication with the `--api-key` flag, you must add it directly after the tcld command and before `namespace update-high-availability`

</TabItem>

</Tabs>

Temporal Cloud disables its health-check initiated failovers.
To restore the default behavior, unselect the option in the WebUI or change `true` to `false` in the CLI command.

## Best practices: Workers and failovers {#worker}

Enabling High Availability for Namespaces doesn't require specific Worker configuration.
The process is invisible to the Workers.
When a Namespace fails over to the replica, the DNS redirection orchestrated by Temporal ensures that your existing Workers continue to poll the Namespace without interruption.

When a Namespace fails over to a replica in a different region, Workers will be communicating cross-region.

- If your application can’t tolerate this latency, deploy a second set of Workers in the replica's region or opt for a replica in the same region:
- In the case of a complete regional outage, Workers in the original region may fail alongside the original Namespace.
  To keep Workflows moving during this level of outage, deploy a second set of Workers to the secondary region.

:::tip

Temporal Cloud enforces a maximum connection lifetime of 5 minutes.
This offers your Workers an opportunity to re-resolve the DNS.

:::

## Best practices: scheduled failover testing {#testing}

Microservices and external dependencies will fail at some point.
Testing failovers ensures your app can handle these failures effectively.
Temporal recommends regular and periodic failover testing for mission-critical applications in production.
By testing in non-emergency conditions, you verify that your app continues to function, even when parts of the infrastructure fail.

<DiscoverableDisclosure label="Why test?">

:::tip Safety First

If this is your first time performing a failover test, run it with a test-specific namespace and application.
This helps you gain operational experience before applying it to your production environment.
Practice runs help ensure the process runs smoothly during real incidents in production.

:::

Failover testing (also known as "<ToolTipTerm term="trigger testing" />)" can:

- **Validate replicated deployments**:
  In multi-region setups, failover testing ensures your app can run from another region when the primary region experiences outages.
  In standard setups, failover testing instead works with an isolation domain.
  This maintains high availability in mission-critical deployments.
  Manual testing confirms the failover mechanism works as expected, so your system handles incidents effectively.

- **Assess replication lag**:
  In multi-region deployment, monitoring [replication lag](/cloud/high-availability/monitoring#replication-lag-metric) between regions is crucial.
  Check the lag before initiating a failover to avoid rolling back Workflow progress.
  This is less important when using isolation domains as failover is usually instantaneous.
  Manual testing helps you practice this critical step and understand its impact.

- **Assess recovery time**:
  Manual testing helps you measure actual recovery time.
  You can check if it meets your expected Recovery Time Objective (RTO) of 20 minutes or less, as stated in the [High Availability Namespace SLA](/cloud/sla).

- **Identify potential issues**:
  Failover testing uncovers problems not visible during normal operation.
  This includes issues like [backlogs and capacity planning](https://temporal.io/blog/workers-in-production#testing-failure-paths-2438) and how external dependencies behave during a failover event.

- **Validate fault-oblivious programming**:
  Temporal uses a "fault-oblivious programming" model, where your app doesn’t need to explicitly handle many types of failures.
  Testing failovers ensures that this model works as expected in your app.

- **Operational readiness**:
  Regular testing familiarizes your team with the failover process, improving their ability to handle real incidents when they arise.

Testing failovers regularly ensures your Temporal-based applications remain resilient and reliable, even when infrastructure fails.

</DiscoverableDisclosure>

---

## Connectivity for High Availability

:::tip Namespaces with High Availability features and AWS PrivateLink

Proper networking configuration is required for failover to be transparent to clients and workers when using PrivateLink.
This page describes how to configure routing for Namespaces with High Availability features on AWS PrivateLink.

:::

To use AWS PrivateLink with High Availability features, you may need to:

- Override the regional DNS zone.
- Ensure network connectivity between the two regions.

These instructions assume you already have the PrivateLink connections in place. If not, follow our [guide for creating AWS PrivateLink connections and configuring private DNS](/cloud/connectivity/aws-connectivity).

## Customer side solutions

When using PrivateLink, you connect to Temporal Cloud through a VPC Endpoint, which uses addresses local to your network.
Temporal treats each `region.<tmprl_domain>` as a separate zone.
This setup allows you to override the default zone, ensuring that traffic is routed internally for the regions you’re using.

A Namespace's active region is reflected in the target of a CNAME record.
For example, if the active region of a Namespace is AWS us-west-2, the DNS configuration would look like this:

| ha-namespace.account-id.tmprl.cloud | CNAME | aws-us-west-2.region.tmprl.cloud |
| ----------------------------------- | ----- | -------------------------------- |

After a failover, the CNAME record will be updated to point to the failover region, for example:

| ha-namespace.account-id.tmprl.cloud | CNAME | aws-us-east-1.region.tmprl.cloud |
| ----------------------------------- | ----- | -------------------------------- |

The Temporal domain did not change, but the CNAME updated from us-west-2 to us-east-1.

<CaptionedImage
    src="/img/cloud/high-availability/private-link.png"
    title="Customer side solution example"
    zoom="true"
/>

## Setting up the DNS override

:::caution

Private connectivity is not yet offered for GCP Multi-region Namespaces.

:::

To set up the DNS override, configure specific regions to target the internal VPC Endpoint IP addresses.
For example, you might set aws-us-west-1.region.tmprl.cloud to target 192.168.1.2.
In AWS, this can be done using a Route 53 private hosted zone for `region.tmprl.cloud`.
Link that private zone to the VPCs you use for Workers.

When your Workers connect to the Namespace, they first resolve the `<ns>.<acct>.<tmprl_domain>` record.
This points to `<aws-active-region>.region.tmprl.cloud`, which then resolves to your internal IP addresses.

Consider how you’ll configure Workers for this setup.
You can either have Workers run in both regions continuously or establish connectivity between regions using Transit Gateway or VPC Peering.
This way, Workers can access the newly activated region once failover occurs.

## Available regions, PrivateLink endpoints, and DNS record overrides

:::caution

The `sa-east-1` region is not yet available for use with Multi-region Namespaces. Currently, it is the only region on the continent.

:::

The following table lists the available Temporal regions, PrivateLink endpoints, and DNS record overrides:

<JsonTable filename="/json/privatelink_aws.json" />

When using a Namespace with High Availability features, the Namespace's DNS record `<ns>.<acct>.<tmprl_domain>` points to a regional DNS record in the format `<region>.region.<tmprl_domain>`.
Here, `<region>` is the currently active region for your Namespace.

During failover, Temporal Cloud changes the target of the Namespace DNS record from one region to another.
Namespace DNS records are configured with a 15 second TTL.
Any DNS cache should re-resolve the record within this time.
As a rule of thumb, receiving an updated DNS record takes about twice (2x) the TTL.
Clients should converge to the newly targeted region within, at most, a 30-second delay.

---

## High Availability

Temporal Cloud's High Availability features use asynchronous <ToolTipTerm term="replication"/> across multiple <ToolTipTerm term="isolation domains" /> to provide enhanced resilience and a 99.99% [SLA](/cloud/sla).
When you enable High Availability features, Temporal deploys your primary and its <ToolTipTerm term="replica"/> in separate isolation domains, giving you control over the location of both. This redundancy, combined with <ToolTipTerm term="failover"/> capability, enhances availability during outages.

## Built-in reliability

Even without High Availability features, Temporal Cloud provides robust reliability and a 99.9% contractual Service Level Agreement ([SLA](/cloud/sla)) guarantee against service errors.

Each standard Temporal Namespace uses replication across three availability zones to ensure high availability.
An availability zone is a part of the system where tasks or operations are handled and executed.
This design helps manage workloads and ensure tasks are completed while improving resource use and reducing delays.

Replication makes sure that any changes to Workflow state or History are saved in all three zones _before_ the Temporal Service acknowledges a change back to the Client.
As a result, your standard Temporal Namespace stays operational even if one of its three zones becomes unavailable.
This provides the basis of our 99.9% service level.

## High Availability features {#high-availability-features}

:::tip Support, stability, and dependency info

Same-region Replication and Multi-cloud Replication are in [Public Preview](/evaluate/development-production-features/release-stages#public-preview).

Multi-region Replication is in [General Availability](/evaluate/development-production-features/release-stages#general-availability)

:::

High Availability features provide three types of replication:

| **Deployment**                          | **Description**                                            |
| --------------------------------------- | ---------------------------------------------------------- |
| **Same&#8209;region&nbsp;Replication**  | Isolation domains are located within the same region.   |
| **Multi&#8209;region&nbsp;Replication** | Isolation domains are located in separate regions.         |
| **Multi&#8209;cloud&nbsp;Replication**  | Isolation domains are located in separate cloud providers. |

### Same-region Replication

Temporal replicates Namespaces across isolation domains within one region.
This option is a good fit when your application is built for one region and you prefer to failover within that region.
This provides a reliable failover mechanism while maintaining deployment simplicity.

### Multi-region Replication

Temporal replicates Namespaces across regions, making sure Workflows and data are available even if a region fails.
Asynchronous replication means changes aren’t immediately reflected in other regions but will sync over time, ensuring data integrity.
This setup allows failovers between replicas without needing immediate consistency across regions.
Replication across different regions enhances resilience and reliability.

### Multi-cloud Replication

Temporal asynchronously replicates all Workflows (live and historical) and data to a Namespace in an entirely different cloud provider.
If a provider outage, regional outage, service disruption, or network issue occurs, traffic automatically shifts to the replica.
Replicated data is securely encrypted and transmitted across the public internet between cloud providers.
Internet connectivity allows workers in one cloud to fail over to a replica in a different cloud.

:::info

When you adopt Temporal's High Availability features, don't forget to consider the reliability of your own workers, infrastructure, and dependencies.
Issues like network outages, hardware failures, or misconfigurations in your own systems can affect your application performance.

For the highest level of reliability, distribute your dependencies across regions, and use our Multi-region or Multi-cloud replication features.
Using physically separated regions improves the fault tolerance of your application.

:::

## Service levels and recovery objectives

Namespaces using High Availability have a 99.99% [uptime SLA](/cloud/sla) with sub-1-minute [RPO](/cloud/rpo-rto) and 20-minute [RTO](/cloud/rpo-rto). For detailed information:

- [Service Level Agreement (SLA)](/cloud/sla)
- [Recovery Point Objective (RPO) and Recovery Time Objective (RTO)](/cloud/rpo-rto)

## Failover

High Availability Namespaces can automatically or manually [fail over](/cloud/high-availability/failovers) to the replica if the primary is unavailable or unhealthy.

## Target workloads

High Availability Namespaces are a great solution for Workloads where an outage would cause:

- Revenue loss
- Poor customer experience
- Problems stemming from policy/legal requirements that demand high availability

These are often major concerns for financial services, e-commerce, gaming, global SaaS platforms, bookings & reservations, delivery & shipping, and order management.

---

## Monitoring High Availability

Temporal Cloud offers several ways for you to track the health and performance of your
[High Availability](/cloud/high-availability) namespaces.

## Replication status

You can monitor your replica status with the Temporal Cloud UI. If the replica is unhealthy, Temporal Cloud disables the
“Trigger a failover” option to prevent failing over to an unhealthy replica. An unhealthy replica might be due to:

- **Data synchronization issues:** The replica fails to remain in sync with the primary due to network or performance
  problems.
- **Replication lag:** The replica falls behind the primary, causing it to be out of sync.
- **Network issues:** Loss of communication between the replica and the primary causes problems.
- **Failed health checks:** If the replica fails health checks, it’s marked as unhealthy.

These issues prevent the replica from being used during a failover, ensuring system stability and consistency.

## Replication lag metric

Temporal Cloud’s High Availability features use asynchronous replication between the primary and the replica. Workflow
updates in the primary, along with associated History Events, are transmitted to the replica. Replication lag refers to
the transmission delay of Workflow updates and history events from the primary to the replica.

:::tip

Temporal Cloud strives to maintain a <ToolTipTerm term="P95" /> replication lag of less than 1 minute. In this context,
P95 means 95% of updates are processed faster than this limit.

:::

A forced failover, when there is significant replication lag, increases the likelihood of rolling back Workflow
progress. Always check the replication lag metrics before initiating a failover.

Temporal Cloud emits three replication lag-specific
[metrics](/cloud/metrics/reference#replication-lag). The following samples demonstrate how you can
use these metrics to monitor and explore replication lag:

**P99 replication lag histogram**:

```
histogram_quantile(0.99, sum(rate(temporal_cloud_v0_replication_lag_bucket[$__rate_interval])) by (temporal_namespace, le))
```

**Average replication lag**:

```
sum(rate(temporal_cloud_v0_replication_lag_sum[$__rate_interval])) by (temporal_namespace)
/
sum(rate(temporal_cloud_v0_replication_lag_count[$__rate_interval])) by (temporal_namespace)
```

When a Namespace is using a replica, you may notice that the Action count in `temporal_cloud_v0_total_action_count` is
2x what it was before adding a replica. This happens because Actions are replicated; they occur on both the primary and
the replica.

## Failover audit log

When Temporal triggers failovers, the [audit log](/cloud/audit-logs) will update with details.

Look for `"operation": "FailoverNamespace"` in the logs.

---

## Temporal Cloud guide

Welcome to the Temporal Cloud guide.

In this guide you will find information about Temporal Cloud, onboarding, features, and how to use them.

To create a Temporal Cloud account, sign up [here](https://temporal.io/get-cloud).

**[Get started with Temporal Cloud.](/cloud/get-started)**

## Become familiar with Temporal Cloud

- [Overview of Temporal Cloud](/cloud/overview)
  - [Security model](/cloud/security)
  - [Service availability](/cloud/service-availability) (availability, region support, throughput, latency, and limits)
  - [Account, Namespace, and application level configurations](/cloud/limits)
  - [Service Level Agreement (SLA)](/cloud/sla)
  - [Pricing](/cloud/pricing)
  - [Support](/cloud/support)

## Feature guides

- [Get started with Temporal Cloud](/cloud/get-started)
  - [Manage certificates](/cloud/certificates)
  - [Manage API keys](/cloud/api-keys)
  - [Manage Namespaces](/cloud/namespaces)
  - [Manage users](/cloud/users)
  - [Manage user groups](/cloud/user-groups)
  - [Manage billing](/cloud/billing-and-cost)
  - [Manage Service Accounts](/cloud/service-accounts)
- [API key feature guide](/cloud/api-keys)
- [Metrics feature guide](/cloud/metrics)
- [Temporal Nexus](/cloud/nexus)
- [SAML authentication feature guide](/cloud/saml)
- [Cloud Ops API](/ops)
- [Audit logging feature guide](/cloud/audit-logs)
- [`tcld` (Temporal Cloud command-line interface) reference](/cloud/tcld)

---

## Account Access

Temporal Cloud offers several ways to manage access to your Temporal Cloud account.

- [**Users**](/cloud/users) - Manage individual user accounts and permissions
- [**User Groups**](/cloud/user-groups) - Organize users into groups for simplified access management
- [**Service Accounts**](/cloud/service-accounts) - Configure service accounts for automated access
- [**SAML**](/cloud/saml) - Configure SAML-based single sign-on integration
- [**SCIM**](/cloud/scim) - Use your IDP to manage Temporal Cloud users and access via SCIM integration

---

## Datadog metrics setup

Datadog in partnership with Temporal Cloud has created a native integration with Temporal Cloud metrics.
This integration is available to all Datadog customers.

Exporting cloud metrics to Datadog provides enhanced observability, allowing you to monitor, alert, and visualize key performance indicators of your applications and infrastructure.
Temporal's integration with Datadog extends the monitoring capabilities of your Temporal Cloud deployment. Benefits of using this integration include:

- Out-of-the-box Temporal Cloud metrics dashboard in Datadog
- No need to run a service, Datadog connects directly to Temporal Cloud

For detailed instructions on how to use the integration, see [the documentation on Datadog's site](https://docs.datadoghq.com/integrations/temporal_cloud/).

---

## General observability setup with metrics

You will learn how to do the following:

- [Configure an endpoint using the UI](#configure-via-ui)
- [Configure an endpoint using tcld](#configure-via-cli-tcld)
- [Query for metrics with a PromQL endpoint](#query-promql)

## Configure using the UI {#configure-via-ui}

**How to configure a metrics endpoint using Temporal Cloud UI**

:::note

To view and manage third-party integration settings, your user account must have the Account Owner or Global Admin [role](/cloud/users#account-level-roles).

:::

To assign a certificate and generate your metrics endpoint, follow these steps:

1. Log in to Temporal Cloud UI with an Account Owner or Global Admin [role](/cloud/users#account-level-roles).
2. Go to **Settings** and select **Observability**.
4. Add your root CA certificate (.pem) and save it.
   Note that if an observability endpoint is already set up, you can append your root CA certificate here to use the generated observability endpoint in your observability tool.
5. To test your endpoint, run the following command on your host:
   ```
   curl -v --cert <path to your client-cert.pem> --key <path to your client-cert.key> "<your generated Temporal Cloud prometheus_endpoint>/api/v1/query?query=temporal_cloud_v0_state_transition_count"
   ```
   If you have Workflows running on a Namespace in your Temporal Cloud instance, you should see some data as a result of running this command.

After the page refreshes, the new metrics endpoint appears below **Endpoint**, in the form `https://<account-id>.tmprl.cloud/prometheus`.
Use the endpoint to configure your observability tool.
For example, if you use Grafana, see [Grafana data sources configuration](/cloud/metrics/prometheus-grafana#grafana-data-sources-configuration).

You can also query via the [Prometheus HTTP API](https://prometheus.io/docs/prometheus/latest/querying/api/) at URLs like:

```
https://<account-id>.tmprl.cloud/prometheus/api/v1/query?query=temporal_cloud_v0_state_transition_count
```

For example:

```
$ curl --cert client.pem --key client-key.pem "https://<account-id>.tmprl.cloud/prometheus/api/v1/query?query=temporal_cloud_v0_state_transition_count" | jq .
{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "__name__": "temporal_cloud_v0_state_transition_count",
          "__rollup__": "true",
          "operation": "WorkflowContext",
          "temporal_account": "your-account",
          "temporal_namespace": "your-namespace.your-account-is",
          "temporal_service_type": "history"
        },
        "value": [
          1672347471.2,
          "0"
        ]
      },
      ...
}
```

## Configure endpoint using tcld {#configure-via-cli-tcld}

**How to configure a metrics endpoint using the tcld CLI.**

To add a certificate to a metrics endpoint, use [`tcld account metrics accepted-client-ca add`](/cloud/tcld/account#add).

To enable a metrics endpoint, use [`tcld account metrics enable`](/cloud/tcld/account#enable).

To disable a metrics endpoint, use [`tcld account metrics disable`](/cloud/tcld/account#disable).

For more information, see [tcld account metrics command](/cloud/tcld/account#metrics).

## Query for metrics with a PromQL endpoint {#query-promql}

Temporal Cloud emits metrics in a Prometheus-supported format.
Prometheus is an open-source toolkit for alerting and monitoring.
The Temporal Service exposes Cloud metrics with a [Prometheus HTTP API endpoint](https://prometheus.io/docs/prometheus/latest/querying/api/).
Temporal Cloud metrics provide a compatible data source for visualizing, monitoring, and observability platforms like Grafana and Datadog.

You can use functions like [rate](https://prometheus.io/docs/prometheus/latest/querying/functions/#rate) or [increase](https://prometheus.io/docs/prometheus/latest/querying/functions/#increase) to calculate the rate of increase for a Temporal Cloud metric:

```
rate(temporal_cloud_v0_frontend_service_request_count[$__rate_interval])
```

Or you might use Prometheus to calculate average latencies or histogram quartiles:

```