# Source: https://www.promptfoo.dev/docs/providers/ai21/

# AI21 Labs

The [AI21 Labs API](https://docs.ai21.com/reference/chat-completion) offers access to AI21 models such as `jamba-1.5-mini` and `jamba-1.5-large`.

## API Key

To use AI21 Labs, you need to set the `AI21_API_KEY` environment variable, or specify the `apiKey` in the provider configuration.

Example of setting the environment variable:

```sh
export AI21_API_KEY=your_api_key_here
```

## Model Selection

You can specify which AI21 model to use in your configuration. Currently, the following models are available:

1. `jamba-1.5-mini`
2. `jamba-1.5-large`

Here's an example config that compares AI21 models:

```yaml
providers:
  - ai21:jamba-1.5-mini
  - ai21:jamba-1.5-large
```

## Options

The AI21 provider supports several options to customize the behavior of the model. These include:

- `temperature`: Controls the randomness of the output.
- `top_p`: Controls nucleus sampling, affecting the randomness of the output.
- `max_tokens`: The maximum length of the generated text.
- `response_format`: Set to `{ type: 'json_object' }` for JSON output or `{ type: 'text' }` for text output.
- `apiKeyEnvar`: An environment variable that contains the API key.
- `apiBaseUrl`: The base URL of the AI21 API.

## Example Configuration

Here's an example configuration for the AI21 provider:

```yaml
providers:
  - ai21:jamba-1.5-mini
    config:
      apiKey: your_api_key_here
      temperature: 0.1
      top_p: 1
      max_tokens: 1024
      response_format: { type: 'json_object' }
```

This configuration uses the `jamba-1.5-mini` model with a temperature of 0.1, top-p sampling with a value of 1, a maximum output length of 1024 tokens, JSON-formatted output, and a custom cost of $0.2 per 1M input tokens.

## Cost

The cost of using AI21 models depends on the model and the number of input and output tokens. Here are the costs for the available models:

- `jamba-1.5-mini`: $0.2 per 1M input tokens, $0.4 per 1M output tokens
- `jamba-1.5-large`: $2 per 1M input tokens, $8 per 1M output tokens

You can set the `cost` option in the provider configuration to specify a custom cost for the model.

## Supported environment variables

These AI21-related environment variables are supported:

| Variable | Description |
| --- | --- |
| `AI21_API_BASE_URL` | The base URL (protocol + hostname + port) to use for the AI21 API. |
| `AI21_API_KEY` | AI21 API key. |