# Azure AI Foundry

Azure AI Foundry provides access to serverless and deployed models, both for OpenAI and other providers. There are multiple ways to access those servers that are supported in GenAIScript:

* without any deployment, using the [Azure AI Model Inference](#azure_ai_inference) provider,
* with deployment for OpenAI models, using the [Azure AI OpenAI Serverless](#azure_serverless) provider,
* with deployments for non-OpenAI models, use the [Azure AI Serverless Models](#azure_serverless_models) provider.

You can deploy “serverless” models through [Azure AI Foundry](https://ai.azure.com/) and pay as you go per token. You can browse the [Azure AI Foundry model catalog](https://ai.azure.com/explore/models) and use the [serverless API](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-serverless-availability) filter to see the available models.

There are two types of serverless deployments that require different configurations: OpenAI models and all other models. The OpenAI models, like `gpt-4o`, are deployed to `.openai.azure.com` endpoints, while the Azure AI models, like `Meta-Llama-3.1-405B-Instruct` are deployed to `.models.ai.azure.com` endpoints.

They are configured slightly differently.

### Azure AI Inference[]()

[Section titled “Azure AI Inference ”](#azure-ai-inference)

The [Azure AI Model Inference API](https://learn.microsoft.com/en-us/azure/ai-foundry/model-inference/reference/reference-model-inference-api?tabs=javascript) provides a single endpoint to access a number of LLMs. This is a great way to experiment as you do not need to create deployments to access models. It supports both Entra ID and key-based authentication.

```js
script({ model: "azure_ai_inference:gpt-4o" });
```

[Play](https://youtube.com/watch?v=kh670Bxe_1E)

#### Managed Identity (Entra ID)

[Section titled “Managed Identity (Entra ID)”](#managed-identity-entra-id)

1. **Follow [these steps](https://learn.microsoft.com/en-us/azure/ai-foundry/model-inference/how-to/configure-entra-id?tabs=rest\&pivots=ai-foundry-portal) carefully** to configure the required Roles for your user.

2. Open <https://ai.azure.com/> and open your project

3. Configure the **Endpoint Target URL** as the `AZURE_AI_INFERENCE_API_ENDPOINT`.

   .env

   ```txt
   AZURE_AI_INFERENCE_API_ENDPOINT=https://<resource-name>.services.ai.azure.com/models
   ```

4. Find the model name in the model catalog with the **Deployment options = Serverless API** filter and use it in your script, `model: "azure_id_inference:model-id"`.

   ```js
   script({ model: "azure_ai_inference:model-id" });
   ```

#### API Key

[Section titled “API Key”](#api-key)

1. Open <https://ai.azure.com/>, open your project and go the **Overview** page.

2. Configure the **Endpoint Target URL** as the `AZURE_AI_INFERENCE_API_ENDPOINT` variable and the key in `AZURE_AI_INFERENCE_API_KEY` in the `.env` file\***\*.\*\***

   .env

   ```txt
   AZURE_AI_INFERENCE_API_ENDPOINT=https://<resourcename>.services.ai.azure.com/models
   AZURE_AI_INFERENCE_API_KEY=...
   ```

3. Find the model name in the model catalog with the **Deployment options = Serverless API** filter and use it in your script, `model: "azure_id_inference:model-id"`.

   ```js
   script({ model: "azure_ai_inference:model-id" });
   ```

#### API Version

[Section titled “API Version”](#api-version)

The default API version for Azure AI Inference is 2025-03-01-preview. You can change it by setting the `AZURE_AI_INFERENCE_API_VERSION` environment variable (see [Azure AI Documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/api-version-deprecation))

.env

```txt
AZURE_AI_INFERENCE_API_VERSION=2025-01-01-preview
```

### Aliases

The following model aliases are attempted by default in GenAIScript.

| Alias            | Model identifier       |
| ---------------- | ---------------------- |
| large            | gpt-4o                 |
| small            | gpt-4o-mini            |
| vision           | gpt-4o                 |
| vision\_small    | gpt-4o-mini            |
| reasoning        | o1                     |
| reasoning\_small | o1-mini                |
| embeddings       | text-embedding-3-small |

### Limitations

* listModels
* logprobs (and top logprobs) ignored
* Ignore prediction of output tokens
* topLogprobs

### Azure AI OpenAI Serverless[]()

[Section titled “Azure AI OpenAI Serverless ”](#azure-ai-openai-serverless)

The `azure_serverless` provider supports OpenAI models deployed through the Azure AI Foundry serverless deployments. It supports both Entra ID and key-based authentication.

```js
script({ model: "azure_serverless:deployment-id" });
```

Note

This kind of deployment is different from the **Azure OpenAI** deployments (`azure` provider).

#### Managed Identity (Entra ID)

[Section titled “Managed Identity (Entra ID)”](#managed-identity-entra-id-1)

1. Open <https://ai.azure.com/>, open your project and go the **Deployments** page.

2. Deploy a **base model** from the catalog. You can use the `Deployment Options` -> `Serverless API` option to deploy a model as a serverless API.

3. Deploy an OpenAI base model. This will also create a new Azure OpenAI resource in your subscription (which may be invisible to you, more later).

4. Update the `.env` file with the deployment endpoint in the `AZURE_SERVERLESS_OPENAI_API_ENDPOINT` variable.

   .env

   ```txt
   AZURE_SERVERLESS_OPENAI_API_ENDPOINT=https://....openai.azure.com
   ```

5. Go back to the **Overview** tab in your Azure AI Foundry project and click on **Open in Management center**.

6. Click on the **Azure OpenAI Service** resource, then click on the **Resource** external link which will take you back to the (underlying) Azure OpenAI service in Azure Portal.

7. Navigate to **Access Control (IAM)**, then **View My Access**. Make sure your user or service principal has the **Cognitive Services OpenAI User/Contributor** role. If you get a `401` error, click on **Add**, **Add role assignment** and add the **Cognitive Services OpenAI User** role to your user.

At this point, you are ready to login with the Azure CLI and use the managed identity.

Note

The resources created by Azure AI Foundry are not visible by default in the Azure Portal. To make them visible, open [All resources](https://portal.azure.com/#browse/all), click **Manage view** and select **Show hidden types**.

1. Install the [Azure CLI](https://learn.microsoft.com/en-us/javascript/api/overview/azure/identity-readme?view=azure-node-latest#authenticate-via-the-azure-cli).

2. Open a terminal and login

   ```sh
   az login
   ```

#### API Key

[Section titled “API Key”](#api-key-1)

1. Open your [Azure OpenAI resource](https://portal.azure.com) and navigate to **Resource Management**, then **Keys and Endpoint**.

2. Update the `.env` file with the endpoint and the secret key (**Key 1** or **Key 2**) and the endpoint.

   .env

   ```txt
   AZURE_SERVERLESS_OPENAI_API_ENDPOINT=https://....openai.azure.com
   AZURE_SERVERLESS_OPENAI_API_KEY=...
   ```

### Aliases

The following model aliases are attempted by default in GenAIScript.

| Alias            | Model identifier       |
| ---------------- | ---------------------- |
| large            | gpt-4o                 |
| small            | gpt-4o-mini            |
| vision           | gpt-4o                 |
| vision\_small    | gpt-4o-mini            |
| reasoning        | o1                     |
| reasoning\_small | o1-mini                |
| embeddings       | text-embedding-3-small |

### Limitations

* listModels
* Ignore prediction of output tokens

### Azure AI Serverless Models[]()

[Section titled “Azure AI Serverless Models ”](#azure-ai-serverless-models)

The `azure_serverless_models` provider supports non-OpenAI models, such as DeepSeek R1/v3, deployed through the Azure AI Foundary serverless deployments.

```js
script({ model: "azure_serverless_models:deployment-id" });
```

#### Managed Identity (Entra ID)

[Section titled “Managed Identity (Entra ID)”](#managed-identity-entra-id-2)

1. Open your **Azure AI Project** resource in the [Azure Portal](https://portal.azure.com)

2. Navigate to **Access Control (IAM)**, then **View My Access**. Make sure your user or service principal has the **Azure AI Developer** role. If you get a `401` error, click on **Add**, **Add role assignment** and add the **Azure AI Developer** role to your user.

3. Configure the **Endpoint Target URL** as the `AZURE_SERVERLESS_MODELS_API_ENDPOINT`.

   .env

   ```txt
   AZURE_SERVERLESS_MODELS_API_ENDPOINT=https://...models.ai.azure.com
   ```

4. Navigate to **deployments** and make sure that you have your LLM deployed and copy the Deployment Info name, you will need it in the script.

5. Update the `model` field in the `script` function to match the model deployment name in your Azure resource.

   ```js
   script({
       model: "azure_serverless:deployment-info-name",
       ...
   })
   ```

#### API Key

[Section titled “API Key”](#api-key-2)

1. Open <https://ai.azure.com/> and open the **Deployments** page.

2. Deploy a **base model** from the catalog. You can use the `Deployment Options` -> `Serverless API` option to deploy a model as a serverless API.

3. Configure the **Endpoint Target URL** as the `AZURE_SERVERLESS_MODELS_API_ENDPOINT` variable and the key in `AZURE_SERVERLESS_MODELS_API_KEY` in the `.env` file\***\*.\*\***

   .env

   ```txt
   AZURE_SERVERLESS_MODELS_API_ENDPOINT=https://...models.ai.azure.com
   AZURE_SERVERLESS_MODELS_API_KEY=...
   ```

4. Find the deployment name and use it in your script, `model: "azure_serverless_models:deployment-id"`.

#### Support for multiple inference deployments

[Section titled “Support for multiple inference deployments”](#support-for-multiple-inference-deployments)

You can update the `AZURE_SERVERLESS_MODELS_API_KEY` with a list of `deploymentid=key` pairs to support multiple deployments (each deployment has a different key).

.env

```txt
AZURE_SERVERLESS_MODELS_API_KEY="
model1=key1
model2=key2
model3=key3
"
```

### Limitations

* listModels
* Ignore prediction of output tokens