# Deploy Finetuned Command Models from AWS Marketplace

> This document provides a guide for bringing your own finetuned models to Amazon SageMaker.

This document shows you how to deploy your own finetuned [HuggingFace Command-R model](https://huggingface.co/CohereForAI/c4ai-command-r-08-2024) using Amazon SageMaker. More specifically, assuming you already have the adapter weights or merged weights from your own finetuned Command model, we will show you how to:

* Merge the adapter weights with the weights of the base model if you only bring the adapter weights;
* Export the merged weights to the TensorRT-LLM inference engine using Amazon SageMaker;
* Deploy the engine as a SageMaker endpoint to serve your business use cases;

You can also find a [companion notebook](https://github.com/cohere-ai/cohere-developer-experience/blob/main/notebooks/finetuning/Deploy%20your%20own%20finetuned%20command-r-0824.ipynb) with working code samples.

## Prerequisites

* Ensure that IAM role used has `AmazonSageMakerFullAccess`
* To deploy your model successfully, ensure that either:
  * Your IAM role has these three permissions, and you have authority to make AWS Marketplace subscriptions in the relevant AWS account:
    * `aws-marketplace:ViewSubscriptions`
    * `aws-marketplace:Unsubscribe`
    * `aws-marketplace:Subscribe`
  * Or, your AWS account has a subscription to the packages for [Cohere Bring Your Own Fine-tuning](https://aws.amazon.com/marketplace/pp/prodview-5wt5pdnw3bbq6). If so, you can skip the "subscribe to the bring your own finetuning algorithm" step below.

**NOTE:** If you're running the companion notebook, know that it contains elements which render correctly in Jupyter interface, so you should open it from an Amazon SageMaker Notebook Instance or Amazon SageMaker Studio.

## Step 1: Subscribe to the bring your own finetuning algorithm

To subscribe to the algorithm:

* Open the algorithm listing page for [Cohere Bring Your Own Fine-tuning](https://aws.amazon.com/marketplace/pp/prodview-5wt5pdnw3bbq6).
* On the AWS Marketplace listing, click on the **Continue to Subscribe** button.
* On the **Subscribe to this software** page, review and click on **Accept Offer** if you and your organization agrees with EULA, pricing, and support terms. On the **Configure and launch** page, make sure the ARN displayed in your region match with the ARN you will use below.

## Step 2: Preliminary setup

First, let's install the Python packages and import them.

```TEXT
pip install "cohere>=5.11.0"
```

```Python PYTHON 
import cohere
import os
import sagemaker as sage

from sagemaker.s3 import S3Uploader
```

Make sure you have access to the resources in your AWS account. For example, you can configure an AWS profile by the command `aws configure sso` (see [here](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sso.html#cli-configure-sso-configure)) and run the command below to set the environment variable `AWS_PROFILE` as your profile name.

```Python PYTHON