# Retrieval augmented generation (RAG) - quickstart

> A quickstart guide for performing retrieval augmented generation (RAG) with Cohere's Command models (v2 API).

Retrieval Augmented Generation (RAG) enables an LLM to ground its responses on external documents, thus improving the accuracy of its responses and minimizing hallucinations.

The Chat endpoint comes with built-in RAG capabilities such as document grounding and citation generation.

This quickstart guide shows you how to perform RAG with the Chat endpoint.

<Steps>
  ### Setup

  First, install the Cohere Python SDK with the following command.

  ```bash
  pip install -U cohere
  ```

  Next, import the library and create a client.

  <Tabs>
    <Tab title="Cohere Platform">
      ```python PYTHON
      import cohere

      co = cohere.ClientV2(
          "COHERE_API_KEY"
      )  # Get your free API key here: https://dashboard.cohere.com/api-keys
      ```
    </Tab>

    <Tab title="Private Deployment">
      ```python PYTHON
      import cohere

      co = cohere.ClientV2(
          api_key="",  # Leave this blank
          base_url="<YOUR_DEPLOYMENT_URL>",
      )
      ```
    </Tab>

    <Tab title="Bedrock">
      ```python PYTHON
      import cohere

      co = cohere.BedrockClientV2(
          aws_region="AWS_REGION",
          aws_access_key="AWS_ACCESS_KEY_ID",
          aws_secret_key="AWS_SECRET_ACCESS_KEY",
          aws_session_token="AWS_SESSION_TOKEN",
      )

      # Get the model name: https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html
      ```
    </Tab>

    <Tab title="SageMaker">
      ```python PYTHON
      import cohere

      co = cohere.SagemakerClientV2(
          aws_region="AWS_REGION",
          aws_access_key="AWS_ACCESS_KEY_ID",
          aws_secret_key="AWS_SECRET_ACCESS_KEY",
          aws_session_token="AWS_SESSION_TOKEN",
      )
      ```
    </Tab>

    <Tab title="Azure AI">
      ```python PYTHON
      import cohere

      co = cohere.ClientV2(
          api_key="AZURE_API_KEY",
          base_url="AZURE_ENDPOINT",
      )
      ```
    </Tab>
  </Tabs>

  ### Documents

  First, define the documents that will passed as the context for RAG. These documents are typically retrieved from sources such as vector databases via semantic search, or any system that can retrieve unstructured data given a user query.

  Each document is a `data` object that can take any number of fields e.g. `title`, `url`, `text`, etc.

  ```python PYTHON
  documents = [
      {
          "data": {
              "text": "Reimbursing Travel Expenses: Easily manage your travel expenses by submitting them through our finance tool. Approvals are prompt and straightforward."
          }
      },
      {
          "data": {
              "text": "Working from Abroad: Working remotely from another country is possible. Simply coordinate with your manager and ensure your availability during core hours."
          }
      },
      {
          "data": {
              "text": "Health and Wellness Benefits: We care about your well-being and offer gym memberships, on-site yoga classes, and comprehensive health insurance."
          }
      },
  ]
  ```

  ### Response Generation

  Next, call the Chat API by passing the documents in the `documents` parameter. This tells the model to run in RAG-mode and use these documents as the context in its response.

  <Tabs>
    <Tab title="Cohere Platform">
      ```python PYTHON
      # Add the user query
      query = "Are there health benefits?"

      # Generate the response
      response = co.chat(
          model="command-a-03-2025",
          messages=[{"role": "user", "content": query}],
          documents=documents,
      )

      # Display the response
      print(response.message.content[0].text)
      ```
    </Tab>

    <Tab title="Private Deployment">
      ```python PYTHON
      # Add the user query
      query = "Are there health benefits?"

      # Generate the response
      response = co.chat(
          model="command-a-03-2025",
          messages=[{"role": "user", "content": query}],
          documents=documents,
      )

      # Display the response
      print(response.message.content[0].text)
      ```
    </Tab>

    <Tab title="Bedrock">
      ```python PYTHON
      # Add the user query
      query = "Are there health benefits?"

      # Generate the response
      response = co.chat(
          model="YOUR_MODEL_NAME",
          messages=[{"role": "user", "content": query}],
          documents=documents,
      )

      # Display the response
      print(response.message.content[0].text)
      ```
    </Tab>

    <Tab title="SageMaker">
      ```python PYTHON
      # Add the user query
      query = "Are there health benefits?"

      # Generate the response
      response = co.chat(
          model="YOUR_ENDPOINT_NAME",
          messages=[{"role": "user", "content": query}],
          documents=documents,
      )

      # Display the response
      print(response.message.content[0].text)
      ```
    </Tab>

    <Tab title="Azure AI">
      ```python PYTHON
      # Add the user query
      query = "Are there health benefits?"

      # Generate the response
      response = co.chat(
          model="model",  # Pass a dummy string
          messages=[{"role": "user", "content": query}],
          documents=documents,
      )

      # Display the response
      print(response.message.content[0].text)
      ```
    </Tab>
  </Tabs>

  ```mdx wordWrap
  Yes, there are health benefits. We offer gym memberships, on-site yoga classes, and comprehensive health insurance.
  ```

  ### Citation Generation

  The response object contains a `citations` field, which contains specific text spans from the documents on which the response is grounded.

  ```python PYTHON
  if response.message.citations:
      for citation in response.message.citations:
          print(citation, "\n")
  ```

  ```mdx wordWrap
  start=14 end=88 text='gym memberships, on-site yoga classes, and comprehensive health insurance.' document_ids=['doc_1'] 

  ```
</Steps>

## Further Resources

* [Chat endpoint API reference](https://docs.cohere.com/reference/chat)
* [Documentation on RAG](https://docs.cohere.com/docs/retrieval-augmented-generation-rag)
* [LLM University module on RAG](https://cohere.com/llmu#rag)