# Observability for Ollama with Opik

> Start here to integrate Opik into your Ollama-based genai application for end-to-end LLM observability, unit testing, and optimization.

[Ollama](https://ollama.com/) allows users to run, interact with, and deploy AI models locally on their machines without the need for complex infrastructure or cloud dependencies.

There are multiple ways to interact with Ollama from Python including but not limited to the [ollama python package](https://pypi.org/project/ollama/), [LangChain](https://python.langchain.com/docs/integrations/providers/ollama/) or by using the [OpenAI library](https://docs.ollama.com/api/openai-compatibility#openai-python-library). We will cover how to trace your LLM calls for each of these methods.

## Account Setup

[Comet](https://www.comet.com/site?from=llm\&utm_source=opik\&utm_medium=colab\&utm_content=ollama\&utm_campaign=opik) provides a hosted version of the Opik platform, [simply create an account](https://www.comet.com/signup?from=llm\&utm_source=opik\&utm_medium=colab\&utm_content=ollama\&utm_campaign=opik) and grab your API Key.

> You can also run the Opik platform locally, see the [installation guide](https://www.comet.com/docs/opik/self-host/overview/?from=llm\&utm_source=opik\&utm_medium=colab\&utm_content=ollama\&utm_campaign=opik) for more information.

## Getting started

### Configure Ollama

Before starting, you will need to have an Ollama instance running. You can install Ollama by following the [quickstart guide](https://github.com/ollama/ollama/blob/main/README.md#quickstart) which will automatically start the Ollama API server. If the Ollama server is not running, you can start it using `ollama serve`.

Once Ollama is running, you can download the llama3.1 model by running `ollama pull llama3.1`. For a full list of models available on Ollama, please refer to the [Ollama library](https://ollama.com/library).

### Installation

You will also need to have Opik installed. You can install it by running:

```bash
pip install opik
```

### Configuring Opik

Configure the Opik Python SDK for your deployment type. See the [Python SDK Configuration guide](/tracing/sdk_configuration) for detailed instructions on:

* **CLI configuration**: `opik configure`
* **Code configuration**: `opik.configure()`
* **Self-hosted vs Cloud vs Enterprise** setup
* **Configuration files** and environment variables

## Tracking Ollama calls made with Ollama Python Package

To get started you will need to install the Ollama Python package:

```bash
pip install --quiet --upgrade ollama
```

We will then utilize the `track` decorator to log all the traces to Opik:

```python
import ollama
from opik import track, opik_context

@track(tags=['ollama', 'python-library'])
def ollama_llm_call(user_message: str):
    # Create the Ollama model
    response = ollama.chat(model='llama3.1', messages=[
        {
            'role': 'user',
            'content': user_message,
        },
    ])

    opik_context.update_current_span(
        metadata={
            'model': response['model'],
            'eval_duration': response['eval_duration'],
            'load_duration': response['load_duration'],
            'prompt_eval_duration': response['prompt_eval_duration'],
            'prompt_eval_count': response['prompt_eval_count'],
            'done': response['done'],
            'done_reason': response['done_reason'],
        },
        usage={
            'completion_tokens': response['eval_count'],
            'prompt_tokens': response['prompt_eval_count'],
            'total_tokens': response['eval_count'] + response['prompt_eval_count']
        }
    )
    return response['message']

ollama_llm_call("Say this is a test")
```

The trace will now be displayed in the Opik platform.

## Tracking Ollama calls made with OpenAI

Ollama is compatible with the OpenAI format and can be used with the OpenAI Python library. You can therefore leverage the Opik integration for OpenAI to trace your Ollama calls:

```python
from openai import OpenAI
from opik.integrations.openai import track_openai
import os

os.environ["OPIK_PROJECT_NAME"] = "ollama-integration"