# Source: https://langfuse.com/docs/observability/sdk/python/evaluation.md

---
title: Evaluations with the Langfuse Python SDK
description: Evaluate your application with the Langfuse Python SDK.
category: SDKs
---

# Evaluations

The Python SDK provides ways to evaluate your application. You can add custom scores to your traces and observations, or use the SDK to execute Dataset Runs.

<Callout type="info">

This page shows the evaluation methods that are supported by the Python SDK. Please refer to the [Evaluation documentation](/docs/evaluation/overview) for more information on how to evaluate your application in Langfuse.

</Callout>

## Create Scores

<Tabs items={["LangfuseSpan / LangfuseGeneration object methods", "Context-aware methods", "Low-level method"]}>

<Tab>
    - `span_or_generation_obj.score()`: Scores the specific observation object.
    - `span_or_generation_obj.score_trace()`: Scores the entire trace to which the object belongs.

    ```python
    from langfuse import get_client

    langfuse = get_client()

    with langfuse.start_as_current_observation(as_type="generation", name="summary_generation") as gen:
        # ... LLM call ...
        gen.update(output="summary text...")
        # Score this specific generation
        gen.score(name="conciseness", value=0.8, data_type="NUMERIC")
        # Score the overall trace
        gen.score_trace(name="user_feedback_rating", value="positive", data_type="CATEGORICAL")
    ```

</Tab>

<Tab>
    - `langfuse.score_current_span()`: Scores the currently active observation in the context.
    - `langfuse.score_current_trace()`: Scores the trace of the currently active observation.

    ```python
    from langfuse import get_client

    langfuse = get_client()

    with langfuse.start_as_current_observation(as_type="span", name="complex_task") as task_span:
        # ... perform task ...
        langfuse.score_current_span(name="task_component_quality", value=True, data_type="BOOLEAN")
        # ...
        if task_is_fully_successful:
             langfuse.score_current_trace(name="overall_success", value=1.0, data_type="NUMERIC")
    ```

</Tab>

<Tab>
    - Creates a score for a specified `trace_id` and optionally `observation_id`.
    - Useful when IDs are known, or for scoring after the trace/observation has completed.

    ```python
    from langfuse import get_client

    langfuse = get_client()

    langfuse.create_score(
        name="fact_check_accuracy",
        value=0.95, # Can be float for NUMERIC/BOOLEAN, string for CATEGORICAL
        trace_id="abcdef1234567890abcdef1234567890",
        observation_id="1234567890abcdef", # Optional: if scoring a specific observation
        session_id="session_123", # Optional: if scoring a specific session
        data_type="NUMERIC", # "NUMERIC", "BOOLEAN", "CATEGORICAL"
        comment="Source verified for 95% of claims."
    )
    ```

</Tab>

</Tabs>

**Score Parameters:**

| Parameter        | Type                      | Description                                                                                                            |
| :--------------- | :------------------------ | :--------------------------------------------------------------------------------------------------------------------- |
| `name`           | `str`                     | Name of the score (e.g., "relevance", "accuracy"). **Required.**                                                       |
| `value`          | `Union[float, str]`       | Score value. Float for `NUMERIC`/`BOOLEAN`, string for `CATEGORICAL`. **Required.**                                    |
| `trace_id`       | `str`                     | ID of the trace to associate with (for `create_score`). **Required.**                                                  |
| `observation_id` | `Optional[str]`           | ID of the specific observation to score (for `create_score`).                                                          |
| `session_id`     | `Optional[str]`           | ID of the specific session to score (for `create_score`).                                                              |
| `score_id`       | `Optional[str]`           | Custom ID for the score (auto-generated if None).                                                                      |
| `data_type`      | `Optional[ScoreDataType]` | `"NUMERIC"`, `"BOOLEAN"`, or `"CATEGORICAL"`. Inferred if not provided based on value type and score config on server. |
| `comment`        | `Optional[str]`           | Optional comment or explanation for the score.                                                                         |
| `config_id`      | `Optional[str]`           | Optional ID of a pre-defined score configuration in Langfuse.                                                          |

See [Scoring](/docs/scores/overview) for more details.

## Dataset Runs

[Langfuse Datasets](/docs/datasets/overview) are essential for evaluating and testing your LLM applications by allowing you to manage collections of inputs and their expected outputs.

### Create a Dataset

- **Creating**: You can programmatically create new datasets with `langfuse.create_dataset(...)` and add items to them using `langfuse.create_dataset_item(...)`.
- **Fetching**: Retrieve a dataset and its items using `langfuse.get_dataset(name: str)`. This returns a `DatasetClient` instance, which contains a list of `DatasetItemClient` objects (accessible via `dataset.items`). Each `DatasetItemClient` holds the `input`, `expected_output`, and `metadata` for an individual data point.

```python
from langfuse import get_client

langfuse = get_client()

# Fetch an existing dataset
dataset = langfuse.get_dataset(name="my-eval-dataset")
for item in dataset.items:
    print(f"Input: {item.input}, Expected: {item.expected_output}")

# Briefly: Creating a dataset and an item
new_dataset = langfuse.create_dataset(name="new-summarization-tasks")
langfuse.create_dataset_item(
    dataset_name="new-summarization-tasks",
    input={"text": "Long article..."},
    expected_output={"summary": "Short summary."}
)
```

### Run experiment on dataset

After fetching your dataset, you can execute a run against it. This will create a new trace for each item in the dataset. Please refer to the [Experiments via SDK documentation](/docs/evaluation/experiments/experiments-via-sdk) for more details.