# Generate 3 candidates per evaluation, select best
prompt = ChatPrompt(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Answer: {question}"},
    ],
    model_parameters={
        "n": 3,  # Generate 3 completions per call
        "temperature": 0.7,  # Higher temp = more variety between candidates
    },
)
```

<Info>
  Higher `temperature` values increase diversity between the N candidates. Consider using `temperature: 0.7-1.0` with `n > 1` to maximize variety.
</Info>

<Info>
  The low-level `call_model` and `call_model_async` helpers return a single
  response unless you pass `return_all=True`. Optimizers handle `n` internally,
  so you only need `return_all` when calling those helpers directly.
</Info>

## Use Cases

<AccordionGroup>
  <Accordion title="Reducing Evaluation Variance">
    Single-sample evaluation is noisy. With `n=3`, the optimizer scores each candidate and uses the best result, which makes optimization more robust to stochastic failures.

    ```python
    # Before: Single sample - noisy evaluation
    prompt = ChatPrompt(model="gpt-4o-mini", messages=[...])
    # Score might be 0.6 or 0.9 depending on luck

    # After: Best-of-3 - more stable evaluation
    prompt = ChatPrompt(
        model="gpt-4o-mini",
        messages=[...],
        model_parameters={"n": 3, "temperature": 0.8},
    )
    # Score reflects best achievable output
    ```
  </Accordion>

  <Accordion title="Pass@k Style Optimization">
    Inspired by code generation benchmarks (pass\@k), this approach measures whether a prompt *can* produce correct output, not just whether it *usually* does.

    ```python
    # Optimize for "can this prompt ever get it right?"
    prompt = ChatPrompt(
        model="gpt-4o-mini",
        messages=[...],
        model_parameters={"n": 5},  # pass@5 style
    )
    ```

    This is useful when:

    * Correctness matters more than consistency
    * You'll use majority voting or best-of-k at inference time
    * Tasks have high variance (creative writing, complex reasoning)
  </Accordion>

  <Accordion title="Handling Stochastic Tasks">
    Some tasks naturally have multiple valid answers. Using `n > 1` helps the optimizer find prompts that can generate *any* valid answer.

    ```python
    # Creative task: multiple valid outputs
    prompt = ChatPrompt(
        model="gpt-4o-mini",
        messages=[
            {"role": "user", "content": "Write a haiku about {topic}"},
        ],
        model_parameters={"n": 3, "temperature": 1.0},
    )
    ```
  </Accordion>
</AccordionGroup>

## Selection Policy

Currently, the optimizer supports these selection policies:

* `best_by_metric` (default): score each candidate with the metric and pick the best.
* `first`: pick the first candidate (fast, deterministic, but ignores scoring).
* `concat`: join all candidates into one output string.
* `random`: pick a random candidate (seeded if provided).
* `max_logprob`: pick the candidate with the highest average token logprob (provider support required; logprobs must be enabled in model kwargs).

Use the `selection_policy` key in `model_parameters` to override. The optimizer
routes these policies through a shared candidate-selection utility so behavior
is consistent across optimizers:

```python
prompt = ChatPrompt(
    model="gpt-4o-mini",
    messages=[...],
    model_parameters={
        "n": 3,
        "selection_policy": "first",
    },
)
```

For `max_logprob`, enable logprobs in your model kwargs (provider support varies):

```python
prompt = ChatPrompt(
    model="gpt-4o-mini",
    messages=[...],
    model_parameters={
        "n": 3,
        "selection_policy": "max_logprob",
        "logprobs": True,
        "top_logprobs": 1,
    },
)
```

When `selection_policy=best_by_metric`, the optimizer:

1. Each candidate is scored independently using your metric function
2. The candidate with the highest score is selected as the final output
3. All scores and the chosen index are logged to the trace metadata

```python