# Context Precision Scorer
[EN] Source: https://mastra.ai/reference/scorers/context-precision

The `createContextPrecisionScorer()` function creates a scorer that evaluates how relevant and well-positioned retrieved context pieces are for generating expected outputs. It uses **Mean Average Precision (MAP)** to reward systems that place relevant context earlier in the sequence.

It is especially useful for these use cases:

**RAG System Evaluation**

Ideal for evaluating retrieved context in RAG pipelines where:

- Context ordering matters for model performance
- You need to measure retrieval quality beyond simple relevance
- Early relevant context is more valuable than later relevant context

**Context Window Optimization**

Use when optimizing context selection for:

- Limited context windows
- Token budget constraints
- Multi-step reasoning tasks

## Parameters

<PropertiesTable
  content={[
    {
      name: "model",
      type: "MastraModelConfig",
      description: "The language model to use for evaluating context relevance",
      required: true,
    },
    {
      name: "options",
      type: "ContextPrecisionMetricOptions",
      description: "Configuration options for the scorer",
      required: true,
      children: [
        {
          name: "context",
          type: "string[]",
          description: "Array of context pieces to evaluate for relevance",
          required: false,
        },
        {
          name: "contextExtractor",
          type: "(input, output) => string[]",
          description:
            "Function to dynamically extract context from the run input and output",
          required: false,
        },
        {
          name: "scale",
          type: "number",
          description: "Scale factor to multiply the final score (default: 1)",
          required: false,
        },
      ],
    },
  ]}
/>

**Note**: Either `context` or `contextExtractor` must be provided. If both are provided, `contextExtractor` takes precedence.

## .run() Returns

<PropertiesTable
  content={[
    {
      name: "score",
      type: "number",
      description:
        "Mean Average Precision score between 0 and scale (default 0-1)",
    },
    {
      name: "reason",
      type: "string",
      description:
        "Human-readable explanation of the context precision evaluation",
    },
  ]}
/>

## Scoring Details

### Mean Average Precision (MAP)

Context Precision uses **Mean Average Precision** to evaluate both relevance and positioning:

1. **Context Evaluation**: Each context piece is classified as relevant or irrelevant for generating the expected output
2. **Precision Calculation**: For each relevant context at position `i`, precision = `relevant_items_so_far / (i + 1)`
3. **Average Precision**: Sum all precision values and divide by total relevant items
4. **Final Score**: Multiply by scale factor and round to 2 decimals

### Scoring Formula

```
MAP = (Σ Precision@k) / R

Where:
- Precision@k = (relevant items in positions 1...k) / k
- R = total number of relevant items
- Only calculated at positions where relevant items appear
```

### Score Interpretation

- **0.9-1.0**: Excellent precision - all relevant context early in sequence
- **0.7-0.8**: Good precision - most relevant context well-positioned
- **0.4-0.6**: Moderate precision - relevant context mixed with irrelevant
- **0.1-0.3**: Poor precision - little relevant context or poorly positioned
- **0.0**: No relevant context found

### Reason analysis

The reason field explains:

- Which context pieces were deemed relevant/irrelevant
- How positioning affected the MAP calculation
- Specific relevance criteria used in evaluation

### Optimization insights

Use results to:

- **Improve retrieval**: Filter out irrelevant context before ranking
- **Optimize ranking**: Ensure relevant context appears early
- **Tune chunk size**: Balance context detail vs. relevance precision
- **Evaluate embeddings**: Test different embedding models for better retrieval

### Example Calculation

Given context: `[relevant, irrelevant, relevant, irrelevant]`

- Position 0: Relevant → Precision = 1/1 = 1.0
- Position 1: Skip (irrelevant)
- Position 2: Relevant → Precision = 2/3 = 0.67
- Position 3: Skip (irrelevant)

MAP = (1.0 + 0.67) / 2 = 0.835 ≈ **0.83**

## Scorer configuration

### Dynamic context extraction

```typescript
const scorer = createContextPrecisionScorer({
  model: "openai/gpt-4o-mini",
  options: {
    contextExtractor: (input, output) => {
      // Extract context dynamically based on the query
      const query = input?.inputMessages?.[0]?.content || "";

      // Example: Retrieve from a vector database
      const searchResults = vectorDB.search(query, { limit: 10 });
      return searchResults.map((result) => result.content);
    },
    scale: 1,
  },
});
```

### Large context evaluation

```typescript
const scorer = createContextPrecisionScorer({
  model: "openai/gpt-4o-mini",
  options: {
    context: [
      // Simulate retrieved documents from vector database
      "Document 1: Highly relevant content...",
      "Document 2: Somewhat related content...",
      "Document 3: Tangentially related...",
      "Document 4: Not relevant...",
      "Document 5: Highly relevant content...",
      // ... up to dozens of context pieces
    ],
  },
});
```

## Examples

### High precision example

This example shows perfect context precision where all relevant context appears early:

```typescript
import { createContextPrecisionScorer } from "@mastra/evals";

const scorer = createContextPrecisionScorer({
  model: "openai/gpt-4o-mini",
  options: {
    context: [
      "Photosynthesis is the process by which plants convert sunlight, carbon dioxide, and water into glucose and oxygen.",
      "The process occurs in the chloroplasts of plant cells, specifically in the thylakoids.",
      "Light-dependent reactions happen in the thylakoid membranes, while the Calvin cycle occurs in the stroma.",
    ],
    scale: 1,
  },
});

const result = await scorer.run({
  input: {
    inputMessages: [
      {
        id: "1",
        role: "user",
        content: "How does photosynthesis work in plants?",
      },
    ],
  },
  output: [
    {
      id: "2",
      role: "assistant",
      content:
        "Photosynthesis is the process where plants convert sunlight, CO2, and water into glucose and oxygen using chloroplasts.",
    },
  ],
});

console.log(result);
// Output:
// {
//   score: 1.0,
//   reason: "The score is 1.0 because all context pieces are highly relevant to explaining photosynthesis and are optimally ordered to support the expected output."
// }
```

### Mixed precision example

This example shows moderate precision with both relevant and irrelevant context:

```typescript
import { createContextPrecisionScorer } from "@mastra/evals";

const scorer = createContextPrecisionScorer({
  model: "openai/gpt-4o-mini",
  options: {
    context: [
      "Regular exercise improves cardiovascular health by strengthening the heart muscle.",
      "A balanced diet should include fruits, vegetables, and whole grains.",
      "Physical activity releases endorphins which improve mood and reduce stress.",
      "The average person should drink 8 glasses of water per day.",
      "Exercise also helps maintain healthy body weight and muscle mass.",
    ],
    scale: 1,
  },
});

const result = await scorer.run({
  input: {
    inputMessages: [
      {
        id: "1",
        role: "user",
        content: "What are the mental and physical benefits of exercise?",
      },
    ],
  },
  output: [
    {
      id: "2",
      role: "assistant",
      content:
        "Exercise provides cardiovascular benefits, improves mood through endorphin release, and helps maintain healthy body composition.",
    },
  ],
});

console.log(result);
// Output:
// {
//   score: 0.72,
//   reason: "The score is 0.72 because contexts 1, 3, and 5 are relevant to exercise benefits, but irrelevant contexts about diet and hydration reduce the precision score."
// }
```

### Low precision example

This example shows poor context precision with mostly irrelevant context:

```typescript
import { createContextPrecisionScorer } from "@mastra/evals";

const scorer = createContextPrecisionScorer({
  model: "openai/gpt-4o-mini",
  options: {
    context: [
      "The weather forecast shows sunny skies this weekend.",
      "Coffee is one of the world's most popular beverages.",
      "Machine learning requires large amounts of training data.",
      "Cats typically sleep 12-16 hours per day.",
      "The capital of France is Paris.",
    ],
    scale: 1,
  },
});

const result = await scorer.run({
  input: {
    inputMessages: [
      {
        id: "1",
        role: "user",
        content: "How does photosynthesis work?",
      },
    ],
  },
  output: [
    {
      id: "2",
      role: "assistant",
      content:
        "Photosynthesis is the process by which plants convert sunlight into energy using chlorophyll.",
    },
  ],
});

console.log(result);
// Output:
// {
//   score: 0.0,
//   reason: "The score is 0.0 because none of the retrieved context pieces are relevant to explaining photosynthesis."
// }
```

## Comparison with Context Relevance

Choose the right scorer for your needs:

| Use Case                 | Context Relevance    | Context Precision         |
| ------------------------ | -------------------- | ------------------------- |
| **RAG evaluation**       | When usage matters   | When ranking matters      |
| **Context quality**      | Nuanced levels       | Binary relevance          |
| **Missing detection**    | ✓ Identifies gaps    | ✗ Not evaluated           |
| **Usage tracking**       | ✓ Tracks utilization | ✗ Not considered          |
| **Position sensitivity** | ✗ Position agnostic  | ✓ Rewards early placement |

## Related

- [Answer Relevancy Scorer](/reference/scorers/answer-relevancy) - Evaluates if answers address the question
- [Faithfulness Scorer](/reference/scorers/faithfulness) - Measures answer groundedness in context
- [Custom Scorers](/docs/scorers/custom-scorers) - Creating your own evaluation metrics


---
title: "Reference: Context Relevance Scorer | Scorers"
description: Documentation for the Context Relevance Scorer in Mastra. Evaluates the relevance and utility of provided context for generating agent responses using weighted relevance scoring.
---

import PropertiesTable from "@site/src/components/PropertiesTable";