# Textual Difference Scorer
[EN] Source: https://mastra.ai/reference/scorers/textual-difference

The `createTextualDifferenceScorer()` function uses sequence matching to measure the textual differences between two strings. It provides detailed information about changes, including the number of operations needed to transform one text into another.

## Parameters

The `createTextualDifferenceScorer()` function does not take any options.

This function returns an instance of the MastraScorer class. See the [MastraScorer reference](./mastra-scorer) for details on the `.run()` method and its input/output.

## .run() Returns

<PropertiesTable
  content={[
    {
      name: "runId",
      type: "string",
      description: "The id of the run (optional).",
    },
    {
      name: "analyzeStepResult",
      type: "object",
      description:
        "Object with difference metrics: { confidence: number, changes: number, lengthDiff: number }",
    },
    {
      name: "score",
      type: "number",
      description: "Similarity ratio (0-1) where 1 indicates identical texts.",
    },
  ]}
/>

`.run()` returns a result in the following shape:

```typescript
{
  runId: string,
  analyzeStepResult: {
    confidence: number,
    ratio: number,
    changes: number,
    lengthDiff: number
  },
  score: number
}
```

## Scoring Details

The scorer calculates several measures:

- **Similarity Ratio**: Based on sequence matching between texts (0-1)
- **Changes**: Count of non-matching operations needed
- **Length Difference**: Normalized difference in text lengths
- **Confidence**: Inversely proportional to length difference

### Scoring Process

1. Analyzes textual differences:
   - Performs sequence matching between input and output
   - Counts the number of change operations required
   - Measures length differences
2. Calculates metrics:
   - Computes similarity ratio
   - Determines confidence score
   - Combines into weighted score

Final score: `(similarity_ratio * confidence) * scale`

### Score interpretation

A textual difference score between 0 and 1:

- **1.0**: Identical texts – no differences detected.
- **0.7–0.9**: Minor differences – few changes needed.
- **0.4–0.6**: Moderate differences – noticeable changes required.
- **0.1–0.3**: Major differences – extensive changes needed.
- **0.0**: Completely different texts.

## Examples

### No differences example

In this example, the texts are exactly the same. The scorer identifies complete similarity with a perfect score and no detected changes.

```typescript title="src/example-no-differences.ts" showLineNumbers copy
import { createTextualDifferenceScorer } from "@mastra/evals/scorers/code";

const scorer = createTextualDifferenceScorer();

const input = "The quick brown fox jumps over the lazy dog";
const output = "The quick brown fox jumps over the lazy dog";

const result = await scorer.run({
  input: [{ role: "user", content: input }],
  output: { role: "assistant", text: output },
});

console.log("Score:", result.score);
console.log("AnalyzeStepResult:", result.analyzeStepResult);
```

#### No differences output

The scorer returns a high score, indicating the texts are identical. The detailed info confirms zero changes and no length difference.

```typescript
{
  score: 1,
  analyzeStepResult: {
    confidence: 1,
    ratio: 1,
    changes: 0,
    lengthDiff: 0,
  },
}
```

### Minor differences example

In this example, the texts have small variations. The scorer detects these minor differences and returns a moderate similarity score.

```typescript title="src/example-minor-differences.ts" showLineNumbers copy
import { createTextualDifferenceScorer } from "@mastra/evals/scorers/code";

const scorer = createTextualDifferenceScorer();

const input = "Hello world! How are you?";
const output = "Hello there! How is it going?";

const result = await scorer.run({
  input: [{ role: "user", content: input }],
  output: { role: "assistant", text: output },
});

console.log("Score:", result.score);
console.log("AnalyzeStepResult:", result.analyzeStepResult);
```

#### Minor differences output

The scorer returns a moderate score reflecting the small variations between the texts. The detailed info includes the number of changes and length difference observed.

```typescript
{
  score: 0.5925925925925926,
  analyzeStepResult: {
    confidence: 0.8620689655172413,
    ratio: 0.5925925925925926,
    changes: 5,
    lengthDiff: 0.13793103448275862
  }
}
```

### Major differences example

In this example, the texts differ significantly. The scorer detects extensive changes and returns a low similarity score.

```typescript title="src/example-major-differences.ts" showLineNumbers copy
import { createTextualDifferenceScorer } from "@mastra/evals/scorers/code";

const scorer = createTextualDifferenceScorer();

const input = "Python is a high-level programming language";
const output = "JavaScript is used for web development";

const result = await scorer.run({
  input: [{ role: "user", content: input }],
  output: { role: "assistant", text: output },
});

console.log("Score:", result.score);
console.log("AnalyzeStepResult:", result.analyzeStepResult);
```

#### Major differences output

The scorer returns a low score due to significant differences between the texts. The detailed `analyzeStepResult` shows numerous changes and a notable length difference.

```typescript
{
  score: 0.3170731707317073,
  analyzeStepResult: {
    confidence: 0.8636363636363636,
    ratio: 0.3170731707317073,
    changes: 8,
    lengthDiff: 0.13636363636363635
  }
}
```

## Related

- [Content Similarity Scorer](./content-similarity)
- [Completeness Scorer](./completeness)
- [Keyword Coverage Scorer](./keyword-coverage)


---
title: "Reference: Tone Consistency Scorer | Scorers"
description: Documentation for the Tone Consistency Scorer in Mastra, which evaluates emotional tone and sentiment consistency in text.
---