# ToneConsistencyMetric
[EN] Source: https://mastra.ai/reference/evals/tone-consistency

:::info Scorers
This documentation refers to the legacy evals API. For the latest scorer features, see [Scorers](/docs/scorers/overview).
:::

The `ToneConsistencyMetric` class evaluates the text's emotional tone and sentiment consistency. It can operate in two modes: comparing tone between input/output pairs or analyzing tone stability within a single text.

## Basic Usage

```typescript
import { ToneConsistencyMetric } from "@mastra/evals/nlp";

const metric = new ToneConsistencyMetric();

// Compare tone between input and output
const result1 = await metric.measure(
  "I love this amazing product!",
  "This product is wonderful and fantastic!",
);

// Analyze tone stability in a single text
const result2 = await metric.measure(
  "The service is excellent. The staff is friendly. The atmosphere is perfect.",
  "", // Empty string for single-text analysis
);

console.log(result1.score); // Tone consistency score from 0-1
console.log(result2.score); // Tone stability score from 0-1
```

## measure() Parameters

<PropertiesTable
  content={[
    {
      name: "input",
      type: "string",
      description: "The text to analyze for tone",
      isOptional: false,
    },
    {
      name: "output",
      type: "string",
      description:
        "Reference text for tone comparison (empty string for stability analysis)",
      isOptional: false,
    },
  ]}
/>

## Returns

<PropertiesTable
  content={[
    {
      name: "score",
      type: "number",
      description: "Tone consistency/stability score (0-1)",
    },
    {
      name: "info",
      type: "object",
      description: "Detailed tone info",
    },
  ]}
/>

### info Object (Tone Comparison)

<PropertiesTable
  content={[
    {
      name: "responseSentiment",
      type: "number",
      description: "Sentiment score for the input text",
    },
    {
      name: "referenceSentiment",
      type: "number",
      description: "Sentiment score for the output text",
    },
    {
      name: "difference",
      type: "number",
      description: "Absolute difference between sentiment scores",
    },
  ]}
/>

### info Object (Tone Stability)

<PropertiesTable
  content={[
    {
      name: "avgSentiment",
      type: "number",
      description: "Average sentiment score across sentences",
    },
    {
      name: "sentimentVariance",
      type: "number",
      description: "Variance in sentiment between sentences",
    },
  ]}
/>

## Scoring Details

The metric evaluates sentiment consistency through tone pattern analysis and mode-specific scoring.

### Scoring Process

1. Analyzes tone patterns:
   - Extracts sentiment features
   - Computes sentiment scores
   - Measures tone variations

2. Calculates mode-specific score:
   **Tone Consistency** (input and output):
   - Compares sentiment between texts
   - Calculates sentiment difference
   - Score = 1 - (sentiment_difference / max_difference)

   **Tone Stability** (single input):
   - Analyzes sentiment across sentences
   - Calculates sentiment variance
   - Score = 1 - (sentiment_variance / max_variance)

Final score: `mode_specific_score * scale`

### Score interpretation

(0 to scale, default 0-1)

- 1.0: Perfect tone consistency/stability
- 0.7-0.9: Strong consistency with minor variations
- 0.4-0.6: Moderate consistency with noticeable shifts
- 0.1-0.3: Poor consistency with major tone changes
- 0.0: No consistency - completely different tones

## Example with Both Modes

```typescript
import { ToneConsistencyMetric } from "@mastra/evals/nlp";

const metric = new ToneConsistencyMetric();

// Tone Consistency Mode
const consistencyResult = await metric.measure(
  "This product is fantastic and amazing!",
  "The product is excellent and wonderful!",
);
// Example output:
// {
//   score: 0.95,
//   info: {
//     responseSentiment: 0.8,
//     referenceSentiment: 0.75,
//     difference: 0.05
//   }
// }

// Tone Stability Mode
const stabilityResult = await metric.measure(
  "Great service! Friendly staff. Perfect atmosphere.",
  "",
);
// Example output:
// {
//   score: 0.9,
//   info: {
//     avgSentiment: 0.6,
//     sentimentVariance: 0.1
//   }
// }
```

## Related

- [Content Similarity Metric](./content-similarity)
- [Toxicity Metric](./toxicity)


---
title: "Reference: ToxicityMetric | Evals"
description: Documentation for the Toxicity Metric in Mastra, which evaluates LLM outputs for racist, biased, or toxic elements.
---