# Tone Consistency Evaluation
[EN] Source: https://mastra.ai/examples/evals/tone-consistency

:::note
The Scorers API is currently in beta. We're actively working on improvements and appreciate your feedback. For questions or feature requests, please reach out on [Discord](https://discord.gg/mastra).
:::

Use `ToneConsistencyMetric` to evaluate emotional tone patterns and sentiment consistency in text. The metric accepts a `query` and a `response`, and returns a score and an `info` object containing sentiment scores and their difference.

## Installation

```bash copy
npm install @mastra/evals
```

## Positive tone example

In this example, the texts exhibit a similar positive sentiment. The metric measures the consistency between the tones, resulting in a high score.

```typescript title="src/example-positive-tone.ts" showLineNumbers copy
import { ToneConsistencyMetric } from "@mastra/evals/nlp";

const metric = new ToneConsistencyMetric();

const query = "This product is fantastic and amazing!";
const response = "The product is excellent and wonderful!";

const result = await metric.measure(query, response);

console.log(result);
```

### Positive tone output

The metric returns a high score reflecting strong sentiment alignment. The `info` field provides sentiment values and the difference between them.

```typescript
{
  score: 0.8333333333333335,
  info: {
    responseSentiment: 1.3333333333333333,
    referenceSentiment: 1.1666666666666667,
    difference: 0.16666666666666652
  }
}
```

## Stable tone example

In this example, the text’s internal tone consistency is analyzed by passing an empty response. This signals the metric to evaluate sentiment stability within the single input text, resulting in a score reflecting how uniform the tone is throughout.

```typescript title="src/example-stable-tone.ts" showLineNumbers copy
import { ToneConsistencyMetric } from "@mastra/evals/nlp";

const metric = new ToneConsistencyMetric();

const query = "Great service! Friendly staff. Perfect atmosphere.";
const response = "";

const result = await metric.measure(query, response);

console.log(result);
```

### Stable tone output

The metric returns a high score indicating consistent sentiment throughout the input text. The `info` field includes the average sentiment and sentiment variance, reflecting tone stability.

```typescript
{
  score: 0.9444444444444444,
  info: {
    avgSentiment: 1.3333333333333333,
    sentimentVariance: 0.05555555555555556
  }
}
```

## Mixed tone example

In this example, the input and response have different emotional tones. The metric picks up on these variations and gives a lower consistency score.

```typescript title="src/example-mixed-tone.ts" showLineNumbers copy
import { ToneConsistencyMetric } from "@mastra/evals/nlp";

const metric = new ToneConsistencyMetric();

const query =
  "The interface is frustrating and confusing, though it has potential.";
const response =
  "The design shows promise but needs significant improvements to be usable.";

const result = await metric.measure(query, response);

console.log(result);
```

### Mixed tone output

The metric returns a low score due to the noticeable differences in emotional tone. The `info` field highlights the sentiment values and the degree of variation between them.

```typescript
{
  score: 0.4181818181818182,
  info: {
    responseSentiment: -0.4,
    referenceSentiment: 0.18181818181818182,
    difference: 0.5818181818181818
  }
}
```

## Metric configuration

You can create a `ToneConsistencyMetric` instance with default settings. No additional configuration is required.

```typescript
const metric = new ToneConsistencyMetric();
```

> See [ToneConsistencyMetric](/reference/evals/tone-consistency) for a full list of configuration options.

## Understanding the results

`ToneConsistencyMetric` returns a result in the following shape:

```typescript
{
  score: number,
  info: {
    responseSentiment?: number,
    referenceSentiment?: number,
    difference?: number,
    avgSentiment?: number,
    sentimentVariance?: number
  }
}
```

### Tone consistency score

A tone consistency score between 0 and 1:

- **0.8–1.0**: Very consistent tone.
- **0.6–0.7**: Generally consistent tone.
- **0.4–0.5**: Mixed tone.
- **0.0–0.3**: Conflicting tone.

### Tone consistency info

An explanation for the score, with details including:

- Sentiment alignment between input and response.
- Tone stability within a single text.
- Degree of sentiment difference or variance.

<GithubLink
  outdated={true}
  marginTop="mt-16"
  link="https://github.com/mastra-ai/mastra/blob/main/examples/basics/evals/tone-consistency"
/>


---
title: "Example: Toxicity Evaluation | Evals"
description: Example of using the Toxicity metric to evaluate responses for harmful content and toxic language.
---