# Main grading entrypoint (must be named `grade`)
def grade(sample: dict, item: dict) -> float:
    """Compute an F1âstyle score for citation extraction answers using RapidFuzz.
    """
    model_passages = (sample.get('output_json') or {}).get('passages', [])
    ref_passages = (item.get('reference_answer') or {}).get('passages', [])

    # If there are no reference passages, return 0.
    if not ref_passages:
        return 0.0

    # Recall: average best match for each reference passage.
    recall_scores = []
    for ref in ref_passages:
        best = 0.0
        for out in model_passages:
            score = fuzz_ratio(ref, out)
            if score > best:
                best = score
        recall_scores.append(best)
    recall = sum(recall_scores) / len(recall_scores)

    # Precision: average best match for each model passage.
    if not model_passages:
        precision = 0.0
    else:
        precision_scores = []
        for out in model_passages:
            best = 0.0
            for ref in ref_passages:
                score = fuzz_ratio(ref, out)
                if score > best:
                    best = score
            precision_scores.append(best)
        precision = sum(precision_scores) / len(precision_scores)

    if precision + recall == 0:
        return 0.0

    return 2 * precision * recall / (precision + recall)
```

Results

> After reinforcement fine-tuning, Harvey saw a **20% increase** in the F1
> score:
>
> - Baseline F1: 0.563
> - Post-RFT F1 - 0.6765
>
> Using RFT, Harvey significantly improved legal fact-extraction performance,
> surpassing GPT-4o efficiency and accuracy. Early trials showed RFT **winning
> or tying in 93% of comparisons** against GPT-4o.
>
> âThe RFT model demonstrated comparable or superior performance to GPT-4o, but
> with significantly faster inference, proving particularly beneficial for
> real-world legal use cases.
>
> âHarvey, AI for legal teams

### 3\. Apply complex rules correctly

This use case involves pulling verifiable facts or entities from unstructured
inputs into clearly defined schemas (e.g., JSON objects, condition codes,
medical codes, legal citations, or financial metrics).

Successful extraction tasks typically benefit from precise, continuous grading
methodologiesâlike span-level F1 scores, fuzzy text-matching metrics, or numeric
accuracy checksâto evaluate how accurately the extracted information aligns with
ground truth. Define explicit success criteria and detailed rubrics. Then, the
model can achieve reliable, repeatable improvements.

#### Expert-level reasoning in tax analysis

Use case

> **Company**: Accordance is building a platform for tax, audit, and CPA teams.
>
> **Problem to solve**: Taxation is a highly complex domain, requiring deep
> reasoning across nuanced fact patterns and intricate regulations. It's also a
> field that continues changing.
>
> **Objective**: Accordance wanted a high-trust system for sophisticated tax
> scenarios while maintaining accuracy. Unlike traditional hardcoded software,
> it's important that their data extraction tool adapts as the tax landscape
> evolves.

Grader code

```text
[+0.05] For correctly identifying Alex (33.33%), Barbara (33.33% â 20%), Chris (33.33%), and Dana (13.33%) ownership percentages
[+0.1] For correctly calculating Barbara's annual allocation as 26.67% and Dana's as 6.67% without closing of books
[+0.15] For properly allocating Alex ($300,000), Barbara ($240,030), Chris ($300,000), and Dana ($60,030) ordinary income
[+0.1] For calculating Alex's ending stock basis as $248,333 and debt basis as $75,000
[+0.05] For calculating Barbara's remaining basis after sale as $264,421
[+0.1] For calculating AAA before distributions as $1,215,000 and ending AAA as $315,000
[+0.1] For identifying all distributions as tax-free return of capital under AAA
[+0.1] For calculating Barbara's capital gain on stock sale as $223,720 ($400,000 - $176,280)
[+0.1] For explaining that closing of books would allocate based on actual half-year results
[+0.05] For identifying the ordering rules: AAA first, then E&P ($120,000), then remaining basis
[+0.05] For noting distributions exceeding $1,215,000 would be dividends up to $120,000 E&P
[+0.05] For correctly accounting for separately stated items in basis calculations (e.g., $50,000 Section 1231 gain)
```

Results

> By collaborating with OpenAI and their in-house tax experts, Accordance
> achieved:
>
> - Almost **40% improvement** in tax analysis tasks over base models
> - Superior performance compared to all other leading models on benchmarks like
>   TaxBench
> - The RFT-trained models demonstrated an ability to handle advanced tax
>   scenarios with high accuracyâwhen evaluated by tax professionals,
>   Accordanceâs fine-tuned models showed expert-level reasoning, with the
>   potential to save thousands of hours of manual work
>
> âWeâve achieved a 38.89% improvement in our tax analysis tasks over base
> models and significantly outperformed all other leading models on key tax
> benchmarks (including TaxBench). The RFT-trained modelsâ abilities to handle
> sophisticated tax scenarios while maintaining accuracy demonstrates the
> readiness of reinforcement fine-tuningâand AI more broadlyâfor professional
> applications. Most importantly, RFT provides a foundation for continuous
> adaptation as the tax landscape evolves, ensuring sustained value and
> relevance. When evaluated by tax experts, our fine-tuned models demonstrated
> expert-level reasoning capabilities that will save thousands of professional
> hoursâthis isnât just an incremental improvement, itâs a paradigm shift in how
> tax work can be done.â
>
> âAccordance, AI tax accounting company

#### Enforcement of nuanced content moderation policies

Use case

> **Company**: SafetyKit is a risk and compliance platform that helps
> organizations make decisions across complex content moderation workflows.
>
> **Problem to solve**: These systems must handle large volumes of content and
> apply intricate policy logic that requires multistep reasoning. Because of the
> volume of data and subtle distinctions in labelling, these types of tasks can
> be difficult for general purpose models.
>
> **Objective**: SafetyKit aimed to replace multiple nodes in their most complex
> workflows with a single reasoning agent using a reinforcement fine-tuned
> model. The goal is to reduce SafetyKitâs time-to-market for novel policy
> enforcements even in challenging, nuanced domains.

Results

> SafetyKit is using their o3-mini RFT model to support advanced content
> moderation capabilities, ensuring user safety for one of the largest AI
> chatbot companies in the world. They have successfully improved F1-score
> **from 86% to 90%**, soon to replace dozens of 4o calls within their
> production pipeline.
>
> "SafetyKitâs RFT-enabled moderation achieved substantial improvements in
> nuanced content moderation tasks, crucial for safeguarding users in dynamic,
> real-world scenarios."
>
> âSafetyKit

#### Legal document reviews, comparisons, and summaries

Use case

> **Company**: Thomson Reuters is an AI and technology company empowering
> professionals with trusted content and workflow automation.
>
> **Problem to solve**: Legal professionals must read through large amounts of
> content before making any decisions. Thomson Reuter's CoCounsel product is
> designed to help these experts move faster by providing an AI assistant with
> content and industry knowledge. The models that power this tool must
> understand complex legal rules.
>
> **Objective**: Thomson Reuters aimed to create a reinforcement fine-tuned
> model excelling in legal AI skills. They conducted preliminary evaluations of
> RFT to see if they could achieve model performance improvements, using
> specialized datasets from three highly-used CoCounsel Legal AI skills for
> legal professionals:
>
> 1.  Review documents: Generates detailed answers to questions asked against
>     contracts, transcripts, and other legal documents
> 2.  Compare documents: Highlights substantive differences between two or more
>     different contracts or documents
> 3.  Summarize: Summarizes the most important information within one or more
>     documents to enable rapid legal review

Results

> ![Provide example data and create a fine-tuning job to optimize model performance for your use case](https://cdn.openai.com/API/docs/images/thomsonreuters-results.png)
>
> "LLM as a judge has been helpful in demonstrating the possibility of improving
> upon the reasoning models - in preliminary evaluations, the RFT model
> consistently performed better than the baseline o3-mini and o1 model"
>
> âThomson Reuters, AI and technology company

## Evals are the foundation

**Before implementing RFT, we strongly recommended creating and running an eval
for the task you intend to fine-tune on**. If the model you intend to fine-tune
scores at either the absolute minimum or absolute maximum possible score, then
RFT wonât be useful to you.

RFT works by reinforcing better answers to provided prompts. If we canât
distinguish the quality of different answers (i.e., if they all receive the
minimum or maximum possible score), then there's no training signal to learn
from. However, if your eval scores somewhere in the range between the minimum
and maximum possible scores, there's enough data to work with.

An effective eval reveals opportunities where human experts consistently agree
but current frontier models struggle, presenting a valuable gap for RFT to
close. [Get started with evals](https://platform.openai.com/docs/guides/evals).

## How to get better results from RFT

To see improvements in your fine-tuned model, there are two main places to
revisit and refine: making sure your task is well defined, and making your
grading scheme more robust.

### Reframe or clarify your task

Good tasks give the model a fair chance to learn and let you quantify
improvements.

- **Start with a task the model can already solve occasionally**. RFT works by
  sampling many answers, keeping what looks best, and nudging the model toward
  those answers. If the model never gets the answer correct today, it cannot
  improve.
- **Make sure each answer can be graded**. A grader must read an answer and
  produce a score without a person in the loop. We support multiple
  [grader types](https://platform.openai.com/docs/guides/graders), including
  custom Python graders and LLM judges. If you can't write code to judge the
  answer with an available grader, RFT is not the right tool.
- **Remove doubt about the ârightâ answer**. If two careful people often
  disagree on the solution, the task is too fuzzy. Rewrite the prompt, add
  context, or split the task into clearer parts until domain experts agree.
- **Limit lucky guesses**. If the task is multiple choice with one obvious best
  pick, the model can win by chance. Add more classes, ask for short openâended
  text, or tweak the format so guessing is costly.

### Strengthen your grader

Clear, robust grading schemes are essential for RFT.

- **Produce a smooth score, not a pass/fail stamp**. A score that shifts
  gradually as answers improve provides a better training signal.
- **Guard against reward hacking**. This happens when the model finds a shortcut
  that earns high scores without real skill.
- **Avoid skewed data**. Datasets in which one label shows up most of the time
  invite the model to guess that label. Balance the set or upâweight rare cases
  so the model must think.
- **Use an LLM judge when code falls short**. For rich, openâended answers, have
  a
  [separate OpenAI model grade](https://platform.openai.com/docs/guides/graders#model-graders)
  your fine-tuned model's answers. Make sure you:
  - **Evaluate the judge**: Run multiple candidate responses and correct answers
    through your LLM judge to ensure the grade returned is stable and aligned
    with preference.
  - **Provide few-shot examples**. Include great, fair, and poor answers in the
    prompt to improve the grader's effectiveness.

Learn more about
[grader types](https://platform.openai.com/docs/guides/graders).

## Other resources

For more inspiration, visit the OpenAI Cookbook, which contains example code and
links to third-party resources, or learn more about our models and reasoning
capabilities:

- [Meet the models](https://platform.openai.com/docs/models)
- [Reinforcement fine-tuning guide](https://platform.openai.com/docs/guides/reinforcement-fine-tuning)
- [Graders](https://platform.openai.com/docs/guides/graders)
- [Model optimization overview](https://platform.openai.com/docs/guides/model-optimization)