# run the grader with a test reference and sample
payload = {
  "grader": grader,
  "item": {
     "reference_answer": 1.0
  },
  "model_sample": "0.9"
}
response = requests.post(
    "https://api.openai.com/v1/fine_tuning/alpha/graders/run",
    json=payload,
    headers=headers
)
print("run response:", response.text)
```

#### Score model grader outputs

Under the hood, the `score_model` grader will query the requested model with the
provided prompt and sampling parameters and will request a response in a
specific response format. The response format that is used is provided below

```json
{
  "result": float,
  "steps": ReasoningStep[],
}
```

Where each reasoning step is of the form

```json
{
    description: string,
    conclusion: string
}
```

This format queries the model not just for the numeric `result` (the reward
value for the query), but also provides the model some space to think through
the reasoning behind the score. When you are writing your grader prompt, it may
be useful to refer to these two fields by name explicitly (e.g. "include
reasoning about the type of chemical bonds present in the molecule in the
conclusion of your reasoning step", or "return a value of -1.0 in the `result`
field if the inputs do not satisfy condition X").

### Label model graders

A label model grader will take the input and a set of passing labels and return
a 1 if the model output is within the label set and 0 otherwise.

```json
{
	"type": "label_model",
    "name": string,
	"model": string,
	"input": Message[],
	"passing_labels": string[],
	"labels": string[],
	"sampling_params": {
        "max_tokens": 32768,
        "top_p": 1,
        "reasoning_effort": "medium"
    }
}
```

To use a label model grader, the input is a list of chat messages, each
containing a `role` and `content`. The output of the grader will be limited to
the given set of labels. Within each message, the same templating can be used as
with other common graders to reference the ground truth or model sample.

Hereâs a full runnable code sample:

```python
import os
import requests