# Source: https://docs.runpod.io/serverless/endpoints/send-requests.md

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.runpod.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Send API requests

> Submit and manage jobs for your queue-based endpoints by sending HTTP requests.

<Warning>
  This guide is for **queue-based endpoints**. If you're building a [load balancing endpoint](/serverless/load-balancing/overview), the request structure and endpoints will depend on how you define your HTTP servers.
</Warning>

After creating a [Severless endpoint](/serverless/endpoints/overview), you can start sending it **requests** to submit jobs and retrieve results. This page covers everything from basic input structure and job submission, to monitoring, troubleshooting, and advanced options for queue-based endpoints.

<iframe className="w-full aspect-video rounded-xl" src="https://www.youtube.com/embed/BURJRgjNNJI?si=wdbNI58YGdPI7JyE" title="Send requests from the console" frameBorder="0" allow="fullscreen; accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowFullScreen />

## How requests work

A request can include parameters, payloads, and headers that define what the endpoint should process. For example, you can send a `POST` request to submit a job, or a `GET` request to check the status of a job, retrieve results, or check endpoint health.

A **job** is a unit of work containing the input data from the request, packaged for processing by your [workers](/serverless/workers/overview).

If no worker is immediately available, the job is queued. Once a worker is available, the job is processed using your worker's [handler function](/serverless/workers/handler-functions).

Queue-based endpoints provide a fixed set of operations for submitting and managing jobs. You can find a full list of operations and sample code in the [sections below](/serverless/endpoints/send-requests#operation-overview).

## Sync vs. async

When you submit a job request, it can be either synchronous or asynchronous depending on which `POST` operation you use:

* `/runsync` submits a synchronous job.
  * Client waits for the job to complete before returning the result.
  * A response is returned as soon as the job is complete.
  * Results are available for 1 minute by default (5 minutes max).
  * Ideal for quick responses and interactive applications.
* `/run` submits an asynchronous job.
  * The job is processed in the background.
  * Retrieve the result by sending a `GET` request to the `/status` endpoint.
  * Results are available for 30 minutes after completion.
  * Ideal for long-running tasks and batch processing.

## Request input structure

When submitting a job with `/runsync` or `/run`, your request must include a JSON object with the key `input` containing the parameters required by your worker's [handler function](/serverless/workers/handler-functions). For example:

```json  theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "input": {
    "prompt": "Your input here"
  }
}
```

The exact parameters required in the `input` object depend on your specific worker implementation (e.g. `prompt` commonly used for endpoints serving LLMs, but not all workers accept it). Check your worker's documentation for a list of required and optional parameters.

## Send requests from the console

The quickest way to test your endpoint is directly in the Runpod console. Navigate to the [Serverless section](https://www.console.runpod.io/serverless), select your endpoint, and click the **Requests** tab.

<Frame>
  <img src="https://mintcdn.com/runpod-b18f5ded/QcR4sHy3480YmZ2d/images/8f34ba77-serverless-get-started-endpoint-details.png?fit=max&auto=format&n=QcR4sHy3480YmZ2d&q=85&s=d68657d269fbc6a2459a586c8fb64058" data-og-width="1403" width="1403" data-og-height="631" height="631" data-path="images/8f34ba77-serverless-get-started-endpoint-details.png" data-optimize="true" data-opv="3" srcset="https://mintcdn.com/runpod-b18f5ded/QcR4sHy3480YmZ2d/images/8f34ba77-serverless-get-started-endpoint-details.png?w=280&fit=max&auto=format&n=QcR4sHy3480YmZ2d&q=85&s=89e717cb22c1fd4d0f0dbb6763a72a66 280w, https://mintcdn.com/runpod-b18f5ded/QcR4sHy3480YmZ2d/images/8f34ba77-serverless-get-started-endpoint-details.png?w=560&fit=max&auto=format&n=QcR4sHy3480YmZ2d&q=85&s=beefd2f40feb862a0cd2611854021183 560w, https://mintcdn.com/runpod-b18f5ded/QcR4sHy3480YmZ2d/images/8f34ba77-serverless-get-started-endpoint-details.png?w=840&fit=max&auto=format&n=QcR4sHy3480YmZ2d&q=85&s=56a0651a2ba910720d401ba7e4a91953 840w, https://mintcdn.com/runpod-b18f5ded/QcR4sHy3480YmZ2d/images/8f34ba77-serverless-get-started-endpoint-details.png?w=1100&fit=max&auto=format&n=QcR4sHy3480YmZ2d&q=85&s=27f35512954ca7927195025770921b1e 1100w, https://mintcdn.com/runpod-b18f5ded/QcR4sHy3480YmZ2d/images/8f34ba77-serverless-get-started-endpoint-details.png?w=1650&fit=max&auto=format&n=QcR4sHy3480YmZ2d&q=85&s=367eb72b0969c42a2bde5776423c958b 1650w, https://mintcdn.com/runpod-b18f5ded/QcR4sHy3480YmZ2d/images/8f34ba77-serverless-get-started-endpoint-details.png?w=2500&fit=max&auto=format&n=QcR4sHy3480YmZ2d&q=85&s=ae1da0c76e3ab9c7796d08b6a0693ed6 2500w" />
</Frame>

You'll see a default test request that you can modify as needed, then click **Run** to test your endpoint. On first execution, your workers will need to initialize, which may take a moment.

The initial response will look something like this:

```json  theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "id": "6de99fd1-4474-4565-9243-694ffeb65218-u1",
  "status": "IN_QUEUE"
}
```

You'll see the full response after the job completes. If there are any errors, the console will display error logs to help you troubleshoot.

## Operation overview

Queue-based endpoints support comprehensive job lifecycle management through multiple operations that allow you to submit, monitor, manage, and retrieve results from jobs.

Here's a quick overview of the operations available for queue-based endpoints:

| Operation      | HTTP method | Description                                                                                   |
| -------------- | ----------- | --------------------------------------------------------------------------------------------- |
| `/runsync`     | POST        | Submit a synchronous job and wait for the complete results in a single response.              |
| `/run`         | POST        | Submit an asynchronous job that processes in the background, and returns an immediate job ID. |
| `/status`      | GET         | Check the current status, execution details, and results of a submitted job.                  |
| `/stream`      | GET         | Receive incremental results from a job as they become available.                              |
| `/cancel`      | POST        | Stop a job that is in progress or waiting in the queue.                                       |
| `/retry`       | POST        | Requeue a failed or timed-out job using the same job ID and input parameters.                 |
| `/purge-queue` | POST        | Clear all pending jobs from the queue without affecting jobs already in progress.             |
| `/health`      | GET         | Monitor the operational status of your endpoint, including worker and job statistics.         |

<Tip>
  If you need to create an endpoint that supports custom API paths, use [load balancing endpoints](/serverless/load-balancing/overview).
</Tip>

## Operation reference

Below you'll find detailed explanations and examples for each operation using `cURL` and the Runpod SDK.

<Tip>
  You can also send requests using standard HTTP request APIs and libraries, such as `fetch` (for JavaScript) and `requests` (for Python).
</Tip>

Before running these examples, you'll need to install the Runpod SDK:

```bash  theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Python
python -m pip install runpod

# JavaScript
npm install --save runpod-sdk

# Go
go get github.com/runpod/go-sdk && go mod tidy
```

You should also set your [API key](/get-started/api-keys) and endpoint ID (found on the Overview tab for your endpoint in the Runpod console) as environment variables. Run the following commands in your local terminal, replacing `YOUR_API_KEY` and `YOUR_ENDPOINT_ID` with your actual API key and endpoint ID:

```bash  theme={"theme":{"light":"github-light","dark":"github-dark"}}
export RUNPOD_API_KEY="YOUR_API_KEY"
export ENDPOINT_ID="YOUR_ENDPOINT_ID"
```

### `/runsync`

Synchronous jobs wait for completion and return the complete result in a single response. This approach works best for shorter tasks where you need immediate results, interactive applications, and simpler client code without status polling.

`/runsync` requests have a maximum payload size of 20 MB.

Results are available for 1 minute by default, but you can append `?wait=x` to the request URL to extend this up to 5 minutes, where `x` is the number of milliseconds to store the results, from 1000 (1 second) to 300000 (5 minutes).

For example, `?wait=120000` will keep your results available for 2 minutes:

```sh  theme={"theme":{"light":"github-light","dark":"github-dark"}}
https://api.runpod.ai/v2/$ENDPOINT_ID/runsync?wait=120000
```

<Note>
  `?wait` is only available for `cURL` and standard HTTP request libraries.
</Note>

<Tabs>
  <Tab title="cURL">
    ```sh  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    curl --request POST \
         --url https://api.runpod.ai/v2/$ENDPOINT_ID/runsync \
         -H "accept: application/json" \
         -H "authorization: $RUNPOD_API_KEY" \
         -H "content-type: application/json" \
         -d '{ "input": {  "prompt": "Hello, world!" }}'
    ```
  </Tab>

  <Tab title="Python">
    ```python  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    import runpod
    import os

    runpod.api_key = os.getenv("RUNPOD_API_KEY")
    endpoint = runpod.Endpoint(os.getenv("ENDPOINT_ID"))

    try:
        run_request = endpoint.run_sync(
            {"prompt": "Hello, world!"},
            timeout=60,  # Client timeout in seconds
        )
        print(run_request)
    except TimeoutError:
        print("Job timed out.")
    ```
  </Tab>

  <Tab title="JavaScript">
    ```javascript  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    const { RUNPOD_API_KEY, ENDPOINT_ID } = process.env;
    import runpodSdk from "runpod-sdk";

    const runpod = runpodSdk(RUNPOD_API_KEY);
    const endpoint = runpod.endpoint(ENDPOINT_ID);

    const result = await endpoint.runSync({
      "input": {
        "prompt": "Hello, World!",
      },
      timeout: 60000, // Client timeout in milliseconds
    }); 
    });

    console.log(result);
    ```
  </Tab>

  <Tab title="Go">
    ```go  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    package main

    import (
        "encoding/json"
        "fmt"
        "log"
        "os"

        "github.com/runpod/go-sdk/pkg/sdk"
        "github.com.runpod/go-sdk/pkg/sdk/config"
        rpEndpoint "github.com/runpod/go-sdk/pkg/sdk/endpoint"
    )

    func main() {
        apiKey := os.Getenv("RUNPOD_API_KEY")
        baseURL := os.Getenv("RUNPOD_BASE_URL")

        endpoint, err := rpEndpoint.New(
            &config.Config{ApiKey: &apiKey},
            &rpEndpoint.Option{EndpointId: &baseURL},
        )
        if err != nil {
            log.Fatalf("Failed to create endpoint: %v", err)
        }

        jobInput := rpEndpoint.RunSyncInput{
            JobInput: &rpEndpoint.JobInput{
                Input: map[string]interface{}{
                    "prompt": "Hello World",
                },
            },
            Timeout: sdk.Int(60), // Client timeout in seconds
        }

        output, err := endpoint.RunSync(&jobInput)
        if err != nil {
            panic(err)
        }

        data, _ := json.Marshal(output)
        fmt.Printf("output: %s\n", data)
    }
    ```
  </Tab>
</Tabs>

`/runsync` returns a response as soon as the job is complete:

```json  theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "delayTime": 824,
  "executionTime": 3391,
  "id": "sync-79164ff4-d212-44bc-9fe3-389e199a5c15",
  "output": [
    {
      "image": "https://image.url",
      "seed": 46578
    }
  ],
  "status": "COMPLETED"
}
```

### `/run`

Asynchronous jobs process in the background and return immediately with a job ID. This approach works best for longer-running tasks that don't require immediate results, operations requiring significant processing time, and managing multiple concurrent jobs.

`/run` requests have a maximum payload size of 10 MB.

Job results are available for 30 minutes after completion.

<Tabs>
  <Tab title="cURL">
    ```sh  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    curl --request POST \
         --url https://api.runpod.ai/v2/$ENDPOINT_ID/run \
         -H "accept: application/json" \
         -H "authorization: $RUNPOD_API_KEY" \
         -H "content-type: application/json" \
        -d '{"input": {"prompt": "Hello, world!"}}'
    ```
  </Tab>

  <Tab title="Python">
    ```python  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    import runpod
    import os

    runpod.api_key = os.getenv("RUNPOD_API_KEY")
    endpoint = runpod.Endpoint(os.getenv("ENDPOINT_ID"))

    # Submit asynchronous job
    run_request = endpoint.run({"prompt": "Hello, World!"})

    # Check initial status
    status = run_request.status()
    print(f"Initial job status: {status}")

    if status != "COMPLETED":
        # Poll for results with timeout
        output = run_request.output(timeout=60)
    else:
        output = run_request.output()
    print(f"Job output: {output}")
    ```
  </Tab>

  <Tab title="JavaScript">
    ```javascript  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    const { RUNPOD_API_KEY, ENDPOINT_ID } = process.env;
    import runpodSdk from "runpod-sdk";

    const runpod = runpodSdk(RUNPOD_API_KEY);
    const endpoint = runpod.endpoint(ENDPOINT_ID);

    const result = await endpoint.run({
      "input": {
        "prompt": "Hello, World!",
      },
    });

    console.log(result);
    ```
  </Tab>

  <Tab title="Go">
    ```go  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    package main

    import (
        "encoding/json"
        "fmt"
        "log"
        "os"
        
        "github.com/runpod/go-sdk/pkg/sdk"
        "github.com/runpod/go-sdk/pkg/sdk/config"
        rpEndpoint "github.com/runpod/go-sdk/pkg/sdk/endpoint"
    )

    func main() {
        client := sdk.New(&config.Config{
            ApiKey:  os.Getenv("RUNPOD_API_KEY"),
            BaseURL: os.Getenv("RUNPOD_BASE_URL"),
        })
        
        endpoint, err := client.NewEndpoint("YOUR_ENDPOINT_ID")
        if err != nil {
            log.Fatalf("Failed to create endpoint: %v", err)
        }

        jobInput := rpEndpoint.RunInput{
            JobInput: &rpEndpoint.JobInput{
                Input: map[string]interface{}{
                    "prompt": "Hello World",
                },
            },
            RequestTimeout: sdk.Int(120),
        }

        output, err := endpoint.Run(&jobInput)
        if err != nil {
            panic(err)
        }

        data, _ := json.Marshal(output)
        fmt.Printf("output: %s\n", data)
    }
    ```
  </Tab>
</Tabs>

`/run` returns a response with the job ID and status:

```json  theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "id": "eaebd6e7-6a92-4bb8-a911-f996ac5ea99d",
  "status": "IN_QUEUE"
}
```

Further results must be retrieved using the `/status` operation.

### `/status`

Check the current state, execution statistics, and results of previously submitted jobs. The status operation provides the current job state, execution statistics like queue delay and processing time, and job output if completed.

<Tip>
  You can configure time-to-live (TTL) for individual jobs by appending a TTL parameter to the request URL.

  For example, `https://api.runpod.ai/v2/$ENDPOINT_ID/status/YOUR_JOB_ID?ttl=6000` sets the TTL to 6 seconds.
</Tip>

<Tabs>
  <Tab title="cURL">
    Replace `YOUR_JOB_ID` with the actual job ID you received in the response to the `/run` operation.

    ```sh  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    curl --request GET \
         --url https://api.runpod.ai/v2/$ENDPOINT_ID/status/YOUR_JOB_ID \
         -H "authorization: $RUNPOD_API_KEY" \
    ```
  </Tab>

  <Tab title="Python">
    Check the status of a job using the `status` method on the `run_request` object:

    ```python  theme={"theme":{"light":"github-light","dark":"github-dark"}}

    import runpod

    runpod.api_key = os.getenv("RUNPOD_API_KEY")
    endpoint = runpod.Endpoint(os.getenv("ENDPOINT_ID"))

    input_payload = {"input": {"prompt": "Hello, World!"}}

    run_request = endpoint.run(input_payload)

    # Initial check without blocking, useful for quick tasks
    status = run_request.status()
    print(f"Initial job status: {status}")

    if status != "COMPLETED":
        # Polling with timeout for long-running tasks
        output = run_request.output(timeout=60)
    else:
        output = run_request.output()
    print(f"Job output: {output}")
    print(f"An error occurred: {e}")

    ```
  </Tab>

  <Tab title="JavaScript">
    Check the status of a job using the ID returned by `endpoint.run`:

    ```javascript  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    const { RUNPOD_API_KEY, ENDPOINT_ID } = process.env;
    import runpodSdk from "runpod-sdk";

    async function main() {
      try {
        const runpod = runpodSdk(RUNPOD_API_KEY);
        const endpoint = runpod.endpoint(ENDPOINT_ID);
        const result = await endpoint.run({
          input: {
            prompt: "Hello, World!",
          },
        });

        const { id } = result;
        if (!id) {
          console.error("No ID returned from endpoint.run");
          return;
        }

        const status = await endpoint.status(id);
        console.log(status);
      } catch (error) {
        console.error("An error occurred:", error);
      }
    }

    main();
    ```
  </Tab>

  <Tab title="Go">
    Replace `YOUR_JOB_ID` with the actual job ID you received in the response to the `/run` request.

    ```go  theme={"theme":{"light":"github-light","dark":"github-dark"}}

    package main

    import (
        "encoding/json"
        "fmt"
        "log"
        "os"

        "github.com/runpod/go-sdk/pkg/sdk"
        "github.com/runpod/go-sdk/pkg/sdk/config"
        rpEndpoint "github.com/runpod/go-sdk/pkg/sdk/endpoint"
    )

    func main() {

        apiKey := os.Getenv("RUNPOD_API_KEY")
        baseURL := os.Getenv("RUNPOD_BASE_URL")

        endpoint, err := rpEndpoint.New(
            &config.Config{ApiKey: &apiKey},
            &rpEndpoint.Option{EndpointId: &baseURL},
        )
        if err != nil {
            log.Fatalf("Failed to create endpoint: %v", err)
        }
        input := rpEndpoint.StatusInput{
            Id: sdk.String("YOUR_JOB_ID"),
        }
        output, err := endpoint.Status(&input)
        if err != nil {
            panic(err)
        }
        dt, _ := json.Marshal(output)
        fmt.Printf("output:%s\n", dt)
    }
    ```
  </Tab>
</Tabs>

`/status` returns a JSON response with the job status (e.g. `IN_QUEUE`, `IN_PROGRESS`, `COMPLETED`, `FAILED`), and an optional `output` field if the job is completed:

```json  theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "delayTime": 31618,
  "executionTime": 1437,
  "id": "60902e6c-08a1-426e-9cb9-9eaec90f5e2b-u1",
  "output": {
    "input_tokens": 22,
    "output_tokens": 16,
    "text": ["Hello! How can I assist you today?\nUSER: I'm having"]
  },
  "status": "COMPLETED"
}
```

### `/stream`

Receive incremental results as they become available from jobs that generate output progressively. This works especially well for text generation tasks where you want to display output as it's created, long-running jobs where you want to show progress, and large outputs that benefit from incremental processing.

To enable streaming, your handler must support the `"return_aggregate_stream": True` option on the `start` method of your handler. Once enabled, use the `stream` method to receive data as it becomes available.

For implementation details, see [Streaming handlers](/serverless/workers/handler-functions#streaming-handlers).

<Tabs>
  <Tab title="cURL">
    Replace `YOUR_JOB_ID` with the actual job ID you received in the response to the `/run` request.

    ```sh  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    curl --request GET \
         --url https://api.runpod.ai/v2/$ENDPOINT_ID/stream/YOUR_JOB_ID \
         -H "accept: application/json" \
         -H "authorization: $RUNPOD_API_KEY" \
    ```
  </Tab>

  <Tab title="Python">
    ```python  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    import runpod

    runpod.api_key = os.getenv("RUNPOD_API_KEY")
    endpoint = runpod.Endpoint(os.getenv("ENDPOINT_ID"))

    run_request = endpoint.run(
        {
            "input": {
                "prompt": "Hello, world!",
            }
        }
    )

    for output in run_request.stream():
        print(output)
    ```
  </Tab>

  <Tab title="JavaScript">
    ```javascript  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    const { RUNPOD_API_KEY, ENDPOINT_ID } = process.env;
    import runpodSdk from "runpod-sdk";

    async function main() {
      const runpod = runpodSdk(RUNPOD_API_KEY);
      const endpoint = runpod.endpoint(ENDPOINT_ID);
      const result = await endpoint.run({
        input: {
          prompt: "Hello, World!",
        },
      });

      console.log(result);

      const { id } = result;
      for await (const result of endpoint.stream(id)) {
        console.log(`${JSON.stringify(result, null, 2)}`);
      }
      console.log("done streaming");
    }

    main();
    ```
  </Tab>

  <Tab title="Go">
    ```go  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    package main

    import (
        "encoding/json"
        "fmt"

        "github.com/runpod/go-sdk/pkg/sdk/config"
        rpEndpoint "github.com/runpod/go-sdk/pkg/sdk/endpoint"
    )

    func main() {

        apiKey := os.Getenv("RUNPOD_API_KEY")
        baseURL := os.Getenv("RUNPOD_BASE_URL")

        endpoint, err := rpEndpoint.New(
            &config.Config{ApiKey: &apiKey},
            &rpEndpoint.Option{EndpointId: &baseURL},
        )
        if err != nil {
            panic(err)
        }

        request, err := endpoint.Run(&rpEndpoint.RunInput{
            JobInput: &rpEndpoint.JobInput{
                Input: map[string]interface{}{
                    "prompt": "Hello World",
                },
            },
        })
        if err != nil {
            panic(err)
        }

        streamChan := make(chan rpEndpoint.StreamResult, 100)

        err = endpoint.Stream(&rpEndpoint.StreamInput{Id: request.Id}, streamChan)
        if err != nil {
            // timeout reached, if we want to get the data that has been streamed
            if err.Error() == "ctx timeout reached" {
                for data := range streamChan {
                    dt, _ := json.Marshal(data)
                    fmt.Printf("output:%s\n", dt)
                }
            }
            panic(err)
        }

        for data := range streamChan {
            dt, _ := json.Marshal(data)
            fmt.Printf("output:%s\n", dt)
        }

    }
    ```
  </Tab>
</Tabs>

<Info>
  The maximum size for a single streamed payload chunk is 1 MB. Larger outputs will be split across multiple chunks.
</Info>

Streaming response format:

```json  theme={"theme":{"light":"github-light","dark":"github-dark"}}
[
  {
    "metrics": {
      "avg_gen_throughput": 0,
      "avg_prompt_throughput": 0,
      "cpu_kv_cache_usage": 0,
      "gpu_kv_cache_usage": 0.0016722408026755853,
      "input_tokens": 0,
      "output_tokens": 1,
      "pending": 0,
      "running": 1,
      "scenario": "stream",
      "stream_index": 2,
      "swapped": 0
    },
    "output": {
      "input_tokens": 0,
      "output_tokens": 1,
      "text": [" How"]
    }
  }
]
```

### `/cancel`

Stop jobs that are no longer needed or taking too long to complete. This operation stops in-progress jobs, removes queued jobs before they start, and returns immediately with the canceled status.

<Tabs>
  <Tab title="cURL">
    Replace `YOUR_JOB_ID` with the actual job ID you received in the response to the `/run` request.

    ```sh  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    curl --request POST \
      --url https://api.runpod.ai/v2/$ENDPOINT_ID/cancel/YOUR_JOB_ID \
      -H "authorization: $RUNPOD_API_KEY" \
    ```
  </Tab>

  <Tab title="Python">
    Cancel a job using the `cancel` method on the `run_request` object. The script below demonstrates how to cancel a job using a keyboard interrupt:

    ```python  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    import time
    import runpod

    runpod.api_key = os.getenv("RUNPOD_API_KEY")
    endpoint = runpod.Endpoint(os.getenv("ENDPOINT_ID"))

    run_request = endpoint.run(
      {
          "input": {
              "prompt": "Hello, world!",
          }
      }
    )

    try:
      while True:
        status = run_request.status()
        print(f"Current job status: {status}")

        if status == "COMPLETED":
          output = run_request.output()
          print("Job output:", output)
          break
        elif status in ["FAILED", "ERROR"]:
          print("Job failed to complete successfully.")
          break
        else:
          time.sleep(10)
    except KeyboardInterrupt:  # Catch KeyboardInterrupt
      print("KeyboardInterrupt detected. Canceling the job...")
      if run_request:  # Check if a job is active
        run_request.cancel()
      print("Job canceled.")
    ```
  </Tab>

  <Tab title="JavaScript">
    Cancel a job by using the `cancel()` function on the run request.

    ```javascript  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    const { RUNPOD_API_KEY, ENDPOINT_ID } = process.env;
    import runpodSdk from "runpod-sdk";

    async function main() {
      try {
        const runpod = runpodSdk(RUNPOD_API_KEY);
        const endpoint = runpod.endpoint(ENDPOINT_ID);
        const result = await endpoint.run({
          input: {
            prompt: "Hello, World!",
          },
        });

        const { id } = result;
        if (!id) {
          console.error("No ID returned from endpoint.run");
          return;
        }

        const cancel = await endpoint.cancel(id);
        console.log(cancel);
      } catch (error) {
        console.error("An error occurred:", error);
      }
    }

    main();
    ```
  </Tab>

  <Tab title="Go">
    ```go  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    package main

    import (
        "encoding/json"
        "fmt"

        "github.com/runpod/go-sdk/pkg/sdk"
        "github.com/runpod/go-sdk/pkg/sdk/config"
        rpEndpoint "github.com/runpod/go-sdk/pkg/sdk/endpoint"
    )

    func main() {

        apiKey := os.Getenv("RUNPOD_API_KEY")
        baseURL := os.Getenv("RUNPOD_BASE_URL")

        endpoint, err := rpEndpoint.New(
            &config.Config{ApiKey: &apiKey},
            &rpEndpoint.Option{EndpointId: &baseURL},
        )
        if err != nil {
            panic(err)
        }

        cancelInput := rpEndpoint.CancelInput{
            Id: sdk.String("00edfd03-8094-46da-82e3-ea47dd9566dc-u1"),
        }
        output, err := endpoint.Cancel(&cancelInput)
        if err != nil {
            panic(err)
        }

        healthData, _ := json.Marshal(output)
        fmt.Printf("health output: %s\n", healthData)

    }
    ```
  </Tab>
</Tabs>

`/cancel` requests return a JSON response with the status of the cancel operation:

```json  theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "id": "724907fe-7bcc-4e42-998d-52cb93e1421f-u1",
  "status": "CANCELLED"
}
```

### `/retry`

Requeue jobs that have failed or timed out without submitting a new request. This operation maintains the same job ID for tracking, requeues with original input parameters, and removes previous output. It can only be used for jobs with `FAILED` or `TIMED_OUT` status.

Replace `YOUR_JOB_ID` with the actual job ID you received in the response to the `/run` request.

```sh  theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl --request POST \
     --url https://api.runpod.ai/v2/$ENDPOINT_ID/retry/YOUR_JOB_ID \
     -H "authorization: $RUNPOD_API_KEY"
```

You'll see the job status updated to `IN_QUEUE` when the job is retried:

```json  theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "id": "60902e6c-08a1-426e-9cb9-9eaec90f5e2b-u1",
  "status": "IN_QUEUE"
}
```

<Note>
  Job results expire after a set period. Asynchronous jobs (`/run`) results are available for 30 minutes, while synchronous jobs (`/runsync`) results are available for 1 minute (up to 5 minutes with `?wait=t`). Once expired, jobs cannot be retried.
</Note>

### `/purge-queue`

Remove all pending jobs from the queue when you need to reset or handle multiple cancellations at once. This is useful for error recovery, clearing outdated requests, resetting after configuration changes, and managing resource allocation.

<Tabs>
  <Tab title="cURL">
    ```sh  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    curl --request POST \
         --url https://api.runpod.ai/v2/$ENDPOINT_ID/purge-queue \
         -H "authorization: $RUNPOD_API_KEY"
        -H 'Authorization: Bearer RUNPOD_API_KEY'
    ```
  </Tab>

  <Tab title="Python">
    ```python  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    import runpod
    import os

    runpod.api_key = os.getenv("RUNPOD_API_KEY")
    endpoint = runpod.Endpoint(os.getenv("ENDPOINT_ID"))

    endpoint.purge_queue(timeout=3)
    ```
  </Tab>

  <Tab title="JavaScript">
    ```javascript  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    const { RUNPOD_API_KEY, ENDPOINT_ID } = process.env;
    import runpodSdk from "runpod-sdk";

    async function main() {
      try {
        const runpod = runpodSdk(RUNPOD_API_KEY);
        const endpoint = runpod.endpoint(ENDPOINT_ID);
        await endpoint.run({
          input: {
            prompt: "Hello, World!",
          },
        });

        const purgeQueue = await endpoint.purgeQueue();
        console.log(purgeQueue);
      } catch (error) {
        console.error("An error occurred:", error);
      }
    }

    main();
    ```
  </Tab>
</Tabs>

<Warning>
  `/purge-queue` operation only affects jobs waiting in the queue. Jobs already in progress will continue to run.
</Warning>

`/purge-queue` requests return a JSON response with the number of jobs removed from the queue and the status of the purge operation:

```json  theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "removed": 2,
  "status": "completed"
}
```

### `/health`

Get a quick overview of your endpoint's operational status including worker availability, job queue status, potential bottlenecks, and scaling requirements.

<Tabs>
  <Tab title="cURL">
    ```sh  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    curl --request GET \
         --url https://api.runpod.ai/v2/$ENDPOINT_ID/health \
         -H "authorization: $RUNPOD_API_KEY"
    ```
  </Tab>

  <Tab title="Python">
    ```python  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    import runpod
    import json
    import os

    runpod.api_key = os.getenv("RUNPOD_API_KEY")
    endpoint = runpod.Endpoint(os.getenv("ENDPOINT_ID"))

    endpoint_health = endpoint.health()
    print(json.dumps(endpoint_health, indent=2))
    ```
  </Tab>

  <Tab title="JavaScript">
    ```javascript  theme={"theme":{"light":"github-light","dark":"github-dark"}}
    const { RUNPOD_API_KEY, ENDPOINT_ID } = process.env;
    import runpodSdk from "runpod-sdk";

    const runpod = runpodSdk(RUNPOD_API_KEY);
    const endpoint = runpod.endpoint(ENDPOINT_ID);

    const health = await endpoint.health();
    console.log(health);
    ```
  </Tab>
</Tabs>

`/health` requests return a JSON response with the current status of the endpoint, including the number of jobs completed, failed, in progress, in queue, and retried, as well as the status of workers.

```json  theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "jobs": {
    "completed": 1,
    "failed": 5,
    "inProgress": 0,
    "inQueue": 2,
    "retried": 0
  },
  "workers": {
    "idle": 0,
    "running": 0
  }
}
```

## Advanced options

Beyond the required `input` object, you can include optional top-level parameters to enable additional functionality for your queue-based endpoints.

### Webhook notifications

Receive notifications when jobs complete by specifying a webhook URL. When your job completes, Runpod will send a `POST` request to your webhook URL containing the same information as the `/status/JOB_ID` endpoint.

```json  theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "input": {
    "prompt": "Your input here"
  },
  "webhook": "https://your-webhook-url.com"
}
```

Your webhook should return a `200` status code to acknowledge receipt. If the call fails, Runpod will retry up to 2 more times with a 10-second delay between attempts.

### Execution policies

Control job execution behavior with custom policies. By default, jobs automatically terminate after 10 minutes without completion to prevent runaway costs.

```json  theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "input": {
    "prompt": "Your input here"
  },
  "policy": {
    "executionTimeout": 900000,
    "lowPriority": false,
    "ttl": 3600000
  }
}
```

Policy options:

| Option             | Description                                 | Default             | Constraints                    |
| ------------------ | ------------------------------------------- | ------------------- | ------------------------------ |
| `executionTimeout` | Maximum job runtime in milliseconds         | 600000 (10 minutes) | Must be > 5000 ms              |
| `lowPriority`      | When true, job won't trigger worker scaling | false               | -                              |
| `ttl`              | Maximum job lifetime in milliseconds        | 86400000 (24 hours) | Must be ≥ 10000 ms, max 1 week |

<Info>
  Setting `executionTimeout` in a request overrides the default endpoint setting for that specific job only.
</Info>

### S3-compatible storage integration

Configure S3-compatible storage for endpoints working with large files. This configuration is passed directly to your worker but not included in responses.

```json  theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "input": {
    "prompt": "Your input here"
  },
  "s3Config": {
    "accessId": "BUCKET_ACCESS_KEY_ID",
    "accessSecret": "BUCKET_SECRET_ACCESS_KEY",
    "bucketName": "BUCKET_NAME",
    "endpointUrl": "BUCKET_ENDPOINT_URL"
  }
}
```

Your worker must contain logic to use this information for storage operations.

<Tip>
  S3 integration works with any S3-compatible provider including MinIO, Backblaze B2, DigitalOcean Spaces, and others.
</Tip>

## Rate limits and quotas

Runpod enforces rate limits to ensure fair platform usage. These limits apply per endpoint and operation:

| Operation      | Method | Rate Limit                   | Concurrent Limit |
| -------------- | ------ | ---------------------------- | ---------------- |
| `/runsync`     | POST   | 2000 requests per 10 seconds | 400 concurrent   |
| `/run`         | POST   | 1000 requests per 10 seconds | 200 concurrent   |
| `/status`      | GET    | 2000 requests per 10 seconds | 400 concurrent   |
| `/stream`      | GET    | 2000 requests per 10 seconds | 400 concurrent   |
| `/cancel`      | POST   | 100 requests per 10 seconds  | 20 concurrent    |
| `/purge-queue` | POST   | 2 requests per 10 seconds    | N/A              |
| `/openai/*`    | POST   | 2000 requests per 10 seconds | 400 concurrent   |
| `/requests`    | GET    | 10 requests per 10 seconds   | 2 concurrent     |

### Dynamic rate limiting

In addition to the base rate limits above, Runpod implements a dynamic rate limiting system that scales with your endpoint's worker count. This helps ensure platform stability while allowing higher throughput as you scale.

Rate limits are calculated using two values:

1. **Base limit**: A fixed rate limit per user per endpoint (shown in the table above)
2. **Worker-based limit**: A dynamic limit calculated as `number_of_running_workers × requests_per_worker`

The system uses **whichever limit is higher** between the base limit and worker-based limit. Requests are blocked with a `429 (Too Many Requests)` status when the request count exceeds this effective limit within a 10-second window. This means as your endpoint scales up workers, your effective rate limit increases proportionally.

For example, if an endpoint has:

* Base limit: 2000 requests per 10 seconds
* Additional limit per worker: 50 requests per 10 seconds
* 20 running workers

The effective rate limit would be `max(2000, 20 × 50) = 2000` requests per 10 seconds (base limit applies). With 50 running workers, it would scale to `max(2000, 50 × 50) = 2500` requests per 10 seconds (worker-based limit applies).

**Key points:**

* Rate limiting is based on request count per 10-second time windows
* The system automatically uses whichever limit gives you more requests

Implement appropriate retry logic with exponential backoff to handle rate limiting gracefully.

## Best practices

Follow these practices to optimize your queue-based endpoint usage:

* Use asynchronous requests for jobs that take more than a few seconds to complete.
* Implement polling with backoff when checking status of asynchronous jobs.
* Set appropriate timeouts in your client applications and monitor endpoint health regularly to detect issues early.
* Implement comprehensive error handling for all API calls.
* Use webhooks for notification-based workflows instead of polling to reduce API calls.
* Cancel unneeded jobs to free up resources and reduce costs.
* During development, use the console testing interface before implementing programmatic integration.

## Error handling and troubleshooting

When sending requests, be prepared to handle these common errors:

| HTTP Status | Meaning               | Solution                                          |
| ----------- | --------------------- | ------------------------------------------------- |
| 400         | Bad Request           | Check your request format and parameters          |
| 401         | Unauthorized          | Verify your API key is correct and has permission |
| 404         | Not Found             | Check your endpoint ID                            |
| 429         | Too Many Requests     | Implement backoff and retry logic                 |
| 500         | Internal Server Error | Check endpoint logs; worker may have crashed      |

Here are some common issues and suggested solutions:

| Issue              | Possible Causes                                 | Solutions                                                                                        |
| ------------------ | ----------------------------------------------- | ------------------------------------------------------------------------------------------------ |
| Job stuck in queue | No available workers, max workers limit reached | Increase max workers, check endpoint health                                                      |
| Timeout errors     | Job takes longer than execution timeout         | Increase timeout in job policy, optimize job processing                                          |
| Failed jobs        | Worker errors, input validation issues          | Check [endpoint logs](/serverless/development/logs), verify input format, retry with fixed input |
| Rate limiting      | Too many requests in short time                 | Implement backoff strategy, batch requests when possible                                         |
| Missing results    | Results expired                                 | Retrieve results within expiration window (30 min for async, 1 min for sync)                     |

Implementing proper [error handling](/serverless/endpoints/error-handling) and retry logic will make your integrations more robust and reliable.