# LM Studio Developer Docs

> Build with LM Studio's local APIs and SDKs — TypeScript, Python, REST, and OpenAI‑compatible endpoints.


```lms_hstack
## Get to know the stack

- TypeScript SDK: [lmstudio-js](/docs/typescript)
- Python SDK: [lmstudio-python](/docs/python)
- OpenAI‑compatible: [Chat, Responses, Embeddings](/docs/developer/openai-compat)
- LM Studio CLI: [`lms`](/docs/cli)

:::split:::

## What you can build

- Chat and text generation with streaming
- Structured output (JSON schema)
- Tool calling and local agents
- Embeddings and tokenization
- Model management (JIT load, TTL, auto‑evict)
```

## Super quick start

### TypeScript (`lmstudio-js`)

```bash
npm install @lmstudio/sdk
```

```ts
import { LMStudioClient } from "@lmstudio/sdk";

const client = new LMStudioClient();
const model = await client.llm.model("openai/gpt-oss-20b");
const result = await model.respond("Who are you, and what can you do?");

console.info(result.content);
```

Full docs: [lmstudio-js](/docs/typescript), Source: [GitHub](https://github.com/lmstudio-ai/lmstudio-js)

### Python (`lmstudio-python`)

```bash
pip install lmstudio
```

```python
import lmstudio as lms

with lms.Client() as client:
    model = client.llm.model("openai/gpt-oss-20b")
    result = model.respond("Who are you, and what can you do?")
    print(result)
```

Full docs: [lmstudio-python](/docs/python), Source: [GitHub](https://github.com/lmstudio-ai/lmstudio-python)

### Try a minimal HTTP request (OpenAI‑compatible)

```bash
lms server start --port 1234
```

```bash
curl http://localhost:1234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-oss-20b",
    "messages": [{"role": "user", "content": "Who are you, and what can you do?"}]
  }'
```

Full docs: [OpenAI‑compatible endpoints](/docs/developer/openai-compat)

## Helpful links

- API Changelog: [/docs/developer/api-changelog](/docs/developer/api-changelog)
- Local server basics: [/docs/developer/core](/docs/developer/core)
- CLI reference: [/docs/cli](/docs/cli)
- Community: [Discord](https://discord.gg/lmstudio)


## API Changelog

> Updates and changes to the LM Studio API.


---

###### LM Studio 0.3.29 • 2025‑10‑06

### OpenAI `/v1/responses` and variant listing

- New OpenAI‑compatible endpoint: `POST /v1/responses`.
  - Stateful interactions via `previous_response_id`.
  - Custom tool calling and Remote MCP support (opt‑in).
  - Reasoning support with `reasoning.effort` for `openai/gpt‑oss‑20b`.
  - Streaming via SSE when `stream: true`.
- CLI: `lms ls --variants` lists all variants for multi‑variant models.
- Docs: [/docs/developer/openai-compat](/docs/developer/openai-compat). Full release notes: [/blog/lmstudio-v0.3.29](/blog/lmstudio-v0.3.29).

---

###### LM Studio 0.3.27 • 2025‑09‑24

### CLI: model resource estimates, status, and interrupts

- New: `lms load --estimate-only <model>` prints estimated GPU and total memory before loading. Honors `--context-length` and `--gpu`, and uses an improved estimator that now accounts for flash attention and vision models.
- `lms chat`: press `Ctrl+C` to interrupt an ongoing prediction.
- `lms ps --json` now reports each model's generation status and the number of queued prediction requests.
- CLI color contrast improved for light mode.
- See docs: [/docs/cli/local-models/load](/docs/cli/local-models/load). Full release notes: [/blog/lmstudio-v0.3.27](/blog/lmstudio-v0.3.27).

---

###### LM Studio 0.3.26 • 2025‑09‑15

### CLI log streaming: server + model

- `lms log stream` now supports multiple sources and filters.
  - `--source server` streams HTTP server logs (startup, endpoints, status)
  - `--source model --filter input,output` streams formatted user input and model output
  - Append `--json` for machine‑readable logs; `--stats` adds tokens/sec and related metrics (model source)
- See usage and examples: [/docs/cli/serve/log-stream](/docs/cli/serve/log-stream). Full release notes: [/blog/lmstudio-v0.3.26](/blog/lmstudio-v0.3.26).

---

###### LM Studio 0.3.25 • 2025‑09‑04

### New model support (API)

- Added support for NVIDIA Nemotron‑Nano‑v2 with tool‑calling via the OpenAI‑compatible endpoints [‡](/blog/lmstudio-v0.3.25).
- Added support for Google EmbeddingGemma for the `/v1/embeddings` endpoint [‡](/blog/lmstudio-v0.3.25).

---

###### LM Studio 0.3.24 • 2025‑08‑28

### Seed‑OSS tool‑calling and template fixes

- Added support for ByteDance/Seed‑OSS including tool‑calling and prompt‑template compatibility fixes in the OpenAI‑compatible API [‡](/blog/lmstudio-v0.3.24).
- Fixed cases where tool calls were not parsed for certain prompt templates [‡](/blog/lmstudio-v0.3.24).

---

###### LM Studio 0.3.23 • 2025‑08‑12

### Reasoning content and tool‑calling reliability

- For `gpt‑oss` on `POST /v1/chat/completions`, reasoning content moves out of `message.content` and into `choices.message.reasoning` (non‑streaming) and `choices.delta.reasoning` (streaming), aligning with `o3‑mini` [‡](/blog/lmstudio-v0.3.23).
- Tool names are normalized (e.g., snake_case) before being provided to the model to improve tool‑calling reliability [‡](/blog/lmstudio-v0.3.23).
- Fixed errors for certain tools‑containing requests to `POST /v1/chat/completions` (e.g., "reading 'properties'") and non‑streaming tool‑call failures [‡](/blog/lmstudio-v0.3.23).

---

###### LM Studio 0.3.19 • 2025‑07‑21

### Bug fixes for streaming and tool calls

- Corrected usage statistics returned by OpenAI‑compatible streaming responses [‡](https://lmstudio.ai/blog/lmstudio-v0.3.19#:~:text=,OpenAI%20streaming%20responses%20were%20incorrect).
- Improved handling of parallel tool calls via the streaming API [‡](https://lmstudio.ai/blog/lmstudio-v0.3.19#:~:text=,API%20were%20not%20handled%20correctly).
- Fixed parsing of correct tool calls for certain Mistral models [‡](https://lmstudio.ai/blog/lmstudio-v0.3.19#:~:text=,Ryzen%20AI%20PRO%20300%20series).

---

###### LM Studio 0.3.18 • 2025‑07‑10

### Streaming options and tool‑calling improvements

- Added support for the `stream_options` object on OpenAI‑compatible endpoints. Setting `stream_options.include_usage` to `true` returns prompt and completion token usage during streaming [‡](https://lmstudio.ai/blog/lmstudio-v0.3.18#:~:text=%2A%20Added%20support%20for%20%60,to%20support%20more%20prompt%20templates).
- Errors returned from streaming endpoints now follow the correct format expected by OpenAI clients [‡](https://lmstudio.ai/blog/lmstudio-v0.3.18#:~:text=,with%20proper%20chat%20templates).
- Tool‑calling support added for Mistral v13 tokenizer models, using proper chat templates [‡](https://lmstudio.ai/blog/lmstudio-v0.3.18#:~:text=,with%20proper%20chat%20templates).
- The `response_format.type` field now accepts `"text"` in chat‑completion requests [‡](https://lmstudio.ai/blog/lmstudio-v0.3.18#:~:text=,that%20are%20split%20across%20multiple).
- Fixed bugs where parallel tool calls split across multiple chunks were dropped and where root‑level `$defs` in tool definitions were stripped [‡](https://lmstudio.ai/blog/lmstudio-v0.3.18#:~:text=,being%20stripped%20in%20tool%20definitions).

---

###### LM Studio 0.3.17 • 2025‑06‑25

### Tool‑calling reliability and token‑count updates

- Token counts now include the system prompt and tool definitions [‡](https://lmstudio.ai/blog/lmstudio-v0.3.17#:~:text=,have%20a%20URL%20in%20the). This makes usage reporting more accurate for both the UI and the API.
- Tool‑call argument tokens are streamed as they are generated [‡](https://lmstudio.ai/blog/lmstudio-v0.3.17#:~:text=Build%206), improving responsiveness when using streamed function calls.
- Various fixes improve MCP and tool‑calling reliability, including correct handling of tools that omit a `parameters` object and preventing hangs when an MCP server reloads [‡](https://lmstudio.ai/blog/lmstudio-v0.3.17#:~:text=,tool%20calls%20would%20hang%20indefinitely).

---

###### LM Studio 0.3.16 • 2025‑05‑23

### Model capabilities in `GET /models`

- The OpenAI‑compatible REST API (`/api/v0`) now returns a `capabilities` array in the `GET /models` response. Each model lists its supported capabilities (e.g. `"tool_use"`) [‡](https://lmstudio.ai/blog/lmstudio-v0.3.16#:~:text=,response) so clients can programmatically discover tool‑enabled models.
- Fixed a streaming bug where an empty function name string was appended after the first packet of streamed tool calls [‡](https://lmstudio.ai/blog/lmstudio-v0.3.16#:~:text=%2A%20Bugfix%3A%20%5BOpenAI,packet%20of%20streamed%20function%20calls).

---

###### [👾 LM Studio 0.3.15](/blog/lmstudio-v0.3.15) • 2025-04-24

### Improved Tool Use API Support

OpenAI-like REST API now supports the `tool_choice` parameter:

```json
{
  "tool_choice": "auto" // or "none", "required"
}
```

- `"tool_choice": "none"` — Model will not call tools
- `"tool_choice": "auto"` — Model decides
- `"tool_choice": "required"` — Model must call tools (llama.cpp only)

Chunked responses now set `"finish_reason": "tool_calls"` when appropriate.

---

###### [👾 LM Studio 0.3.14](/blog/lmstudio-v0.3.14) • 2025-03-27

### [API/SDK] Preset Support

RESTful API and SDKs support specifying presets in requests.

_(example needed)_

###### [👾 LM Studio 0.3.10](/blog/lmstudio-v0.3.10) • 2025-02-18

### Speculative Decoding API

Enable speculative decoding in API requests with `"draft_model"`:

```json
{
  "model": "deepseek-r1-distill-qwen-7b",
  "draft_model": "deepseek-r1-distill-qwen-0.5b",
  "messages": [ ... ]
}
```

Responses now include a `stats` object for speculative decoding:

```json
"stats": {
  "tokens_per_second": ...,
  "draft_model": "...",
  "total_draft_tokens_count": ...,
  "accepted_draft_tokens_count": ...,
  "rejected_draft_tokens_count": ...,
  "ignored_draft_tokens_count": ...
}
```

---

###### [👾 LM Studio 0.3.9](blog/lmstudio-v0.3.9) • 2025-01-30

### Idle TTL and Auto Evict

Set a TTL (in seconds) for models loaded via API requests (docs article: [Idle TTL and Auto-Evict](/docs/developer/core/ttl-and-auto-evict))

```diff
curl http://localhost:1234/api/v0/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-r1-distill-qwen-7b",
    "messages": [ ... ]
+   "ttl": 300,
}'
```

With `lms`:

```
lms load --ttl <seconds>
```

### Separate `reasoning_content` in Chat Completion responses

For DeepSeek R1 models, get reasoning content in a separate field. See more [here](/blog/lmstudio-v0.3.9#separate-reasoningcontent-in-chat-completion-responses).

Turn this on in App Settings > Developer.

---

###### [👾 LM Studio 0.3.6](blog/lmstudio-v0.3.6) • 2025-01-06

### Tool and Function Calling API

Use any LLM that supports Tool Use and Function Calling through the OpenAI-like API.

Docs: [Tool Use and Function Calling](/docs/developer/core/tools).

---

###### [👾 LM Studio 0.3.5](blog/lmstudio-v0.3.5) • 2024-10-22

### Introducing `lms get`: download models from the terminal

You can now download models directly from the terminal using a keyword

```bash
lms get deepseek-r1
```

or a full Hugging Face URL

```bash
lms get <hugging face url>
```

To filter for MLX models only, add `--mlx` to the command.

```bash
lms get deepseek-r1 --mlx
```


## core

### Idle TTL and Auto-Evict

> Optionally auto-unload idle models after a certain amount of time (TTL)


## Background

- `JIT loading` makes it easy to use your LM Studio models in other apps: you don't need to manually load the model first before being able to use it. However, this also means that models can stay loaded in memory even when they're not being used. `[Default: enabled]`

- (New) `Idle TTL` (technically: Time-To-Live) defines how long a model can stay loaded in memory without receiving any requests. When the TTL expires, the model is automatically unloaded from memory. You can set a TTL using the `ttl` field in your request payload. `[Default: 60 minutes]`

- (New) `Auto-Evict` is a feature that unloads previously JIT loaded models before loading new ones. This enables easy switching between models from client apps without having to manually unload them first. You can enable or disable this feature in Developer tab > Server Settings. `[Default: enabled]`

## Idle TTL

**Use case**: imagine you're using an app like [Zed](https://github.com/zed-industries/zed/blob/main/crates/lmstudio/src/lmstudio.rs#L340), [Cline](https://github.com/cline/cline/blob/main/src/api/providers/lmstudio.ts), or [Continue.dev](https://docs.continue.dev/customize/model-providers/more/lmstudio) to interact with LLMs served by LM Studio. These apps leverage JIT to load models on-demand the first time you use them.

**Problem**: When you're not actively using a model, you might don't want it to remain loaded in memory.

**Solution**: Set a TTL for models loaded via API requests. The idle timer resets every time the model receives a request, so it won't disappear while you use it. A model is considered idle if it's not doing any work. When the idle TTL expires, the model is automatically unloaded from memory.

### Set App-default Idle TTL

By default, JIT-loaded models have a TTL of 60 minutes. You can configure a default TTL value for any model loaded via JIT like so:

<img src="/assets/docs/app-default-ttl.png" style="width: 500px; " data-caption="Set a default TTL value. Will be used for all JIT loaded models unless specified otherwise in the request payload" />

### Set per-model TTL-model in API requests

When JIT loading is enabled, the **first request** to a model will load it into memory. You can specify a TTL for that model in the request payload.

This works for requests targeting both the [OpenAI compatibility API](/docs/developer/openai-api) and the [LM Studio's REST API](/docs/developer/rest):

```diff
curl http://localhost:1234/api/v0/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-r1-distill-qwen-7b",
+   "ttl": 300,
    "messages": [ ... ]
}'
```

###### This will set a TTL of 5 minutes (300 seconds) for this model if it is JIT loaded.

### Set TTL for models loaded with `lms`

By default, models loaded with `lms load` do not have a TTL, and will remain loaded in memory until you manually unload them.

You can set a TTL for a model loaded with `lms` like so:

```bash
lms load <model> --ttl 3600
```

###### Load a `<model>` with a TTL of 1 hour (3600 seconds)

### Specify TTL when loading models in the server tab

You can also set a TTL when loading a model in the server tab like so

<img src="/assets/docs/ttl-server-model.png" style="width: 100%;" data-caption="Set a TTL value when loading a model in the server tab" />

## Configure Auto-Evict for JIT loaded models

With this setting, you can ensure new models loaded via JIT automatically unload previously loaded models first.

This is useful when you want to switch between models from another app without worrying about memory building up with unused models.

<img src="/assets/docs/auto-evict-and-ttl.png" style="width: 500px; margin-top:30px" data-caption="Enable or disable Auto-Evict for JIT loaded models in the Developer tab > Server Settings" />

**When Auto-Evict is ON** (default):

- At most `1` model is kept loaded in memory at a time (when loaded via JIT)
- Non-JIT loaded models are not affected

**When Auto-Evict is OFF**:

- Switching models from an external app will keep previous models loaded in memory
- Models will remain loaded until either:
  - Their TTL expires
  - You manually unload them

This feature works in tandem with TTL to provide better memory management for your workflow.

### Nomenclature

`TTL`: Time-To-Live, is a term borrowed from networking protocols and cache systems. It defines how long a resource can remain allocated before it's considered stale and evicted.


### Run LM Studio as a service (headless)

> GUI-less operation of LM Studio: run in the background, start on machine login, and load models on demand


LM Studio can be run as a service without the GUI. This is useful for running LM Studio on a server or in the background on your local machine. This works on Mac, Windows, and Linux machines with a graphical user interface.

## Run LM Studio as a service

Running LM Studio as a service consists of several new features intended to make it more efficient to use LM Studio as a developer tool.

1. The ability to run LM Studio without the GUI
2. The ability to start the LM Studio LLM server on machine login, headlessly
3. On-demand model loading

## Run the LLM service on machine login

To enable this, head to app settings (`Cmd` / `Ctrl` + `,`) and check the box to run the LLM server on login.

<img src="/assets/docs/headless-settings.png" style="" data-caption="Enable the LLM server to start on machine login" />

When this setting is enabled, exiting the app will minimize it to the system tray, and the LLM server will continue to run in the background.

## Just-In-Time (JIT) model loading for OpenAI endpoints

Useful when utilizing LM Studio as an LLM service with other frontends or applications.

<img src="/assets/docs/jit-loading.png" style="" data-caption="Load models on demand" />

#### When JIT loading is ON:

- Call to `/v1/models` will return all downloaded models, not only the ones loaded into memory
- Calls to inference endpoints will load the model into memory if it's not already loaded

#### When JIT loading is OFF:

- Call to `/v1/models` will return only the models loaded into memory
- You have to first load the model into memory before being able to use it

##### What about auto unloading?

As of LM Studio 0.3.5, auto unloading is not yet in place. Models that are loaded via JIT loading will remain in memory until you unload them.
We expect to implement more sophisticated memory management in the near future. Let us know if you have any feedback or suggestions.

## Auto Server Start

Your last server state will be saved and restored on app or service launch.

To achieve this programmatically, you can use the following command:

```bash
lms server start
```

### Community

Chat with other LM Studio developers, discuss LLMs, hardware, and more on the [LM Studio Discord server](https://discord.gg/aPQfnNkxGC).

Please report bugs and issues in the [lmstudio-bug-tracker](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues) GitHub repository.


### server

### LM Studio as a Local LLM API Server

> Run an LLM API server on `localhost` with LM Studio


You can serve local LLMs from LM Studio's Developer tab, either on `localhost` or on the network.

LM Studio's APIs can be used through [REST API](/docs/developer/rest), client libraries like [lmstudio-js](/docs/typescript) and [lmstudio-python](/docs/python), and [OpenAI compatibility endpoints](/docs/developer/openai-compat)

<img src="/assets/docs/server.png" style="" data-caption="Load and serve LLMs from LM Studio" />

### Running the server

To run the server, go to the Developer tab in LM Studio, and toggle the "Start server" switch to start the API server.

<img src="/assets/docs/server-start.png" style="" data-caption="Start the LM Studio API Server" />



Alternatively, you can use `lms` ([LM Studio's CLI](/docs/cli)) to start the server from your terminal:

```bash
lms server start
```


### API options

- [LM Studio REST API](/docs/developer/rest)
- [TypeScript SDK](/docs/typescript) - `lmstudio-js`
- [Python SDK](/docs/python) - `lmstudio-python`
- [OpenAI compatibility endpoints](/docs/developer/openai-compat)


#### Server Settings

> Configure server settings for LM Studio API Server


You can configure server settings, such as the port number, whether to allow other API clients to access the server and MCP features.

<img src="/assets/docs/server-config.png" style="" data-caption="Configure LM Studio API Server settings" />


### Settings information

```lms_params
- name: Server Port
  type: Integer
  optional: false
  description: Port number on which the LM Studio API server listens for incoming connections.
  unstyledName: true
- name: Serve on Local Network
  type: Switch
  description: Allow other devices on the same local network to access the API server. Learn more in the [Serve on Local Network](/docs/developer/core/server/serve-on-network) section.
  unstyledName: true
- name: Allow Per Request Remote MCPs
  type: Switch
  description: Enable sending requests to remote MCP (Model Control Protocol) servers on a per-request basis.
  unstyledName: true
- name: Enable CORS
  type: Switch
  description: Enable Cross-Origin Resource Sharing (CORS) to allow applications from different origins to access the API.
  unstyledName: true
- name: Just in Time Model Loading
  type: Switch
  description: Load models dynamically at request time to save memory.
  unstyledName: true
- name: Auto Unload Unused JIT Models
  type: Switch
  description: Automatically unload JIT-loaded models from memory when they are no longer in use.
  unstyledName: true
- name: Only Keep Last JIT Loaded Model
  type: Switch
  description: Keep only the most recently used JIT-loaded model in memory to minimize RAM usag
  unstyledName: true
```


#### Serve on Local Network

> Allow other devices in your network use this LM Studio API server



Enabling the "Serve on Local Network" option allows the LM Studio API server running on your machine to be accessible by other devices connected to the same local network.

This is useful for scenarios where you want to:
- Use a local LLM on your other less powerful devices by connecting them to a more powerful machine running LM Studio.
- Let multiple people use a single LM Studio instance on the network.
- Use the API from IoT devices, edge computing units, or other services in your local setup.

Once enabled, the server will bind to your local network IP address instead of localhost. The API access URL will be updated accordingly which you can use in your applications.

<img src="/assets/docs/serve-local-network.png" style="" data-caption="Serve LM Studio API Server on Local Network" />


## rest

### REST API v0

> The REST API includes enhanced stats such as Token / Second and Time To First Token (TTFT), as well as rich information about models such as loaded vs unloaded, max context, quantization, and more.


##### Requires [LM Studio 0.3.6](/download) or newer.

LM Studio now has its own REST API, in addition to OpenAI compatibility mode ([learn more](/docs/developer/openai-compat)).

The REST API includes enhanced stats such as Token / Second and Time To First Token (TTFT), as well as rich information about models such as loaded vs unloaded, max context, quantization, and more.

#### Supported API Endpoints

- [`GET /api/v0/models`](#get-apiv0models) - List available models
- [`GET /api/v0/models/{model}`](#get-apiv0modelsmodel) - Get info about a specific model
- [`POST /api/v0/chat/completions`](#post-apiv0chatcompletions) - Chat Completions (messages -> assistant response)
- [`POST /api/v0/completions`](#post-apiv0completions) - Text Completions (prompt -> completion)
- [`POST /api/v0/embeddings`](#post-apiv0embeddings) - Text Embeddings (text -> embedding)

---

### Start the REST API server

To start the server, run the following command:

```bash
lms server start
```

```lms_protip
You can run LM Studio as a service and get the server to auto-start on boot without launching the GUI. [Learn about Headless Mode](/docs/developer/core/headless).
```

## Endpoints

### `GET /api/v0/models`

List all loaded and downloaded models

**Example request**

```bash
curl http://localhost:1234/api/v0/models
```

**Response format**

```json
{
  "object": "list",
  "data": [
    {
      "id": "qwen2-vl-7b-instruct",
      "object": "model",
      "type": "vlm",
      "publisher": "mlx-community",
      "arch": "qwen2_vl",
      "compatibility_type": "mlx",
      "quantization": "4bit",
      "state": "not-loaded",
      "max_context_length": 32768
    },
    {
      "id": "meta-llama-3.1-8b-instruct",
      "object": "model",
      "type": "llm",
      "publisher": "lmstudio-community",
      "arch": "llama",
      "compatibility_type": "gguf",
      "quantization": "Q4_K_M",
      "state": "not-loaded",
      "max_context_length": 131072
    },
    {
      "id": "text-embedding-nomic-embed-text-v1.5",
      "object": "model",
      "type": "embeddings",
      "publisher": "nomic-ai",
      "arch": "nomic-bert",
      "compatibility_type": "gguf",
      "quantization": "Q4_0",
      "state": "not-loaded",
      "max_context_length": 2048
    }
  ]
}
```

---

### `GET /api/v0/models/{model}`

Get info about one specific model

**Example request**

```bash
curl http://localhost:1234/api/v0/models/qwen2-vl-7b-instruct
```

**Response format**

```json
{
  "id": "qwen2-vl-7b-instruct",
  "object": "model",
  "type": "vlm",
  "publisher": "mlx-community",
  "arch": "qwen2_vl",
  "compatibility_type": "mlx",
  "quantization": "4bit",
  "state": "not-loaded",
  "max_context_length": 32768
}
```

---

### `POST /api/v0/chat/completions`

Chat Completions API. You provide a messages array and receive the next assistant response in the chat.

**Example request**

```bash
curl http://localhost:1234/api/v0/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "granite-3.0-2b-instruct",
    "messages": [
      { "role": "system", "content": "Always answer in rhymes." },
      { "role": "user", "content": "Introduce yourself." }
    ],
    "temperature": 0.7,
    "max_tokens": -1,
    "stream": false
  }'
```

**Response format**

```json
{
  "id": "chatcmpl-i3gkjwthhw96whukek9tz",
  "object": "chat.completion",
  "created": 1731990317,
  "model": "granite-3.0-2b-instruct",
  "choices": [
    {
      "index": 0,
      "logprobs": null,
      "finish_reason": "stop",
      "message": {
        "role": "assistant",
        "content": "Greetings, I'm a helpful AI, here to assist,\nIn providing answers, with no distress.\nI'll keep it short and sweet, in rhyme you'll find,\nA friendly companion, all day long you'll bind."
      }
    }
  ],
  "usage": {
    "prompt_tokens": 24,
    "completion_tokens": 53,
    "total_tokens": 77
  },
  "stats": {
    "tokens_per_second": 51.43709529007664,
    "time_to_first_token": 0.111,
    "generation_time": 0.954,
    "stop_reason": "eosFound"
  },
  "model_info": {
    "arch": "granite",
    "quant": "Q4_K_M",
    "format": "gguf",
    "context_length": 4096
  },
  "runtime": {
    "name": "llama.cpp-mac-arm64-apple-metal-advsimd",
    "version": "1.3.0",
    "supported_formats": ["gguf"]
  }
}
```

---

### `POST /api/v0/completions`

Text Completions API. You provide a prompt and receive a completion.

**Example request**

```bash
curl http://localhost:1234/api/v0/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "granite-3.0-2b-instruct",
    "prompt": "the meaning of life is",
    "temperature": 0.7,
    "max_tokens": 10,
    "stream": false,
    "stop": "\n"
  }'
```

**Response format**

```json
{
  "id": "cmpl-p9rtxv6fky2v9k8jrd8cc",
  "object": "text_completion",
  "created": 1731990488,
  "model": "granite-3.0-2b-instruct",
  "choices": [
    {
      "index": 0,
      "text": " to find your purpose, and once you have",
      "logprobs": null,
      "finish_reason": "length"
    }
  ],
  "usage": {
    "prompt_tokens": 5,
    "completion_tokens": 9,
    "total_tokens": 14
  },
  "stats": {
    "tokens_per_second": 57.69230769230769,
    "time_to_first_token": 0.299,
    "generation_time": 0.156,
    "stop_reason": "maxPredictedTokensReached"
  },
  "model_info": {
    "arch": "granite",
    "quant": "Q4_K_M",
    "format": "gguf",
    "context_length": 4096
  },
  "runtime": {
    "name": "llama.cpp-mac-arm64-apple-metal-advsimd",
    "version": "1.3.0",
    "supported_formats": ["gguf"]
  }
}
```

---

### `POST /api/v0/embeddings`

Text Embeddings API. You provide a text and a representation of the text as an embedding vector is returned.

**Example request**

```bash
curl http://localhost:1234/api/v0/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "model": "text-embedding-nomic-embed-text-v1.5",
    "input": "Some text to embed"
  }
```

**Example response**

```json
{
  "object": "list",
  "data": [
    {
      "object": "embedding",
      "embedding": [
        -0.016731496900320053,
        0.028460891917347908,
        -0.1407836228609085,
        ... (truncated for brevity) ...,
        0.02505224384367466,
        -0.0037634256295859814,
        -0.04341062530875206
      ],
      "index": 0
    }
  ],
  "model": "text-embedding-nomic-embed-text-v1.5@q4_k_m",
  "usage": {
    "prompt_tokens": 0,
    "total_tokens": 0
  }
}
```

---

Please report bugs by opening an issue on [Github](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues).


## openai-compat

## OpenAI Compatibility Endpoints

> Send requests to Responses, Chat Completions (text and images), Completions, and Embeddings endpoints.


### Supported endpoints

<table class="flexible-cols">
  <thead>
    <tr>
      <th>Endpoint</th>
      <th>Method</th>
      <th>Docs</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><code>/v1/models</code></td>
      <td><apimethod method="GET" /></td>
      <td><a href="/docs/developer/openai-compat/models">Models</a></td>
    </tr>
    <tr>
      <td><code>/v1/responses</code></td>
      <td><apimethod method="POST" /></td>
      <td><a href="/docs/developer/openai-compat/responses">Responses</a></td>
    </tr>
    <tr>
      <td><code>/v1/chat/completions</code></td>
      <td><apimethod method="POST" /></td>
      <td><a href="/docs/developer/openai-compat/chat-completions">Chat Completions</a></td>
    </tr>
    <tr>
      <td><code>/v1/embeddings</code></td>
      <td><apimethod method="POST" /></td>
      <td><a href="/docs/developer/openai-compat/embeddings">Embeddings</a></td>
    </tr>
    <tr>
      <td><code>/v1/completions</code></td>
      <td><apimethod method="POST" /></td>
      <td><a href="/docs/developer/openai-compat/completions">Completions</a></td>
    </tr>
  </tbody>
</table>

<hr>

## Set the `base url` to point to LM Studio

You can reuse existing OpenAI clients (in Python, JS, C#, etc) by switching up the "base URL" property to point to your LM Studio instead of OpenAI's servers.

Note: The following examples assume the server port is `1234`

### Python Example

```diff
from openai import OpenAI

client = OpenAI(
+    base_url="http://localhost:1234/v1"
)