# Load Parameters

Set load-time parameters such as the context length, GPU offload ratio, and more.

### Set Load Parameters with `.model()`

The `.model()` retrieves a handle to a model that has already been loaded, or loads a new one on demand (JIT loading).

**Note**: if the model is already loaded, the given configuration will be **ignored**.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        model = lms.llm("qwen2.5-7b-instruct", config={
            "contextLength": 8192,
            "gpu": {
              "ratio": 0.5,
            }
        })

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            model = client.llm.model(
                "qwen2.5-7b-instruct",
                config={
                    "contextLength": 8192,
                    "gpu": {
                      "ratio": 0.5,
                    }
                }
            )

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            model = await client.llm.model(
                "qwen2.5-7b-instruct",
                config={
                    "contextLength": 8192,
                    "gpu": {
                      "ratio": 0.5,
                    }
                }
            )

```

See [`LLMLoadModelConfig`](./../../typescript/api-reference/llm-load-model-config) in the
Typescript SDK documentation for all configurable fields.

### Set Load Parameters with `.load_new_instance()`

The `.load_new_instance()` method creates a new model instance and loads it with the specified configuration.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        client = lms.get_default_client()
        model = client.llm.load_new_instance("qwen2.5-7b-instruct", config={
            "contextLength": 8192,
            "gpu": {
              "ratio": 0.5,
            }
        })

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            model = client.llm.load_new_instance(
                "qwen2.5-7b-instruct",
                config={
                    "contextLength": 8192,
                    "gpu": {
                      "ratio": 0.5,
                    }
                }
            )

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            model = await client.llm.load_new_instance(
                "qwen2.5-7b-instruct",
                config={
                    "contextLength": 8192,
                    "gpu": {
                      "ratio": 0.5,
                    }
                }
            )

```

See [`LLMLoadModelConfig`](./../../typescript/api-reference/llm-load-model-config) in the
Typescript SDK documentation for all configurable fields.


### Working with Chats

> APIs for representing a chat conversation with an LLM


SDK methods such as `llm.respond()`, `llm.applyPromptTemplate()`, or `llm.act()`
take in a chat parameter as an input.
There are a few ways to represent a chat when using the SDK.

## Option 1: Input a Single String

If your chat only has one single user message, you can use a single string to represent the chat.
Here is an example with the `.respond` method.

```lms_code_snippet
variants:
  "Single string":
    language: python
    code: |
      prediction = llm.respond("What is the meaning of life?")
```

## Option 2: Using the `Chat` Helper Class

For more complex tasks, it is recommended to use the `Chat` helper class.
It provides various commonly used methods to manage the chat.
Here is an example with the `Chat` class, where the initial system prompt
is supplied when initializing the chat instance, and then the initial user
message is added via the corresponding method call.

```lms_code_snippet
variants:
  "Simple chat":
    language: python
    code: |
      chat = Chat("You are a resident AI philosopher.")
      chat.add_user_message("What is the meaning of life?")

      prediction = llm.respond(chat)
```

You can also quickly construct a `Chat` object using the `Chat.from_history` method.

```lms_code_snippet
variants:
  "Chat history data":
    language: python
    code: |
      chat = Chat.from_history({"messages": [
        { "role": "system", "content": "You are a resident AI philosopher." },
        { "role": "user", "content": "What is the meaning of life?" },
      ]})

  "Single string":
    language: python
    code: |
      # This constructs a chat with a single user message
      chat = Chat.from_history("What is the meaning of life?")

```

## Option 3: Providing Chat History Data Directly

As the APIs that accept chat histories use `Chat.from_history` internally,
they also accept the chat history data format as a regular dictionary:

```lms_code_snippet
variants:
  "Chat history data":
    language: python
    code: |
      prediction = llm.respond({"messages": [
        { "role": "system", "content": "You are a resident AI philosopher." },
        { "role": "user", "content": "What is the meaning of life?" },
      ]})
```


## agent

### The `.act()` call

> How to use the `.act()` call to turn LLMs into autonomous agents that can perform tasks on your local machine.


## Automatic tool calling

We introduce the concept of execution "rounds" to describe the combined process of running a tool, providing its output to the LLM, and then waiting for the LLM to decide what to do next.

**Execution Round**

```
 â¢ run a tool ->
 â   â¢ provide the result to the LLM ->
 â       â¢ wait for the LLM to generate a response
 â
 ââââââââââââââââââââââââââââââââââââââââââ ââ (return)
```

A model might choose to run tools multiple times before returning a final result. For example, if the LLM is writing code, it might choose to compile or run the program, fix errors, and then run it again, rinse and repeat until it gets the desired result.

With this in mind, we say that the `.act()` API is an automatic "multi-round" tool calling API.

### Quick Example

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        def multiply(a: float, b: float) -> float:
            """Given two numbers a and b. Returns the product of them."""
            return a * b

        model = lms.llm("qwen2.5-7b-instruct")
        model.act(
          "What is the result of 12345 multiplied by 54321?",
          [multiply],
          on_message=print,
        )
```

### What does it mean for an LLM to "use a tool"?

LLMs are largely text-in, text-out programs. So, you may ask "how can an LLM use a tool?". The answer is that some LLMs are trained to ask the human to call the tool for them, and expect the tool output to to be provided back in some format.

Imagine you're giving computer support to someone over the phone. You might say things like "run this command for me ... OK what did it output? ... OK now click there and tell me what it says ...". In this case you're the LLM! And you're "calling tools" vicariously through the person on the other side of the line.

### Running multiple tool calls in parallel

By default, version 1.4.0 and later of the Python SDK will only run a single tool call request at a time,
even if the model requests multiple tool calls in a single response message. This ensures the requests will
be processed correctly even if the tool implementations do not support multiple concurrent calls.

When the tool implementations are known to be thread-safe, and are both slow and frequent enough to be worth
running in parallel, the `max_parallel_tool_calls` option specifies the maximum number of tool call requests
that will be processed in parallel from a single model response. This value defaults to 1 (waiting for each
tool call to complete before starting the next one). Setting this value to `None` will automatically scale
the maximum number of parallel tool calls to a multiple of the number of CPU cores available to the process.

### Important: Model Selection

The model selected for tool use will greatly impact performance.

Some general guidance when selecting a model:

- Not all models are capable of intelligent tool use
- Bigger is better (i.e., a 7B parameter model will generally perform better than a 3B parameter model)
- We've observed [Qwen2.5-7B-Instruct](https://model.lmstudio.ai/download/lmstudio-community/Qwen2.5-7B-Instruct-GGUF) to perform well in a wide variety of cases
- This guidance may change

### Example: Multiple Tools

The following code demonstrates how to provide multiple tools in a single `.act()` call.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import math
        import lmstudio as lms

        def add(a: int, b: int) -> int:
            """Given two numbers a and b, returns the sum of them."""
            return a + b

        def is_prime(n: int) -> bool:
            """Given a number n, returns True if n is a prime number."""
            if n < 2:
                return False
            sqrt = int(math.sqrt(n))
            for i in range(2, sqrt):
                if n % i == 0:
                    return False
            return True

        model = lms.llm("qwen2.5-7b-instruct")
        model.act(
          "Is the result of 12345 + 45668 a prime? Think step by step.",
          [add, is_prime],
          on_message=print,
        )
```

### Example: Chat Loop with Create File Tool

The following code creates a conversation loop with an LLM agent that can create files.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import readline # Enables input line editing
        from pathlib import Path

        import lmstudio as lms

        def create_file(name: str, content: str):
            """Create a file with the given name and content."""
            dest_path = Path(name)
            if dest_path.exists():
                return "Error: File already exists."
            try:
                dest_path.write_text(content, encoding="utf-8")
            except Exception as exc:
                return "Error: {exc!r}"
            return "File created."

        def print_fragment(fragment, round_index=0):
            # .act() supplies the round index as the second parameter
            # Setting a default value means the callback is also
            # compatible with .complete() and .respond().
            print(fragment.content, end="", flush=True)

        model = lms.llm()
        chat = lms.Chat("You are a task focused AI assistant")

        while True:
            try:
                user_input = input("You (leave blank to exit): ")
            except EOFError:
                print()
                break
            if not user_input:
                break
            chat.add_user_message(user_input)
            print("Bot: ", end="", flush=True)
            model.act(
                chat,
                [create_file],
                on_message=chat.append,
                on_prediction_fragment=print_fragment,
            )
            print()

```

### Progress Callbacks

Complex interactions with a tool using agent may take some time to process.

The regular progress callbacks for any prediction request are available,
but the expected capabilities differ from those for single round predictions.

* `on_prompt_processing_progress`: called during prompt processing for each
  prediction round. Receives the progress ratio (as a float) and the round
  index as positional arguments.
* `on_first_token`: called after prompt processing is complete for each prediction round.
  Receives the round index as its sole argument.
* `on_prediction_fragment`: called for each prediction fragment received by the client.
  Receives the prediction fragment and the round index as positional arguments.
* `on_message`: called with an assistant response message when each prediction round is
  complete, and with tool result messages as each tool call request is completed.
  Intended for appending received messages to a chat history instance, and hence
  does *not* receive the round index as an argument.

The following additional callbacks are available to monitor the prediction rounds:

* `on_round_start`: called before submitting the prediction request for each round.
  Receives the round index as its sole argument.
* `on_prediction_completed`: called after the prediction for the round has been completed,
  but before any requested tool calls have been initiated. Receives the round's prediction
  result as its sole argument. A round prediction result is a regular prediction result
  with an additional `round_index` attribute.
* `on_round_end`: called after any tool call requests for the round have been resolved.

Finally, applications may request notifications when agents emit invalid tool requests:

* `handle_invalid_tool_request`: called when a tool request was unable to be processed.
  Receives the exception that is about to be reported, as well as the original tool
  request that resulted in the problem. When no tool request is given, this is
  purely a notification of an unrecoverable error before the agent interaction raises
  the given exception (allowing the application to raise its own exception instead).
  When a tool request is given, it indicates that rather than being raised locally,
  the text description of the exception is going to be passed back to the agent
  as the result of that failed tool request. In these cases, the callback may either
  return `None` to indicate that the error description should be sent to the agent,
  raise the given exception (or a different exception) locally, or return a text
  string that should be sent to the agent instead of the error description.

For additional details on defining tools, and an example of overriding the invalid
tool request handling to raise all exceptions locally instead of passing them to
back the agent, refer to [Tool Definition](./tools.md).


### Tool Definition

> Define tools to be called by the LLM, and pass them to the model in the `act()` call.


You can define tools as regular Python functions and pass them to the model in the `act()` call.
Alternatively, tools can be defined with `lmstudio.ToolFunctionDef` in order to control the
name and description passed to the language model.

## Anatomy of a Tool

Follow one of the following examples to define functions as tools (the first approach
is typically going to be the most convenient):

```lms_code_snippet
  variants:
    "Python function":
      language: python
      code: |
        # Type hinted functions with clear names and docstrings
        # may be used directly as tool definitions
        def add(a: int, b: int) -> int:
            """Given two numbers a and b, returns the sum of them."""
            # The SDK ensures arguments are coerced to their specified types
            return a + b

        # Pass `add` directly to `act()` as a tool definition

    "ToolFunctionDef.from_callable":
      language: python
      code: |
        from lmstudio import ToolFunctionDef

        def cryptic_name(a: int, b: int) -> int:
            return a + b

        # Type hinted functions with cryptic names and missing or poor docstrings
        # can be turned into clear tool definitions with `from_callable`
        tool_def = ToolFunctionDef.from_callable(
          cryptic_name,
          name="add",
          description="Given two numbers a and b, returns the sum of them."
        )
        # Pass `tool_def` to `act()` as a tool definition

    "ToolFunctionDef":
      language: python
      code: |
        from lmstudio import ToolFunctionDef

        def cryptic_name(a, b):
            return a + b

        # Functions without type hints can be used without wrapping them
        # at runtime by defining a tool function directly.
        tool_def = ToolFunctionDef(
          name="add",
          description="Given two numbers a and b, returns the sum of them.",
          parameters={
            "a": int,
            "b": int,
          },
          implementation=cryptic_name,
        )
        # Pass `tool_def` to `act()` as a tool definition

```

**Important**: The tool name, description, and the parameter definitions are all passed to the model!

This means that your wording will affect the quality of the generation. Make sure to always provide a clear description of the tool so the model knows how to use it.

## Tools with External Effects (like Computer Use or API Calls)

Tools can also have external effects, such as creating files or calling programs and even APIs. By implementing tools with external effects, you
can essentially turn your LLMs into autonomous agents that can perform tasks on your local machine.

## Example: `create_file_tool`

### Tool Definition

```lms_code_snippet
  title: "create_file_tool.py"
  variants:
    Python:
      language: python
      code: |
        from pathlib import Path

        def create_file(name: str, content: str):
            """Create a file with the given name and content."""
            dest_path = Path(name)
            if dest_path.exists():
                return "Error: File already exists."
            try:
                dest_path.write_text(content, encoding="utf-8")
            except Exception as exc:
                return "Error: {exc!r}"
            return "File created."

```

### Example code using the `create_file` tool:

```lms_code_snippet
  title: "example.py"
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms
        from create_file_tool import create_file

        model = lms.llm("qwen2.5-7b-instruct")
        model.act(
          "Please create a file named output.txt with your understanding of the meaning of life.",
          [create_file],
        )
```

## Handling tool calling errors

By default, version 1.3.0 and later of the Python SDK will automatically convert
exceptions raised by tool calls to text and report them back to the language model.
In many cases, when notified of an error in this way, a language model is able
to either adjust its request to avoid the failure, or else accept the failure as
a valid response to its request (consider a prompt like `Attempt to divide 1 by 0
using the provided tool. Explain the result.`, where the expected
response is an explanation of the `ZeroDivisionError` exception the Python
interpreter raises when instructed to divide by zero).

This error handling behaviour can be overridden using the `handle_invalid_tool_request`
callback. For example, the following code reverts the error handling back to raising
exceptions locally in the client:

```lms_code_snippet
  title: "example.py"
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        def divide(numerator: float, denominator: float) -> float:
            """Divide the given numerator by the given denominator. Return the result."""
            return numerator / denominator

        model = lms.llm("qwen2.5-7b-instruct")
        chat = Chat()
        chat.add_user_message(
            "Attempt to divide 1 by 0 using the tool. Explain the result."
        )

        def _raise_exc_in_client(
            exc: LMStudioPredictionError, request: ToolCallRequest | None
        ) -> None:
            raise exc

        act_result = llm.act(
            chat,
            [divide],
            handle_invalid_tool_request=_raise_exc_in_client,
        )
```

When a tool request is passed in, the callback results are processed as follows:

* `None`: the original exception text is passed to the LLM unmodified
* a string: the returned string is passed to the LLM instead of the original
  exception text
* raising an exception (whether the passed in exception or a new exception):
  the raised exception is propagated locally in the client, terminating the
  prediction process

If no tool request is passed in, the callback invocation is a notification only,
and the exception cannot be converted to text for passing pack to the LLM
(although it can still be replaced with a different exception). These cases
indicate failures in the expected communication with the server API that mean
the prediction process cannot reasonably continue, so if the callback doesn't
raise an exception, the calling code will raise the original exception directly.


## embedding

## Embedding

> Generate text embeddings from input text


Generate embeddings for input text. Embeddings are vector representations of text that capture semantic meaning. Embeddings are a building block for RAG (Retrieval-Augmented Generation) and other similarity-based tasks.

### Prerequisite: Get an Embedding Model

If you don't yet have an embedding model, you can download a model like `nomic-ai/nomic-embed-text-v1.5` using the following command:

```bash
lms get nomic-ai/nomic-embed-text-v1.5
```

## Create Embeddings

To convert a string to a vector representation, pass it to the `embed` method on the corresponding embedding model handle.

```lms_code_snippet
  title: "example.py"
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        model = lms.embedding_model("nomic-embed-text-v1.5")

        embedding = model.embed("Hello, world!")

```


## tokenization

## Tokenization

> Tokenize text using a model's tokenizer


Models use a tokenizer to internally convert text into "tokens" they can deal with more easily. LM Studio exposes this tokenizer for utility.

## Tokenize

You can tokenize a string with a loaded LLM or embedding model using the SDK.
In the below examples, the LLM reference can be replaced with an
embedding model reference without requiring any other changes.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        model = lms.llm()

        tokens = model.tokenize("Hello, world!")

        print(tokens) # Array of token IDs.
```

## Count tokens

If you only care about the number of tokens, simply check the length of the resulting array.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        token_count = len(model.tokenize("Hello, world!"))
        print("Token count:", token_count)
```

### Example: count context

You can determine if a given conversation fits into a model's context by doing the following:

1. Convert the conversation to a string using the prompt template.
2. Count the number of tokens in the string.
3. Compare the token count to the model's context length.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        def does_chat_fit_in_context(model: lms.LLM, chat: lms.Chat) -> bool:
            # Convert the conversation to a string using the prompt template.
            formatted = model.apply_prompt_template(chat)
            # Count the number of tokens in the string.
            token_count = len(model.tokenize(formatted))
            # Get the current loaded context length of the model
            context_length = model.get_context_length()
            return token_count < context_length

        model = lms.llm()

        chat = lms.Chat.from_history({
            "messages": [
                { "role": "user", "content": "What is the meaning of life." },
                { "role": "assistant", "content": "The meaning of life is..." },
                # ... More messages
            ]
        })

        print("Fits in context:", does_chat_fit_in_context(model, chat))

```


## manage-models

### List Downloaded Models

> APIs to list the available models in a given local environment


You can iterate through locally available models using the downloaded model listing methods.

The listing results offer `.model()` and `.load_new_instance()` methods, which allow the
downloaded model reference to be converted in the full SDK handle for a loaded model.

## Available Models on the LM Studio Server

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        downloaded = lms.list_downloaded_models()
        llm_only = lms.list_downloaded_models("llm")
        embedding_only = lms.list_downloaded_models("embedding")

        for model in downloaded:
            print(model)

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            downloaded = client.list_downloaded_models()
            llm_only = client.llm.list_downloaded()
            embedding_only = client.embedding.list_downloaded()

        for model in downloaded:
            print(model)

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            downloaded = await client.list_downloaded_models()
            llm_only = await client.llm.list_downloaded()
            embedding_only = await client.embedding.list_downloaded()

        for model in downloaded:
            print(model)

```
This will give you results equivalent to using [`lms ls`](../../cli/ls) in the CLI.


### Example output:

```python
DownloadedLlm(model_key='qwen2.5-7b-instruct-1m', display_name='Qwen2.5 7B Instruct 1M', architecture='qwen2', vision=False)
DownloadedEmbeddingModel(model_key='text-embedding-nomic-embed-text-v1.5', display_name='Nomic Embed Text v1.5', architecture='nomic-bert')
```


### List Loaded Models

> Query which models are currently loaded


You can iterate through models loaded into memory using the functions and methods shown below.

The results are full SDK model handles, allowing access to all model functionality. 


## List Models Currently Loaded in Memory

This will give you results equivalent to using [`lms ps`](../../cli/ps) in the CLI.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        all_loaded_models = lms.list_loaded_models()
        llm_only = lms.list_loaded_models("llm")
        embedding_only = lms.list_loaded_models("embedding")

        print(all_loaded_models)

    Python (scoped resource API):
      language: python
      code: |
        import lms

        with lms.Client() as client:
            all_loaded_models = client.list_loaded_models()
            llm_only = client.llm.list_loaded()
            embedding_only = client.embedding.list_loaded()

            print(all_loaded_models)

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            all_loaded_models = await client.list_loaded_models()
            llm_only = await client.llm.list_loaded()
            embedding_only = await client.embedding.list_loaded()

            print(all_loaded_models)

```


### Manage Models in Memory

> APIs to load, access, and unload models from memory


AI models are huge. It can take a while to load them into memory. LM Studio's SDK allows you to precisely control this process.

**Model namespaces:**

- LLMs are accessed through the `client.llm` namespace
- Embedding models are accessed through the `client.embedding` namespace
- `lmstudio.llm` is equivalent to `client.llm.model` on the default client
- `lmstudio.embedding_model` is equivalent to `client.embedding.model` on the default client

**Most commonly:**

- Use `.model()` to get any currently loaded model
- Use `.model("model-key")` to use a specific model

**Advanced (manual model management):**

- Use `.load_new_instance("model-key")` to load a new instance of a model
- Use `.unload("model-key")` or `model_handle.unload()` to unload a model from memory

## Get the Current Model with `.model()`

If you already have a model loaded in LM Studio (either via the GUI or `lms load`),
you can use it by calling `.model()` without any arguments.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        model = lms.llm()

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            model = client.llm.model()

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            model = await client.llm.model()

```

## Get a Specific Model with `.model("model-key")`

If you want to use a specific model, you can provide the model key as an argument to `.model()`.

#### Get if Loaded, or Load if not

Calling `.model("model-key")` will load the model if it's not already loaded, or return the existing instance if it is.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        model = lms.llm("qwen/qwen3-4b-2507")

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            model = client.llm.model("qwen/qwen3-4b-2507")

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            model = await client.llm.model("qwen/qwen3-4b-2507")

```

<!--
Learn more about the `.model()` method and the parameters it accepts in the [API Reference](../api-reference/model).
-->

## Load a New Instance of a Model with `.load_new_instance()`

Use `load_new_instance()` to load a new instance of a model, even if one already exists.
This allows you to have multiple instances of the same or different models loaded at the same time.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        client = lms.get_default_client()
        model = client.llm.load_new_instance("qwen/qwen3-4b-2507")
        another_model = client.llm.load_new_instance("qwen/qwen3-4b-2507", "my-second-model")

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            model = client.llm.load_new_instance("qwen/qwen3-4b-2507")
            another_model = client.llm.load_new_instance("qwen/qwen3-4b-2507", "my-second-model")

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            model = await client.llm.load_new_instance("qwen/qwen3-4b-2507")
            another_model = await client.llm.load_new_instance("qwen/qwen3-4b-2507", "my-second-model")

```

<!--
Learn more about the `.load_new_instance()` method and the parameters it accepts in the [API Reference](../api-reference/load_new_instance).
-->

### Note about Instance Identifiers

If you provide an instance identifier that already exists, the server will throw an error.
So if you don't really care, it's safer to not provide an identifier, in which case
the server will generate one for you. You can always check in the server tab in LM Studio, too!

## Unload a Model from Memory with `.unload()`

Once you no longer need a model, you can unload it by simply calling `unload()` on its handle.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        model = lms.llm()
        model.unload()

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            model = client.llm.model()
            model.unload()

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            model = await client.llm.model()
            await model.unload()

```

## Set Custom Load Config Parameters

You can also specify the same load-time configuration options when loading a model, such as Context Length and GPU offload.

See [load-time configuration](../llm-prediction/parameters) for more.

## Set an Auto Unload Timer (TTL)

You can specify a _time to live_ for a model you load, which is the idle time (in seconds)
after the last request until the model unloads. See [Idle TTL](/docs/app/api/ttl-and-auto-evict) for more on this.

```lms_protip
If you specify a TTL to `model()`, it will only apply if `model()` loads
a new instance, and will _not_ retroactively change the TTL of an existing instance.
```

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        model = lms.llm("qwen/qwen3-4b-2507", ttl=3600)

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            model = client.llm.model("qwen/qwen3-4b-2507", ttl=3600)

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            model = await client.llm.model("qwen/qwen3-4b-2507", ttl=3600)

```

<!--
(TODO?: Cover the JIT implications of setting a TTL, and the default TTL variations)
-->


## model-info

### Get Context Length

> API to get the maximum context length of a model.


LLMs and embedding models, due to their fundamental architecture, have a property called `context length`, and more specifically a **maximum** context length. Loosely speaking, this is how many tokens the models can "keep in memory" when generating text or embeddings. Exceeding this limit will result in the model behaving erratically.

## Use the `get_context_length()` function on the model object

It's useful to be able to check the context length of a model, especially as an extra check before providing potentially long input to the model.

```lms_code_snippet
  title: "example.py"
  variants:
    "Python (convenience API)":
      language: python
      code: |
        context_length = model.get_context_length()
```

The `model` in the above code snippet is an instance of a loaded model you get from the `llm.model` method. See [Manage Models in Memory](../manage-models/loading) for more information.

### Example: Check if the input will fit in the model's context window

You can determine if a given conversation fits into a model's context by doing the following:

1. Convert the conversation to a string using the prompt template.
2. Count the number of tokens in the string.
3. Compare the token count to the model's context length.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        def does_chat_fit_in_context(model: lms.LLM, chat: lms.Chat) -> bool:
            # Convert the conversation to a string using the prompt template.
            formatted = model.apply_prompt_template(chat)
            # Count the number of tokens in the string.
            token_count = len(model.tokenize(formatted))
            # Get the current loaded context length of the model
            context_length = model.get_context_length()
            return token_count < context_length

        model = lms.llm()

        chat = lms.Chat.from_history({
            "messages": [
                { "role": "user", "content": "What is the meaning of life." },
                { "role": "assistant", "content": "The meaning of life is..." },
                # ... More messages
            ]
        })

        print("Fits in context:", does_chat_fit_in_context(model, chat))

```


### Get Load Config

> Get the load configuration of the model


*Required Python SDK version*: **1.2.0**

LM Studio allows you to configure certain parameters when loading a model
[through the server UI](/docs/advanced/per-model) or [through the API](/docs/api/sdk/load-model).

You can retrieve the config with which a given model was loaded using the SDK.

In the below examples, the LLM reference can be replaced with an
embedding model reference without requiring any other changes.

```lms_protip
Context length is a special case that [has its own method](/docs/api/sdk/get-context-length).
```

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        model = lms.llm()

        print(model.get_load_config())

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            model = client.llm.model()

            print(model.get_load_config())

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.Client() as client:
            model = await client.llm.model()

            print(await model.get_load_config())

```


### Get Model Info

> Get information about the model


You can access general information and metadata about a model itself from a loaded
instance of that model.

In the below examples, the LLM reference can be replaced with an
embedding model reference without requiring any other changes.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        model = lms.llm()

        print(model.get_info())

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            model = client.llm.model()

            print(model.get_info())

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            model = await client.llm.model()

            print(await model.get_info())

```

## Example output

```python
LlmInstanceInfo.from_dict({
  "architecture": "qwen2",
  "contextLength": 4096,
  "displayName": "Qwen2.5 7B Instruct 1M",
  "format": "gguf",
  "identifier": "qwen2.5-7b-instruct",
  "instanceReference": "lpFZPBQjhSZPrFevGyY6Leq8",
  "maxContextLength": 1010000,
  "modelKey": "qwen2.5-7b-instruct-1m",
  "paramsString": "7B",
  "path": "lmstudio-community/Qwen2.5-7B-Instruct-1M-GGUF/Qwen2.5-7B-Instruct-1M-Q4_K_M.gguf",
  "sizeBytes": 4683073888,
  "trainedForToolUse": true,
  "type": "llm",
  "vision": false
})
```