# Interactive Convenience, Deterministic Resource Management, or Structured Concurrency?

As shown in the example above, there are three distinct approaches for working
with the LM Studio Python SDK.

The first is the interactive convenience API (listed as "Python (convenience API)"
in examples), which focuses on the use of a default LM Studio client instance for
convenient interactions at a synchronous Python prompt, or when using Jupyter notebooks.

The second is a synchronous scoped resource API (listed as "Python (scoped resource API)"
in examples), which uses context managers to ensure that allocated resources
(such as network connections) are released deterministically, rather than
potentially remaining open until the entire process is terminated.

The last is an asynchronous structured concurrency API (listed as "Python (asynchronous API)" in
examples), which is designed for use in asynchronous programs that follow the design principles of
["structured concurrency"](https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/)
in order to ensure the background tasks handling the SDK's connections to the API server host
are managed correctly. Asynchronous applications which do not adhere to those design principles
will need to rely on threaded access to the synchronous scoped resource API rather than attempting
to use the SDK's native asynchronous API. Python SDK version 1.5.0 is the first version to fully
support the asynchronous API.

Some examples are common between the interactive convenience API and the synchronous scoped
resource API. These examples are listed as "Python (synchronous API)".

## Timeouts in the synchronous API

_Required Python SDK version_: **1.5.0**

Starting in Python SDK version 1.5.0, the synchronous API defaults to timing out after 60 seconds
with no activity when waiting for a response or streaming event notification from the API server.

The number of seconds to wait for responses and event notifications can be adjusted using the
`lmstudio.set_sync_api_timeout()` function. Setting the timeout to `None` disables the timeout
entirely (restoring the behaviour of previous SDK versions).

The current synchronous API timeout can be queried using the `lmstudio.get_sync_api_timeout()`
function.

## Timeouts in the asynchronous API

_Required Python SDK version_: **1.5.0**

As asynchronous coroutines support cancellation, there is no specific timeout support implemented
in the asynchronous API. Instead, general purpose async timeout mechanisms, such as
[`asyncio.wait_for()`](https://docs.python.org/3/library/asyncio-task.html#asyncio.wait_for) or
[`anyio.move_on_after()`](https://anyio.readthedocs.io/en/stable/cancellation.html#timeouts),
should be used.


## getting-started

### Project Setup

> Set up your `lmstudio-python` app or script.


`lmstudio` is a library published on PyPI that allows you to use `lmstudio-python` in your own projects.
It is open source and developed on GitHub.
You can find the source code [here](https://github.com/lmstudio-ai/lmstudio-python).

## Installing `lmstudio-python`

As it is published to PyPI, `lmstudio-python` may be installed using `pip`
or your preferred project dependency manager (`pdm` and `uv` are shown, but other
Python project management tools offer similar dependency addition commands).

```lms_code_snippet
  variants:
    pip:
      language: bash
      code: |
        pip install lmstudio
    pdm:
      language: bash
      code: |
        pdm add lmstudio
    uv:
      language: bash
      code: |
        uv add lmstudio
```

## Customizing the server API host and TCP port

All of the examples in the documentation assume that the server API is running locally
on one of the default application ports (Note: in Python SDK versions prior to 1.5.0, the
SDK also required that the optional HTTP REST server be enabled).

The network location of the server API can be overridden by
passing a `"host:port"` string when creating the client instance.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms
        SERVER_API_HOST = "localhost:1234"

        # This must be the *first* convenience API interaction (otherwise the SDK
        # implicitly creates a client that accesses the default server API host)
        lms.configure_default_client(SERVER_API_HOST)

        # Note: the dedicated configuration API was added in lmstudio-python 1.3.0
        # For compatibility with earlier SDK versions, it is still possible to use
        # lms.get_default_client(SERVER_API_HOST) to configure the default client

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms
        SERVER_API_HOST = "localhost:1234"

        # When using the scoped resource API, each client instance
        # can be configured to use a specific server API host
        with lms.Client(SERVER_API_HOST) as client:
            model = client.llm.model()

            for fragment in model.respond_stream("What is the meaning of life?"):
                print(fragment.content, end="", flush=True)
            print() # Advance to a new line at the end of the response

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms
        SERVER_API_HOST = "localhost:1234"

        # When using the asynchronous API, each client instance
        # can be configured to use a specific server API host
        async with lms.AsyncClient(SERVER_API_HOST) as client:
            model = await client.llm.model()

            for fragment in await model.respond_stream("What is the meaning of life?"):
                print(fragment.content, end="", flush=True)
            print() # Advance to a new line at the end of the response
```

### Checking a specified API server host is running

*Required Python SDK version*: **1.5.0**

While the most common connection pattern is to let the SDK raise an exception if it can't
connect to the specified API server host, the SDK also supports running the API check directly
without creating an SDK client instance first:

```lms_code_snippet
  variants:
    "Python (synchronous API)":
      language: python
      code: |
        import lmstudio as lms
        SERVER_API_HOST = "localhost:1234"

        if lms.Client.is_valid_api_host(SERVER_API_HOST):
            print(f"An LM Studio API server instance is available at {SERVER_API_HOST}")
        else:
            print("No LM Studio API server instance found at {SERVER_API_HOST}")

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms
        SERVER_API_HOST = "localhost:1234"

        if await lms.AsyncClient.is_valid_api_host(SERVER_API_HOST):
            print(f"An LM Studio API server instance is available at {SERVER_API_HOST}")
        else:
            print("No LM Studio API server instance found at {SERVER_API_HOST}")
```


### Determining the default local API server port

*Required Python SDK version*: **1.5.0**

When no API server host is specified, the SDK queries a number of ports on the local loopback
interface for a running API server instance. This scan is repeated for each new client instance
created. Rather than letting the SDK perform this scan implicitly, the SDK also supports running
the scan explicitly, and passing in the reported API server details when creating clients:

```lms_code_snippet
  variants:
    "Python (synchronous API)":
      language: python
      code: |
        import lmstudio as lms

        api_host = lms.Client.find_default_local_api_host()
        if api_host is not None:
            print(f"An LM Studio API server instance is available at {api_host}")
          else:
            print("No LM Studio API server instance found on any of the default local ports")

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        api_host = await lms.AsyncClient.find_default_local_api_host()
        if api_host is not None:
            print(f"An LM Studio API server instance is available at {api_host}")
          else:
            print("No LM Studio API server instance found on any of the default local ports")
```


### Using `lmstudio-python` in REPL

> You can use `lmstudio-python` in REPL (Read-Eval-Print Loop) to interact with LLMs, manage models, and more.


To simplify interactive use, `lmstudio-python` offers a convenience API which manages
its resources via `atexit` hooks, allowing a default synchronous client session
to be used across multiple interactive commands.

This convenience API is shown in the examples throughout the documentation as the
`Python (convenience API)` tab (alongside the `Python (scoped resource API)` examples,
which use `with` statements to ensure deterministic cleanup of network communication
resources).

The convenience API allows the standard Python REPL, or more flexible alternatives like
Juypter Notebooks, to be used to interact with AI models loaded into LM Studio. For
example:

```lms_code_snippet
  title: "Python REPL"
  variants:
    "Interactive chat session":
      language: python
      code: |
        >>> import lmstudio as lms
        >>> loaded_models = lms.list_loaded_models()
        >>> for idx, model in enumerate(loaded_models):
        ...     print(f"{idx:>3} {model}")
        ...
          0 LLM(identifier='qwen2.5-7b-instruct')
        >>> model = loaded_models[0]
        >>> chat = lms.Chat("You answer questions concisely")
        >>> chat = lms.Chat("You answer questions concisely")
        >>> chat.add_user_message("Tell me three fruits")
        UserMessage(content=[TextData(text='Tell me three fruits')])
        >>> print(model.respond(chat, on_message=chat.append))
        Banana, apple, orange.
        >>> chat.add_user_message("Tell me three more fruits")
        UserMessage(content=[TextData(text='Tell me three more fruits')])
        >>> print(model.respond(chat, on_message=chat.append))
        Mango, strawberry, avocado.
        >>> chat.add_user_message("How many fruits have you told me?")
        UserMessage(content=[TextData(text='How many fruits have you told me?')])
        >>> print(model.respond(chat, on_message=chat.append))
        You asked for three initial fruits and three more, so I've listed a total of six fruits.

```

While not primarily intended for use this way, the SDK's asynchronous structured concurrency API
is compatible with the asynchronous Python REPL that is launched by `python -m asyncio`.
For example:

```lms_code_snippet
  title: "Python REPL"
  variants:
    "Asynchronous chat session":
      language: python
      code: |
        # Note: assumes use of the "python -m asyncio" asynchronous REPL (or equivalent)
        # Requires Python SDK version 1.5.0 or later
        >>> from contextlib import AsyncExitStack
        >>> import lmstudio as lms
        >>> resources = AsyncExitStack()
        >>> client = await resources.enter_async_context(lms.AsyncClient())
        >>> loaded_models = await client.llm.list_loaded()
        >>> for idx, model in enumerate(loaded_models):
        ...     print(f"{idx:>3} {model}")
        ...
          0 AsyncLLM(identifier='qwen2.5-7b-instruct-1m')
        >>> model = loaded_models[0]
        >>> chat = lms.Chat("You answer questions concisely")
        >>> chat.add_user_message("Tell me three fruits")
        UserMessage(content=[TextData(text='Tell me three fruits')])
        >>> print(await model.respond(chat, on_message=chat.append))
        Apple, banana, and orange.
        >>> chat.add_user_message("Tell me three more fruits")
        UserMessage(content=[TextData(text='Tell me three more fruits')])
        >>> print(await model.respond(chat, on_message=chat.append))
        Mango, strawberry, and pineapple.
        >>> chat.add_user_message("How many fruits have you told me?")
        UserMessage(content=[TextData(text='How many fruits have you told me?')])
        >>> print(await model.respond(chat, on_message=chat.append))
        You asked for three fruits initially, then three more, so I've listed six fruits in total.

```


### Authentication

> Using API Tokens in LM Studio


##### Requires [LM Studio 0.4.0](/download) or newer.

LM Studio supports API Tokens for authentication, providing a secure and convenient way to access the LM Studio API.

By default, the LM Studio API runs **without enforcing authentication**. For production or shared environments, enable API Token authentication for secure access.

```lms_info
To enable API Token authentication, create tokens and control granular permissions, check [this guide](/docs/developer/core/authentication) for more details.
```

## Providing the API Token

The API Token can be provided in two ways:

1. **Environment Variable (Recommended)**: Set the `LM_API_TOKEN` environment variable, and the SDK will automatically read it.
2. **Function Argument**: Pass the token directly as the `api_token` parameter.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        # Configure the default client with an API token
        lms.configure_default_client(api_token="your-token-here")

        model = lms.llm()
        result = model.respond("What is the meaning of life?")
        print(result)

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        # Pass api_token to the Client constructor
        with lms.Client(api_token="your-token-here") as client:
            model = client.llm.model()
            result = model.respond("What is the meaning of life?")
            print(result)

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        # Pass api_token to the AsyncClient constructor
        async with lms.AsyncClient(api_token="your-token-here") as client:
            model = await client.llm.model()
            result = await model.respond("What is the meaning of life?")
            print(result)
```


## llm-prediction

### Chat Completions

> APIs for a multi-turn chat conversations with an LLM


Use `llm.respond(...)` to generate completions for a chat conversation.

## Quick Example: Generate a Chat Response

The following snippet shows how to obtain the AI's response to a quick chat prompt.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        model = lms.llm()
        print(model.respond("What is the meaning of life?"))

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            model = client.llm.model()
            print(model.respond("What is the meaning of life?"))

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            model = await client.llm.model()
            print(await model.respond("What is the meaning of life?"))

```

## Streaming a Chat Response

The following snippet shows how to stream the AI's response to a chat prompt,
displaying text fragments as they are received (rather than waiting for the
entire response to be generated before displaying anything).

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms
        model = lms.llm()

        for fragment in model.respond_stream("What is the meaning of life?"):
            print(fragment.content, end="", flush=True)
        print() # Advance to a new line at the end of the response

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            model = client.llm.model()

            for fragment in model.respond_stream("What is the meaning of life?"):
                print(fragment.content, end="", flush=True)
            print() # Advance to a new line at the end of the response

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            model = await client.llm.model()

            async for fragment in model.respond_stream("What is the meaning of life?"):
                print(fragment.content, end="", flush=True)
            print() # Advance to a new line at the end of the response

```

## Cancelling a Chat Response

See the [Cancelling a Prediction](./cancelling-predictions) section for how to cancel a prediction in progress.

## Obtain a Model

First, you need to get a model handle.
This can be done using the top-level `llm` convenience API,
or the `model` method in the `llm` namespace when using the scoped resource API.
For example, here is how to use Qwen2.5 7B Instruct.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        model = lms.llm("qwen2.5-7b-instruct")

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            model = client.llm.model("qwen2.5-7b-instruct")

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            model = await client.llm.model("qwen2.5-7b-instruct")

```

There are other ways to get a model handle. See [Managing Models in Memory](./../manage-models/loading) for more info.

## Manage Chat Context

The input to the model is referred to as the "context".
Conceptually, the model receives a multi-turn conversation as input,
and it is asked to predict the assistant's response in that conversation.

```lms_code_snippet
  variants:
    "Constructing a Chat object":
      language: python
      code: |
        import lmstudio as lms

        # Create a chat with an initial system prompt.
        chat = lms.Chat("You are a resident AI philosopher.")

        # Build the chat context by adding messages of relevant types.
        chat.add_user_message("What is the meaning of life?")
        # ... continued in next example

  "From chat history data":
      language: python
      code: |
        import lmstudio as lms

        # Create a chat object from a chat history dict
        chat = lms.Chat.from_history({
            "messages": [
                { "role": "system", "content": "You are a resident AI philosopher." },
                { "role": "user", "content": "What is the meaning of life?" },
            ]
        })
        # ... continued in next example

```

See [Working with Chats](./working-with-chats) for more information on managing chat context.

<!-- , and [`Chat`](./../api-reference/chat) for API reference for the `Chat` class. -->

## Generate a response

You can ask the LLM to predict the next response in the chat context using the `respond()` method.

```lms_code_snippet
  variants:
    "Non-streaming (synchronous API)":
      language: python
      code: |
        # The `chat` object is created in the previous step.
        result = model.respond(chat)

        print(result)

    "Streaming (synchronous API)":
      language: python
      code: |
        # The `chat` object is created in the previous step.
        prediction_stream = model.respond_stream(chat)

        for fragment in prediction_stream:
            print(fragment.content, end="", flush=True)
        print() # Advance to a new line at the end of the response

    "Non-streaming (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        # The `chat` object is created in the previous step.
        result = await model.respond(chat)

        print(result)

    "Streaming (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        # The `chat` object is created in the previous step.
        prediction_stream = await model.respond_stream(chat)

        async for fragment in prediction_stream:
            print(fragment.content, end="", flush=True)
        print() # Advance to a new line at the end of the response

```

## Customize Inferencing Parameters

You can pass in inferencing parameters via the `config` keyword parameter on `.respond()`.

```lms_code_snippet
  variants:
    "Non-streaming (synchronous API)":
      language: python
      code: |
        result = model.respond(chat, config={
            "temperature": 0.6,
            "maxTokens": 50,
        })

    "Streaming (synchronous API)":
      language: python
      code: |
        prediction_stream = model.respond_stream(chat, config={
            "temperature": 0.6,
            "maxTokens": 50,
        })

    "Non-streaming (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        result = await model.respond(chat, config={
            "temperature": 0.6,
            "maxTokens": 50,
        })

    "Streaming (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        prediction_stream = await model.respond_stream(chat, config={
            "temperature": 0.6,
            "maxTokens": 50,
        })

```

See [Configuring the Model](./parameters) for more information on what can be configured.

## Print prediction stats

You can also print prediction metadata, such as the model used for generation, number of generated
tokens, time to first token, and stop reason.

```lms_code_snippet
  variants:
    "Non-streaming":
      language: python
      code: |
        # `result` is the response from the model.
        print("Model used:", result.model_info.display_name)
        print("Predicted tokens:", result.stats.predicted_tokens_count)
        print("Time to first token (seconds):", result.stats.time_to_first_token_sec)
        print("Stop reason:", result.stats.stop_reason)

    "Streaming":
      language: python
      code: |
        # After iterating through the prediction fragments,
        # the overall prediction result may be obtained from the stream
        result = prediction_stream.result()

        print("Model used:", result.model_info.display_name)
        print("Predicted tokens:", result.stats.predicted_tokens_count)
        print("Time to first token (seconds):", result.stats.time_to_first_token_sec)
        print("Stop reason:", result.stats.stop_reason)

```

Both the non-streaming and streaming result access is consistent across the synchronous and
asynchronous APIs, as `prediction_stream.result()` is a non-blocking API that raises an exception
if no result is available (either because the prediction is still running, or because the
prediction request failed). Prediction streams also offer a blocking (synchronous API) or
awaitable (asynchronous API) `prediction_stream.wait_for_result()` method that internally handles
iterating the stream to completion before returning the result.

## Example: Multi-turn Chat

```lms_code_snippet
  title: "chatbot.py"
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        model = lms.llm()
        chat = lms.Chat("You are a task focused AI assistant")

        while True:
            try:
                user_input = input("You (leave blank to exit): ")
            except EOFError:
                print()
                break
            if not user_input:
                break
            chat.add_user_message(user_input)
            prediction_stream = model.respond_stream(
                chat,
                on_message=chat.append,
            )
            print("Bot: ", end="", flush=True)
            for fragment in prediction_stream:
                print(fragment.content, end="", flush=True)
            print()

```

### Progress Callbacks

Long prompts will often take a long time to first token, i.e. it takes the model a long time to process your prompt.
If you want to get updates on the progress of this process, you can provide a float callback to `respond`
that receives a float from 0.0-1.0 representing prompt processing progress.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        llm = lms.llm()

        response = llm.respond(
            "What is LM Studio?",
            on_prompt_processing_progress = (lambda progress: print(f"{progress*100}% complete")),
        )

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            llm = client.llm.model()

            response = llm.respond(
                "What is LM Studio?",
                on_prompt_processing_progress = (lambda progress: print(f"{progress*100}% complete")),
            )

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            llm = await client.llm.model()

            response = await llm.respond(
                "What is LM Studio?",
                on_prompt_processing_progress = (lambda progress: print(f"{progress*100}% complete")),
            )


```

In addition to `on_prompt_processing_progress`, the other available progress callbacks are:

- `on_first_token`: called after prompt processing is complete and the first token is being emitted.
  Does not receive any arguments (use the streaming iteration API or `on_prediction_fragment`
  to process tokens as they are emitted).
- `on_prediction_fragment`: called for each prediction fragment received by the client.
  Receives the same prediction fragments as iterating over the stream iteration API.
- `on_message`: called with an assistant response message when the prediction is complete.
  Intended for appending received messages to a chat history instance.


### Image Input

> API for passing images as input to the model


*Required Python SDK version*: **1.1.0**

Some models, known as VLMs (Vision-Language Models), can accept images as input. You can pass images to the model using the `.respond()` method.

### Prerequisite: Get a VLM (Vision-Language Model)

If you don't yet have a VLM, you can download a model like `qwen2-vl-2b-instruct` using the following command:

```bash
lms get qwen2-vl-2b-instruct
```

## 1. Instantiate the Model

Connect to LM Studio and obtain a handle to the VLM (Vision-Language Model) you want to use.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        model = lms.llm("qwen2-vl-2b-instruct")

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            model = client.llm.model("qwen2-vl-2b-instruct")

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            model = await client.llm.model("qwen2-vl-2b-instruct")

```

## 2. Prepare the Image

Use the `prepare_image()` function or `files` namespace method to
get a handle to the image that can subsequently be passed to the model.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        image_path = "/path/to/image.jpg" # Replace with the path to your image
        image_handle = lms.prepare_image(image_path)

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            image_path = "/path/to/image.jpg" # Replace with the path to your image
            image_handle = client.files.prepare_image(image_path)

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            image_path = "/path/to/image.jpg" # Replace with the path to your image
            image_handle = await client.files.prepare_image(image_path)

```

If you only have the raw data of the image, you can supply the raw data directly as a bytes
object without having to write it to disk first. Due to this feature, binary filesystem
paths are *not* supported (as they will be handled as malformed image data rather than as
filesystem paths).

Binary IO objects are also accepted as local file inputs.

The LM Studio server supports JPEG, PNG, and WebP image formats.

## 3. Pass the Image to the Model in `.respond()`

Generate a prediction by passing the image to the model in the `.respond()` method.

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms

        image_path = "/path/to/image.jpg" # Replace with the path to your image
        image_handle = lms.prepare_image(image_path)
        model = lms.llm("qwen2-vl-2b-instruct")
        chat = lms.Chat()
        chat.add_user_message("Describe this image please", images=[image_handle])
        prediction = model.respond(chat)

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            image_path = "/path/to/image.jpg" # Replace with the path to your image
            image_handle = client.files.prepare_image(image_path)
            model = client.llm.model("qwen2-vl-2b-instruct")
            chat = lms.Chat()
            chat.add_user_message("Describe this image please", images=[image_handle])
            prediction = model.respond(chat)

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            image_path = "/path/to/image.jpg" # Replace with the path to your image
            image_handle = client.files.prepare_image(image_path)
            model = await client.llm.model("qwen2-vl-2b-instruct")
            chat = lms.Chat()
            chat.add_user_message("Describe this image please", images=[image_handle])
            prediction = await model.respond(chat)

```


### Cancelling Predictions

> Stop an ongoing prediction in `lmstudio-python`


One benefit of using the streaming API is the ability to cancel the
prediction request based on criteria that can't be represented using
the `stopStrings` or `maxPredictedTokens` configuration settings.

The following snippet illustrates cancelling the request in response
to an application specification cancellation condition (such as polling
an event set by another thread).

```lms_code_snippet
  variants:
    "Python (convenience API)":
      language: python
      code: |
        import lmstudio as lms
        model = lms.llm()

        prediction_stream = model.respond_stream("What is the meaning of life?")
        cancelled = False
        for fragment in prediction_stream:
            if ...: # Cancellation condition will be app specific
                cancelled = True
                prediction_stream.cancel()
                # Note: it is recommended to let the iteration complete,
                # as doing so allows the partial result to be recorded.
                # Breaking the loop *is* permitted, but means the partial result
                # and final prediction stats won't be available to the client
        # The stream allows the prediction result to be retrieved after iteration
        if not cancelled:
            print(prediction_stream.result())

    "Python (scoped resource API)":
      language: python
      code: |
        import lmstudio as lms

        with lms.Client() as client:
            model = client.llm.model()

            prediction_stream = model.respond_stream("What is the meaning of life?")
            cancelled = False
            for fragment in prediction_stream:
                if ...: # Cancellation condition will be app specific
                    cancelled = True
                    prediction_stream.cancel()
                    # Note: it is recommended to let the iteration complete,
                    # as doing so allows the partial result to be recorded.
                    # Breaking the loop *is* permitted, but means the partial result
                    # and final prediction stats won't be available to the client
            # The stream allows the prediction result to be retrieved after iteration
            if not cancelled:
                print(prediction_stream.result())

    "Python (asynchronous API)":
      language: python
      code: |
        # Note: assumes use of an async function or the "python -m asyncio" asynchronous REPL
        # Requires Python SDK version 1.5.0 or later
        import lmstudio as lms

        async with lms.AsyncClient() as client:
            model = await client.llm.model()

            prediction_stream = await model.respond_stream("What is the meaning of life?")
            cancelled = False
            async for fragment in prediction_stream:
                if ...: # Cancellation condition will be app specific
                    cancelled = True
                    await prediction_stream.cancel()
                    # Note: it is recommended to let the iteration complete,
                    # as doing so allows the partial result to be recorded.
                    # Breaking the loop *is* permitted, but means the partial result
                    # and final prediction stats won't be available to the client
            # The stream allows the prediction result to be retrieved after iteration
            if not cancelled:
                print(prediction_stream.result())

```


### Structured Response

> Enforce a structured response from the model using Pydantic models or JSON Schema


You can enforce a particular response format from an LLM by providing a JSON schema to the `.respond()` method.
This guarantees that the model's output conforms to the schema you provide.

The JSON schema can either be provided directly,
or by providing an object that implements the `lmstudio.ModelSchema` protocol,
such as `pydantic.BaseModel` or `lmstudio.BaseModel`.

The `lmstudio.ModelSchema` protocol is defined as follows:

```python
@runtime_checkable
class ModelSchema(Protocol):
    """Protocol for classes that provide a JSON schema for their model."""

    @classmethod
    def model_json_schema(cls) -> DictSchema:
        """Return a JSON schema dict describing this model."""
        ...

```

When a schema is provided, the prediction result's `parsed` field will contain a string-keyed dictionary that conforms
to the given schema (for unstructured results, this field is a string field containing the same value as `content`).


## Enforce Using a Class Based Schema Definition

If you wish the model to generate JSON that satisfies a given schema,
it is recommended to provide a class based schema definition using a library
such as [`pydantic`](https://docs.pydantic.dev/) or [`msgspec`](https://jcristharif.com/msgspec/).

Pydantic models natively implement the `lmstudio.ModelSchema` protocol,
while `lmstudio.BaseModel` is a `msgspec.Struct` subclass that implements `.model_json_schema()` appropriately.

#### Define a Class Based Schema

```lms_code_snippet
  variants:
    "pydantic.BaseModel":
      language: python
      code: |
        from pydantic import BaseModel

        # A class based schema for a book
        class BookSchema(BaseModel):
            title: str
            author: str
            year: int

    "lmstudio.BaseModel":
      language: python
      code: |
        from lmstudio import BaseModel

        # A class based schema for a book
        class BookSchema(BaseModel):
            title: str
            author: str
            year: int

```

#### Generate a Structured Response

```lms_code_snippet
  variants:
    "Non-streaming":
      language: python
      code: |
        result = model.respond("Tell me about The Hobbit", response_format=BookSchema)
        book = result.parsed

        print(book)
        #           ^
        # Note that `book` is correctly typed as { title: string, author: string, year: number }

    Streaming:
      language: python
      code: |
        prediction_stream = model.respond_stream("Tell me about The Hobbit", response_format=BookSchema)

        # Optionally stream the response
        # for fragment in prediction:
        #   print(fragment.content, end="", flush=True)
        # print()
        # Note that even for structured responses, the *fragment* contents are still only text

        # Get the final structured result
        result = prediction_stream.result()
        book = result.parsed

        print(book)
        #           ^
        # Note that `book` is correctly typed as { title: string, author: string, year: number }
```

## Enforce Using a JSON Schema

You can also enforce a structured response using a JSON schema.

#### Define a JSON Schema

```python