# Source: https://deepinfra.com/docs/advanced/langchain

We use essential cookies to make our site work. With your consent, we may also
use non-essential cookies to improve user experience and analyze website
traffic…

AcceptReject

[FLUX.2 is live!](https://deepinfra.com/models?q=flux-2) High-fidelity image
generation made simple.

[](/)

[Models](/models)

By Category

* * *

[Automatic Speech Recognition](/models/automatic-speech-
recognition)[Embeddings](/models/embeddings)[Reranker](/models/reranker)[Text
Generation](/models/text-generation)[Text To Image](/models/text-to-
image)[Text To Speech](/models/text-to-speech)[Text To Video](/models/text-to-
video)[Zero Shot Image Classification](/models/zero-shot-image-classification)

[View all models](/models)

By Family

* * *

[![anthropic
logo](/_next/static/media/anthropic.c6636fa8.svg)/Claude](/claude)[![deepseek-
ai
logo](/_next/static/media/deepseek.b1ec6c4e.svg)/DeepSeek](/deepseek)[![black-
forest-labs logo](/_next/static/media/bfl.7e050ff6.svg)/Flux](/flux)[![google
logo](/_next/static/media/google.09551b71.svg)/Gemini](/gemini)[![meta-llama
logo](/_next/static/media/meta.56a2e6fd.svg)/Llama](/llama)[![mistralai
logo](/_next/static/media/mistralai.ecbe51d4.svg)/Mistral](/mistral)[![nvidia
logo](/_next/static/media/nvidia.2165073d.svg)/Nemotron](/nemotron)[![qwen
logo](/_next/static/media/qwen.d6d74288.svg)/Qwen](/qwen)

[Docs](/docs)[Pricing](/pricing)[GPUs](/gpu-
instances)[Chat](/chat)[DeepStart](/deepstart)[Blog](/blog)

[Contact Sales](/contact-sales)[Log In](/login)

[Models](/models)

[Automatic Speech Recognition](/models/automatic-speech-
recognition)[Embeddings](/models/embeddings)[Reranker](/models/reranker)[Text
Generation](/models/text-generation)[Text To Image](/models/text-to-
image)[Text To Speech](/models/text-to-speech)[Text To Video](/models/text-to-
video)[Zero Shot Image Classification](/models/zero-shot-image-classification)

[Docs](/docs)

[Pricing](/pricing)

[GPUs](/gpu-instances)

[Chat](/chat)

[DeepStart](/deepstart)

[Blog](/blog)

Feedback

[Contact Sales](/contact-sales)

[Log In](/login)

  1. [Getting Started](/docs)

     1. [Quick Start Guide](/docs/getting-started)

     2. [Available Models](/docs/models)

     3. [Running Inference](/docs/inference)

     4. [Data Privacy & Security](/docs/data)

  2. [API Reference](/docs/api-reference)

     1. [OpenAI-Compatible API](/docs/openai_api)

     2. [DeepInfra Native API](/docs/deep_infra_api)

     3. [Rate Limits](/docs/advanced/rate-limits)

     4. [Webhooks](/docs/advanced/webhooks)

     5. [Authentication & Tokens](/docs/advanced/scoped_jwt)

  3. [Model Features](/docs/model-features)

     1. [Function Calling](/docs/advanced/function_calling)

     2. [JSON Mode](/docs/advanced/json_mode)

     3. [Multimodal Models](/docs/advanced/multimodal)

     4. [Log Probabilities](/docs/advanced/log_probs)

     5. [Max Output Tokens](/docs/advanced/max_tokens_limit)

  4. [GPU Instances](/docs/gpu-instances)

     1. [Containers](/docs/gpu-instances/containers)

  5. [Custom Deployments](/docs/custom-deployments)

     1. [Custom LLMs](/docs/advanced/custom_llms)

     2. [LoRA Adapter Models](/docs/advanced/lora)

     3. [LoRA Image Adapters](/docs/advanced/lora_text_to_image)

  6. [Integrations](/docs/integrations)

     1. LangChain

     2. [LlamaIndex](/docs/advanced/llama-index)

     3. [AI SDK](/docs/advanced/aisdk)

     4. [AutoGen](/docs/advanced/autogen)

     5. [Okta SSO](/docs/advanced/okta)

  7. [Tutorials & Examples](/docs/tutorials)

     1. [Stable Diffusion (Text to Image)](/docs/tutorials/stable-diffusion)

     2. [Whisper (Speech to Text)](/docs/tutorials/whisper)

     3. [Deprecated Models](/docs/advanced/deprecated)

  8. [Miscellaneous](/docs/misc)

     1. [Data Subprocessors](/docs/misc/subprocessors)

Documentation

  1. [Getting Started](/docs)

     1. [Quick Start Guide](/docs/getting-started)

     2. [Available Models](/docs/models)

     3. [Running Inference](/docs/inference)

     4. [Data Privacy & Security](/docs/data)

  2. [API Reference](/docs/api-reference)

     1. [OpenAI-Compatible API](/docs/openai_api)

     2. [DeepInfra Native API](/docs/deep_infra_api)

     3. [Rate Limits](/docs/advanced/rate-limits)

     4. [Webhooks](/docs/advanced/webhooks)

     5. [Authentication & Tokens](/docs/advanced/scoped_jwt)

  3. [Model Features](/docs/model-features)

     1. [Function Calling](/docs/advanced/function_calling)

     2. [JSON Mode](/docs/advanced/json_mode)

     3. [Multimodal Models](/docs/advanced/multimodal)

     4. [Log Probabilities](/docs/advanced/log_probs)

     5. [Max Output Tokens](/docs/advanced/max_tokens_limit)

  4. [GPU Instances](/docs/gpu-instances)

     1. [Containers](/docs/gpu-instances/containers)

  5. [Custom Deployments](/docs/custom-deployments)

     1. [Custom LLMs](/docs/advanced/custom_llms)

     2. [LoRA Adapter Models](/docs/advanced/lora)

     3. [LoRA Image Adapters](/docs/advanced/lora_text_to_image)

  6. [Integrations](/docs/integrations)

     1. LangChain

     2. [LlamaIndex](/docs/advanced/llama-index)

     3. [AI SDK](/docs/advanced/aisdk)

     4. [AutoGen](/docs/advanced/autogen)

     5. [Okta SSO](/docs/advanced/okta)

  7. [Tutorials & Examples](/docs/tutorials)

     1. [Stable Diffusion (Text to Image)](/docs/tutorials/stable-diffusion)

     2. [Whisper (Speech to Text)](/docs/tutorials/whisper)

     3. [Deprecated Models](/docs/advanced/deprecated)

  8. [Miscellaneous](/docs/misc)

     1. [Data Subprocessors](/docs/misc/subprocessors)

# LangChain

LangChain is a framework for developing applications powered by language
models. To learn more, visit the [LangChain
website](https://python.langchain.com/).

We offer the following modules:

  * [Chat adapter](https://python.langchain.com/docs/integrations/chat/deepinfra) for most of [our LLMs](/models/text-generation)
  * [LLM adapter](https://python.langchain.com/docs/integrations/llms/deepinfra) for most of [our LLMs](/models/text-generation)
  * [Embeddings adapter](https://python.langchain.com/docs/integrations/text_embedding/deepinfra) for all of [our Embeddings models](/models/embeddings)

# Install LangChain

    
    
    pip install langchain
    pip install langchain-community
    
    
    copy

# LLM Examples

The examples below show how to use LangChain with DeepInfra for language
models. Make sure to get your API key from DeepInfra. You have to
[Login](https://deepinfra.com/login?from=%2Fdash) and get your token.

Please set `os.environ["DEEPINFRA_API_TOKEN"]` with your token.

_Read comments in the code for better understanding._

    
    
    import os
    from langchain_community.llms import DeepInfra
    from langchain.prompts import PromptTemplate
    from langchain.chains import LLMChain
    
    # Make sure to get your API key from DeepInfra. You have to Login and get a new token.
    os.environ["DEEPINFRA_API_TOKEN"] = '<your DeepInfra API token>'
    
    # Create the DeepInfra instance. You can view a list of available parameters in the model page
    llm = DeepInfra(model_id="meta-llama/Meta-Llama-3-8B-Instruct")
    llm.model_kwargs = {
        "temperature": 0.7,
        "repetition_penalty": 1.2,
        "max_new_tokens": 250,
        "top_p": 0.9,
    }
    
    
    def example1():
        # run inference
        print(llm.invoke("Who let the dogs out?"))
    
    def example2():
        # run streaming inference
        for chunk in llm.stream("Who let the dogs out?"):
            print(chunk)
    
    def example3():
        # create a prompt template for Question and Answer
        template = """Question: {question}
    
        Answer: Let's think step by step."""
        prompt = PromptTemplate(template=template, input_variables=["question"])
    
        # initiate the chain
        llm_chain = prompt | llm
    
        # provide a question and run the LLMChain
        question = "Can penguins reach the North pole?"
        print(llm_chain.invoke(question))
    
    # run examples
    example1()
    
    
    copy

## Chat Examples

Ensure the `DEEPINFRA_API_KEY` env is set to your api key.

    
    
    import os
    
    # or pass deepinfra_api_token parameter to the ChatDeepInfra constructor
    os.environ["DEEPINFRA_API_TOKEN"] = DEEPINFRA_API_TOKEN
    
    from langchain_community.chat_models import ChatDeepInfra
    from langchain_core.messages import HumanMessage
    from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
    
    messages = [
        HumanMessage(
            content="Translate this sentence from English to French. I love programming."
        )
    ]
    
    def example_sync():
      chat = ChatDeepInfra(model="meta-llama/Meta-Llama-3-8B-Instruct")
      print(chat.invoke(messages))
    
    async def example_async():
      chat = ChatDeepInfra(model="meta-llama/Meta-Llama-3-8B-Instruct")
      await chat.agenerate([messages])
    
    def example_stream():
      chat = ChatDeepInfra(
          streaming=True,
          verbose=True,
          callbacks=[StreamingStdOutCallbackHandler()],
      )
      print(chat.invoke(messages))
    
    
    copy

## Embeddings

    
    
    import os
    
    os.environ["DEEPINFRA_API_TOKEN"] = DEEPINFRA_API_TOKEN
    
    from langchain_community.embeddings import DeepInfraEmbeddings
    
    embeddings = DeepInfraEmbeddings(
        model_id="sentence-transformers/clip-ViT-B-32",
        query_instruction="",
        embed_instruction="",
    )
    
    docs = ["Dog is not a cat", "Beta is the second letter of Greek alphabet"]
    document_result = embeddings.embed_documents(docs)
    print(document_result)
    
    
    copy

[Integrations](/docs/integrations)[LlamaIndex](/docs/advanced/llama-index)

![Footer Logo](/_next/static/media/footer_logo.b3e9d8d3.svg)

![SOC 2
Certified](https://static.sprinto.com/_next/static/images/framework/soc2.png)![ISO
27001
Certified](https://static.sprinto.com/_next/static/images/framework/iso-27001.png)

Have questions or need a custom solution?

[Contact Sales](/contact-sales)

Company

[Pricing](/pricing)

[Docs](/docs)

[Compare](/compare)

[DeepStart](/deepstart)

[About](/about_us)

[Careers](https://jobs.gem.com/deep-infra)

[Contact us](/contact-sales)

[Trust Center](https://trust.deepinfra.com)

[DeepGPT](https://deepgpt.com)

Latest Models

[anthropic/claude-3-7-sonnet-latest](/anthropic/claude-3-7-sonnet-
latest)[moonshotai/Kimi-K2-Instruct-0905](/moonshotai/Kimi-K2-Instruct-0905)[deepseek-
ai/DeepSeek-V3.1](/deepseek-ai/DeepSeek-V3.1)[zai-org/GLM-4.6](/zai-
org/GLM-4.6)[deepseek-ai/DeepSeek-V3.2-Exp](/deepseek-ai/DeepSeek-V3.2-Exp)

Featured Models

[anthropic/claude-4-sonnet](/anthropic/claude-4-sonnet)[meta-
llama/Llama-4-Scout-17B-16E-Instruct](/meta-
llama/Llama-4-Scout-17B-16E-Instruct)[nvidia/Nemotron-3-Nano-30B-A3B](/nvidia/Nemotron-3-Nano-30B-A3B)[anthropic/claude-4-opus](/anthropic/claude-4-opus)[deepseek-
ai/DeepSeek-R1-Distill-Llama-70B](/deepseek-ai/DeepSeek-R1-Distill-Llama-70B)

![Built With Love in Palo Alto](/_next/static/media/love.ce60156e.svg)

[](https://linkedin.com/company/deep-
infra)[](https://x.com/DeepInfra)[](https://github.com/DeepInfra)[](https://discord.gg/x88dCvhqYq)

© 2026 Deep Infra. All rights reserved.

[Privacy Policy](/privacy)[Terms of Service](/terms)