# Comet Docs Opik

> Documentation for Comet Docs Opik

## Pages

- [Comet Docs Opik Documentation](comet-docs-opik-documentation.md)
- [Home](home.md): Build, test, and optimize GenAI apps from prototype to production. Comprehensive tracing, evaluation, and prompt opti...
- [Quickstart](quickstart.md): Integrate Opik with your LLM application to log calls and chains efficiently. Get started with our step-by-step guide.
- [Quickstart notebook](quickstart-notebook.md): Learn to automate LLM workflow evaluation and track calls with Opik using Chain of Density Summarization for Arxiv pa...
- [Quickstart notebook - Summarization](quickstart-notebook-summarization.md): In this notebook, we will look at how you can use Opik to track your LLM calls, chains and agents. We will introduce ...
- [Configure Opik](configure-opik.md): opik.configure()
- [Use a dedicated quickstart endpoint, replace with your own OpenAI API Key in your own code](use-a-dedicated-quickstart-endpoint-replace-with-your-own-openai-api-key-in-your.md): openai_client = track_openai(
- [We will define the response format so the output has the correct schema. You can also use structured outputs with Pydantic models for this.](we-will-define-the-response-format-so-the-output-has-the-correct-schema-you-can-.md): json_schema = {
- [Custom Metric: One template/prompt to extract 4 scores/results](custom-metric-one-templateprompt-to-extract-4-scoresresults.md): class EvaluateSummary(base_metric.BaseMetric):
- [Load and extract text from PDFs](load-and-extract-text-from-pdfs.md): @opik.track
- [Roadmap](roadmap.md): Explore Opik's evolving roadmap and contribute ideas to enhance our open-source project as we develop new features an...
- [FAQ](faq.md): Explore common questions about Opik, learn how to get started, and find assistance to enhance your experience with ou...
- [January 27, 2026](january-27-2026.md): Here are the most relevant improvements we've made since the last release:
- [January 13, 2026](january-13-2026.md): Here are the most relevant improvements we've made since the last release:
- [December 18, 2025](december-18-2025.md): Here are the most relevant improvements we've made since the last release:
- [December 9, 2025](december-9-2025.md): Here are the most relevant improvements we've made since the last release:
- [November 18, 2025](november-18-2025.md): Here are the most relevant improvements we've made since the last release:
- [November 4, 2025](november-4-2025.md): Here are the most relevant improvements we've made since the last release:
- [October 21, 2025](october-21-2025.md): Here are the most relevant improvements we've made since the last release:
- [October 3, 2025](october-3-2025.md): Here are the most relevant improvements we've made since the last release:
- [September 5, 2025](september-5-2025.md): Here are the most relevant improvements we've made since the last release:
- [August 22, 2025](august-22-2025.md): Here are the most relevant improvements we've made in the last couple of weeks:
- [August 1, 2025](august-1-2025.md): We've expanded filtering and search capabilities to help you find and analyze data more effectively:
- [July 18, 2025](july-18-2025.md): We now support **thread-level LLMs-as-a-Judge metrics**!
- [July 4, 2025](july-4-2025.md): The Opik Agent Optimizer now supports full agentic systems and not just single prompts.
- [June 20, 2025](june-20-2025.md): * Added **CloudFlare's WorkersAI** integration ([docs](/docs/opik/integrations/cloudflare-workers-ai))
- [June 6, 2025](june-6-2025.md): * Ability to upload **CSV datasets** directly through the user interface
- [May 23, 2025](may-23-2025.md): * **Opik Agent Optimizer**: A comprehensive toolkit designed to enhance the performance and efficiency of your Large ...
- [May 5, 2025](may-5-2025.md): **Opik Dashboard**:
- [April 28, 2025](april-28-2025.md): **Opik Dashboard**:
- [April 21, 2025](april-21-2025.md): **Opik Dashboard**:
- [April 14, 2025](april-14-2025.md): **Opik Dashboard:**
- [April 7, 2025](april-7-2025.md): **Opik Dashboard:**
- [March 31, 2025](march-31-2025.md): **Opik Dashboard:**
- [March 24, 2025](march-24-2025.md): **General**
- [March 17, 2025](march-17-2025.md): **Opik Dashboard:**
- [March 10, 2025](march-10-2025.md): **Opik Dashboard:**
- [March 3, 2025](march-3-2025.md): **Opik Dashboard**:
- [February 24, 2025](february-24-2025.md): **Opik Dashboard**:
- [February 17, 2025](february-17-2025.md): **Opik Dashboard**:
- [February 10, 2025](february-10-2025.md): **Opik Dashboard**:
- [February 3, 2025](february-3-2025.md): **Opik Dashboard**:
- [January 27, 2025](january-27-2025.md): **Opik Dashboard**:
- [January 20, 2025](january-20-2025.md): **Opik Dashboard**:
- [January 13, 2025](january-13-2025.md): **Opik Dashboard**:
- [January 6, 2025](january-6-2025.md): **Opik Dashboard**:
- [December 30, 2024](december-30-2024.md): **Opik Dashboard**:
- [December 23, 2024](december-23-2024.md): **SDK**:
- [December 16, 2024](december-16-2024.md): **Opik Dashboard**:
- [December 9, 2024](december-9-2024.md): **Opik Dashboard**:
- [December 2, 2024](december-2-2024.md): **Opik Dashboard**:
- [November 25, 2024](november-25-2024.md): **Opik Dashboard**:
- [November 18, 2024](november-18-2024.md): **Opik Dashboard**:
- [November 11, 2024](november-11-2024.md): **Opik Dashboard**:
- [November 4, 2024](november-4-2024.md): **Opik Dashboard**:
- [October 21, 2024](october-21-2024.md): **Opik Dashboard**:
- [October 18, 2024](october-18-2024.md): **Opik Dashboard**:
- [October 14, 2024](october-14-2024.md): **Opik Dashboard**:
- [October 7, 2024](october-7-2024.md): **Opik Dashboard**:
- [September 30, 2024](september-30-2024.md): **Opik Dashboard**:
- [Tracing Core Concepts](tracing-core-concepts.md): Learn about the core concepts of Opik's tracing system, including traces, spans, threads, and how they work together ...
- [Log traces](log-traces.md): Monitor the flow of your LLM applications with tracing to identify issues and optimize performance using Opik's power...
- [Log conversations](log-conversations.md): Capture and analyze chat conversations on Opik, enabling better tracking of user interactions and improving chatbot p...
- [Log user feedback](log-user-feedback.md): Capture user feedback and track performance metrics to enhance your agent's effectiveness with Opik's logging capabil...
- [Log media & attachments](log-media-attachments.md): Capture multimodal traces in Opik by logging images, videos, and audio files using the Python SDK's Attachment type.
- [Example: Reading a file into memory and logging it](example-reading-a-file-into-memory-and-logging-it.md): with open("image.png", "rb") as f:
- [Analyze an image from a URL](analyze-an-image-from-a-url.md): result = analyze_remote_image("
- [Create a trace](create-a-trace.md): trace = client.trace(
- [Log attachment with file path](log-attachment-with-file-path.md): span_with_file = client.span(
- [Log attachment with raw bytes](log-attachment-with-raw-bytes.md): binary_data = b"Hello, this is binary content!"
- [Get list of attachments](get-list-of-attachments.md): attachments_details = attachment_client.get_attachment_list(
- [Download an attachment](download-an-attachment.md): attachment_data = attachment_client.download_attachment(
- [Upload a new attachment](upload-a-new-attachment.md): attachment_client.upload_attachment(
- [Make sure to wrap the OpenAI client to enable Opik tracing](make-sure-to-wrap-the-openai-client-to-enable-opik-tracing.md): client = track_openai(OpenAI())
- [Cost tracking](cost-tracking.md): Monitor and analyze cost patterns in your LLM apps using Opik's dashboard to identify anomalies and optimize spending.
- [Returns estimated cost in USD, or None for unsupported models](returns-estimated-cost-in-usd-or-none-for-unsupported-models.md): print(span.total_estimated_cost)
- [Returns estimated cost in USD, or None for unsupported models](returns-estimated-cost-in-usd-or-none-for-unsupported-models-2.md): print(trace.total_estimated_cost)
- [Define your own cost mapping for different models](define-your-own-cost-mapping-for-different-models.md): TOKEN_COST = {
- [This part would be custom for your use-case and is only here for example](this-part-would-be-custom-for-your-use-case-and-is-only-here-for-example.md): def compute_cost_for_span(span: SpanPublic):
- [Example usage in a CRON job](example-usage-in-a-cron-job.md): if __name__ == "__main__":
- [Export by SDK, REST, and UI](export-by-sdk-rest-and-ui.md): Learn to export traces and spans using Opik SDK, REST, and UI for enhanced model tuning and deeper analysis.
- [Import/Export by command line](importexport-by-command-line.md): Learn to export, import, migrate, and backup project data using Opik's command line tools efficiently.
- [Export specific dataset by exact name](export-specific-dataset-by-exact-name.md): opik export my-workspace dataset "my-test-dataset"
- [Export specific project by exact name](export-specific-project-by-exact-name.md): opik export my-workspace project "my-project" --filter "status = 'completed'"
- [Export specific experiment by exact name](export-specific-experiment-by-exact-name.md): opik export my-workspace experiment "my-experiment" --force
- [Export experiment with dataset filtering](export-experiment-with-dataset-filtering.md): opik export my-workspace experiment "my-experiment" --dataset "my-dataset"
- [Export experiment with trace limit](export-experiment-with-trace-limit.md): opik export my-workspace experiment "my-experiment" --max-traces 100
- [Export specific prompt by exact name](export-specific-prompt-by-exact-name.md): opik export my-workspace prompt "my-template"
- [Export with custom output directory](export-with-custom-output-directory.md): opik export my-workspace dataset "my-dataset" --path ./backup_data
- [Export using default directory (opik_exports)](export-using-default-directory-opik-exports.md): opik export my-workspace dataset "my-dataset"
- [Export with filter and limit](export-with-filter-and-limit.md): opik export my-workspace project "my-project" --filter "start_time >= '2024-01-01T00:00:00Z'" --max-results 100
- [Export traces in CSV format for analysis](export-traces-in-csv-format-for-analysis.md): opik export my-workspace project "my-project" --format csv --path ./csv_data
- [Export with debug output](export-with-debug-output.md): opik export my-workspace dataset "my-dataset" --debug --force
- [Export datasets in CSV format for analysis](export-datasets-in-csv-format-for-analysis.md): opik export my-workspace dataset "my-dataset" --format csv --path ./analysis_data
- [Export prompts in CSV format for analysis](export-prompts-in-csv-format-for-analysis.md): opik export my-workspace prompt "my-template" --format csv --path ./analysis_data
- [Export experiments in CSV format for analysis](export-experiments-in-csv-format-for-analysis.md): opik export my-workspace experiment "my-experiment" --format csv --path ./analysis_data
- [Import datasets from default directory (opik_exports)](import-datasets-from-default-directory-opik-exports.md): opik import my-workspace dataset "my-dataset"
- [Import projects from default directory](import-projects-from-default-directory.md): opik import my-workspace project "my-project"
- [Import experiments from default directory (automatically recreates experiments)](import-experiments-from-default-directory-automatically-recreates-experiments.md): opik import my-workspace experiment "my-experiment"
- [Import prompts from default directory](import-prompts-from-default-directory.md): opik import my-workspace prompt "my-prompt"
- [Import with name pattern matching](import-with-name-pattern-matching.md): opik import my-workspace dataset "test"
- [Import from custom path](import-from-custom-path.md): opik import my-workspace dataset "my-dataset" --path ./custom-exports
- [Dry run to see what would be imported](dry-run-to-see-what-would-be-imported.md): opik import my-workspace project "my-project" --dry-run
- [Import with debug output](import-with-debug-output.md): opik import my-workspace experiment "my-experiment" --debug
- [Export all data from source project](export-all-data-from-source-project.md): opik export my-workspace project "old-project" --path ./migration_data
- [Import to new workspace](import-to-new-workspace.md): opik import my-workspace project "old-project" --path ./migration_data
- [Create backup of specific data (requires knowing exact names)](create-backup-of-specific-data-requires-knowing-exact-names.md): opik export my-workspace dataset "my-dataset" --path ./backup_$(date +%Y%m%d)
- [Sync from staging to production](sync-from-staging-to-production.md): opik export my-workspace project "staging-project" --filter "tags contains 'ready-for-prod'"
- [Export specific traces for analysis](export-specific-traces-for-analysis.md): opik export my-workspace project "my-project" --filter "start_time >= '2024-01-01T00:00:00Z'" --max-results 1000
- [Analyze the JSON files locally](analyze-the-json-files-locally.md)
- [Export specific dataset from a workspace](export-specific-dataset-from-a-workspace.md): opik export my-workspace dataset "my-dataset"
- [Import datasets to another workspace (uses default opik_exports directory)](import-datasets-to-another-workspace-uses-default-opik-exports-directory.md): opik import my-workspace dataset "my-dataset"
- [Export traces in CSV format for analysis](export-traces-in-csv-format-for-analysis-2.md): opik export my-workspace project "my-project" --format csv --path ./analysis_data
- [Export datasets in CSV format for analysis](export-datasets-in-csv-format-for-analysis-2.md): opik export my-workspace dataset "my-dataset" --format csv --path ./analysis_data
- [Export experiments in CSV format for analysis](export-experiments-in-csv-format-for-analysis-2.md): opik export my-workspace experiment "my-experiment" --format csv --path ./analysis_data
- [Export prompts in CSV format for analysis](export-prompts-in-csv-format-for-analysis-2.md): opik export my-workspace prompt "my-template" --format csv --path ./analysis_data
- [df = pd.read_csv('./analysis_data/my-workspace/projects/my-project/traces_my-project.csv')](df-pdread-csvanalysis-datamy-workspaceprojectsmy-projecttraces-my-projectcsv.md)
- [Export specific prompt from a workspace](export-specific-prompt-from-a-workspace.md): opik export my-workspace prompt "my-template" --path ./prompt_backup
- [Export another prompt template](export-another-prompt-template.md): opik export my-workspace prompt "system-prompt" --path ./templates
- [Import prompts to another workspace](import-prompts-to-another-workspace.md): opik import my-workspace prompt "my-template" --path ./prompt_backup
- [Import with name pattern matching](import-with-name-pattern-matching-2.md): opik import my-workspace prompt "production" --path ./prompt_backup
- [Export specific experiment from source workspace](export-specific-experiment-from-source-workspace.md): opik export my-workspace experiment "my-experiment" --path ./experiment_data
- [Export experiment with dataset filtering](export-experiment-with-dataset-filtering-2.md): opik export my-workspace experiment "evaluation-exp" --dataset "test-dataset"
- [Export experiment with trace limit (useful for large experiments)](export-experiment-with-trace-limit-useful-for-large-experiments.md): opik export my-workspace experiment "large-experiment" --max-traces 50
- [Import experiments (automatically recreates experiments)](import-experiments-automatically-recreates-experiments.md): opik import my-workspace experiment "my-experiment" --path ./experiment_data
- [Import experiments matching name pattern](import-experiments-matching-name-pattern.md): opik import my-workspace experiment "evaluation" --path ./experiment_data
- [Check system health](check-system-health.md): opik healthcheck
- [1. Export specific data from source workspace (JSON format)](1-export-specific-data-from-source-workspace-json-format.md): opik export my-workspace dataset "my-dataset" --path ./temp_data
- [Alternative: Export experiment with specific dataset filtering](alternative-export-experiment-with-specific-dataset-filtering.md): opik export my-workspace experiment "evaluation-exp" --dataset "test-dataset" --max-traces 100 --path ./temp_data
- [2. Inspect the exported data](2-inspect-the-exported-data.md): ls ./temp_data/my-workspace/datasets/
- [3. Dry run import to see what would be imported](3-dry-run-import-to-see-what-would-be-imported.md): opik import my-workspace dataset "my-dataset" --path ./temp_data --dry-run
- [4. Actually import all data including experiments and prompts](4-actually-import-all-data-including-experiments-and-prompts.md): opik import my-workspace dataset "my-dataset" --path ./temp_data
- [5. Clean up temporary data](5-clean-up-temporary-data.md): rm -rf ./temp_data
- [1. Export data in CSV format for analysis](1-export-data-in-csv-format-for-analysis.md): opik export my-workspace project "my-source-project" --format csv --path ./csv_data
- [2. Inspect the CSV files](2-inspect-the-csv-files.md): ls ./csv_data/my-workspace/projects/my-source-project/
- [Each trace is exported to its own CSV file](each-trace-is-exported-to-its-own-csv-file.md): head -5 ./csv_data/my-workspace/projects/my-source-project/trace_*.csv | head -20
- [3. Analyze with pandas (optional)](3-analyze-with-pandas-optional.md): python -c "
- [Read all trace CSV files and combine them](read-all-trace-csv-files-and-combine-them.md): trace_files = glob.glob('./csv_data/my-workspace/projects/my-source-project/trace_*.csv')
- [4. For import, you would need to convert back to JSON format](4-for-import-you-would-need-to-convert-back-to-json-format.md)
- [(CSV format is primarily for analysis, not import)](csv-format-is-primarily-for-analysis-not-import.md): This workflow ensures you can safely migrate all data including experiments and prompts between workspaces while main...
- [✨ Opik Assist (Beta)](opik-assist-beta.md): AI-powered trace analysis and debugging assistant for your LLM applications
- [SDK configuration](sdk-configuration.md): Configure Python and TypeScript SDKs effectively. Set up your API key and instance URL for seamless routing and authe...
- [SDK Configuration](sdk-configuration-2.md): This guide covers configuration for both Python and TypeScript SDKs, including basic setup, advanced options, and deb...
- [Opik Configuration](opik-configuration.md): OPIK_API_KEY="YOUR_OPIK_API_KEY"
- [LLM Provider API Keys (if needed)](llm-provider-api-keys-if-needed.md): OPENAI_API_KEY="YOUR_OPENAI_API_KEY"
- [Logging Configuration (see Debug Mode and Logging section below)](logging-configuration-see-debug-mode-and-logging-section-below.md): OPIK_CONSOLE_LOGGING_LEVEL="WARNING" # Python: Control console output (DEBUG, INFO, WARNING, ERROR, CRITICAL)
- [Your Opik code here](your-opik-code-here.md): **TypeScript usage with .env file:**
- [Reduce console output to warnings and errors only](reduce-console-output-to-warnings-and-errors-only.md): export OPIK_CONSOLE_LOGGING_LEVEL="WARNING"
- [Console Logging (reduce noise)](console-logging-reduce-noise.md): OPIK_CONSOLE_LOGGING_LEVEL="WARNING"
- [File Logging (for debug)](file-logging-for-debug.md): OPIK_FILE_LOGGING_LEVEL="DEBUG"
- [Console: Show only warnings and errors](console-show-only-warnings-and-errors.md): OPIK_CONSOLE_LOGGING_LEVEL="WARNING"
- [File: Log everything for debugging](file-log-everything-for-debugging.md): OPIK_FILE_LOGGING_LEVEL="DEBUG"
- [Your Opik code here - console will be quiet, debug logs go to file](your-opik-code-here-console-will-be-quiet-debug-logs-go-to-file.md): The TypeScript SDK uses structured logging with configurable levels:
- [Use the Opik SDK as usual](use-the-opik-sdk-as-usual.md): The TypeScript SDK uses batching for optimal performance. You can configure batching behavior:
- [Log Agent Graphs](log-agent-graphs.md): Learn to log agent execution graphs in Opik for enhanced debugging and flow visualization across frameworks like Lang...
- [Log distributed traces](log-distributed-traces.md): Learn to track distributed traces in complex LLM applications using Opik's built-in support for multi-service tracing.
- [Overview](overview.md): Evaluate the quality of LLM outputs using Opik's framework to systematically test prompts and models against datasets...
- [Create a dataset that contains the samples you want to evaluate](create-a-dataset-that-contains-the-samples-you-want-to-evaluate.md): opik_client = opik.Opik()
- [Run the evaluation](run-the-evaluation.md): result = evaluate_prompt(
- [Retrieve and print the aggregated scores statistics (mean, min, max, std) per metric](retrieve-and-print-the-aggregated-scores-statistics-mean-min-max-std-per-metric.md): scores = result.aggregate_evaluation_scores()
- [Concepts](concepts.md): Learn to streamline evaluation of LLM applications with Opik for faster iterations and enhanced output review.
- [Manage datasets](manage-datasets.md): Evaluate your LLM using datasets. Learn to create and manage them via Python SDK, TypeScript SDK, or the Traces table.
- [Evaluate single prompts](evaluate-single-prompts.md): Evaluate and compare prompt performance in Opik using the playground or Python SDK to enhance your prompt engineering...
- [Evaluate your agent](evaluate-your-agent.md): Evaluate your LLM applications confidently. Learn the five steps to assess complex LLM chains or agents effectively.
- [Extra dependencies you want to provide to the task](extra-dependencies-you-want-to-provide-to-the-task.md): MODEL = "gpt-4o"
- [Evaluate agent trajectories](evaluate-agent-trajectories.md): Evaluate agent trajectories to optimize tool selection and reasoning paths, ensuring efficient agent behavior before ...
- [Run the evaluation](run-the-evaluation-2.md): experiment = evaluate(
- [Evaluate multimodal traces](evaluate-multimodal-traces.md): Evaluate multimodal prompts with Opik. Learn to run experiments in the UI or SDK, customize models, and understand im...
- [Directly using LangChain message objects](directly-using-langchain-message-objects.md): plain_messages = [
- [Using a ChatPromptTemplate with multimodal content](using-a-chatprompttemplate-with-multimodal-content.md): chat_prompt = ChatPromptTemplate.from_messages([
- [Evaluate multi-turn agents](evaluate-multi-turn-agents.md): Learn to evaluate multi-turn agents using simulation techniques to enhance chatbot performance and improve user inter...
- [Generate a user message that will start the conversation](generate-a-user-message-that-will-start-the-conversation.md): print(user_simulator.generate_response([
- [Generate a user message based on a couple of back and forth turns](generate-a-user-message-based-on-a-couple-of-back-and-forth-turns.md): print(user_simulator.generate_response([
- [Manually logging experiments](manually-logging-experiments.md): Evaluate your LLM application by logging pre-computed experiments and boosting confidence in performance with this de...
- [No authentication headers required for local deployments](no-authentication-headers-required-for-local-deployments.md): curl -X PUT ' \
- [Re-running an existing experiment](re-running-an-existing-experiment.md): Learn how to update existing experiments in Opik by changing names, configurations, and scores effectively.
- [Update experiment name](update-experiment-name.md): client.update_experiment(
- [Update experiment configuration](update-experiment-configuration.md): client.update_experiment(
- [Update both name and configuration](update-both-name-and-configuration.md): client.update_experiment(
- [Replace "my-experiment" with the name of your experiment which can be found in the Opik UI](replace-my-experiment-with-the-name-of-your-experiment-which-can-be-found-in-the.md): evaluate_experiment(experiment_name="my-experiment", scoring_metrics=[hallucination_metric])
- [Define the task to evaluate](define-the-task-to-evaluate.md): openai_client = track_openai(openai.OpenAI())
- [Define the evaluation task](define-the-evaluation-task.md): def evaluation_task(x):
- [Create a simple dataset](create-a-simple-dataset.md): client = Opik()
- [Define the metrics](define-the-metrics.md): hallucination_metric = Hallucination()
- [Annotation Queues](annotation-queues.md): Optimize your AI projects by enabling SMEs to efficiently review and annotate outputs using Opik's intuitive Annotati...
- [Overview](overview-2.md): Describes all the built-in evaluation metrics provided by Opik
- [Overview](overview-3.md): Opik provides a set of built-in evaluation metrics that you can mix and match to evaluate LLM behaviour. These metric...
- [Heuristic metrics](heuristic-metrics.md): Describes all the built-in heuristic metrics provided by Opik
- [Single reference](single-reference.md): score = metric.score(
- [Multiple references](multiple-references.md): score = metric.score(
- [Analyze sentiment](analyze-sentiment.md): score = metric.score(output="I love this product! It's amazing.")
- [Negative sentiment example](negative-sentiment-example.md): score = metric.score(output="This is terrible, I hate it.")
- [Single reference](single-reference-2.md): score = metric.score(
- [Multiple references](multiple-references-2.md): score = metric.score(
- [Hallucination](hallucination.md): Describes the Hallucination metric
- [LLM Juries](llm-juries.md): Combine multiple judges into an ensemble with LLMJuriesJudge
- [LLM Juries Judge](llm-juries-judge.md): `LLMJuriesJudge`averages the results of multiple judge metrics to deliver a single ensemble score. It is useful when...
- [G-Eval](g-eval.md): Describes Opik's built-in G-Eval metric which is a task agnostic LLM as a Judge metric
- [Compliance risk](compliance-risk.md): Flag non-compliant or high-risk assistant replies with ComplianceRiskJudge
- [Compliance Risk Judge](compliance-risk-judge.md): `ComplianceRiskJudge`inspects an assistant response for regulatory, legal, or policy issues. It builds on Opik's GEv...
- [Prompt uncertainty](prompt-uncertainty.md): Estimate prompt ambiguity with PromptUncertaintyJudge
- [Prompt Uncertainty](prompt-uncertainty-2.md): Prompt uncertainty scoring helps you triage risky or underspecified user requests before they reach your production m...
- [Moderation](moderation.md): Describes the Moderation metric
- [Meaning Match](meaning-match.md): Describes the Meaning Match metric
- [Meaning Match](meaning-match-2.md): The Meaning Match metric evaluates whether an LLM's output semantically matches a ground truth answer, regardless of ...
- [Usefulness](usefulness.md): The usefulness metric allows you to evaluate how useful an LLM response is given an input. It uses a language model t...
- [Summarization consistency](summarization-consistency.md): Ensure autogenerated summaries stay faithful to the source content
- [Summarization Consistency Judge](summarization-consistency-judge.md): `SummarizationConsistencyJudge`compares a generated summary with the original document (or transcript) and scores ho...
- [Summarization coherence](summarization-coherence.md): Rate how readable and well-structured a summary is with SummarizationCoherenceJudge
- [Summarization Coherence Judge](summarization-coherence-judge.md): `SummarizationCoherenceJudge`evaluates the writing quality of a summary: structure, clarity, and logical flow. It co...
- [Dialogue helpfulness](dialogue-helpfulness.md): Measure how helpful an assistant reply is within a dialogue
- [Dialogue Helpfulness Judge](dialogue-helpfulness-judge.md): `DialogueHelpfulnessJudge`inspects the latest assistant reply in the context of preceding turns. It rewards response...
- [Answer relevance](answer-relevance.md): Describes the Answer Relevance metric
- [Context precision](context-precision.md): Describes the Context Precision metric
- [Context recall](context-recall.md): Describes the Context Recall metric
- [Trajectory accuracy](trajectory-accuracy.md): Score whether an agent followed the expected action path
- [Trajectory Accuracy](trajectory-accuracy-2.md): `TrajectoryAccuracy`checks how closely a ReAct-style agent followed a sensible sequence of thoughts, actions, and ob...
- [Agent task completion](agent-task-completion.md): Verify whether an agent fulfilled its assigned objective
- [Agent Task Completion Judge](agent-task-completion-judge.md): `AgentTaskCompletionJudge`reviews an agent run (often a natural-language summary of what happened) and decides wheth...
- [Agent tool correctness](agent-tool-correctness.md): Evaluate whether an agent invoked and interpreted tools correctly
- [Agent Tool Correctness Judge](agent-tool-correctness-judge.md): `AgentToolCorrectnessJudge`checks if an agent called the right tools with valid arguments and interpreted the output...
- [Conversational metrics](conversational-metrics.md): Describes metrics related to scoring the conversational threads
- [Custom model](custom-model.md): Describes how to use a custom model for Opik's built-in LLM as a Judge metrics
- [Ensure these are set securely, e.g., via environment variables](ensure-these-are-set-securely-eg-via-environment-variables.md): API_KEY = os.getenv("MY_CUSTOM_LLM_API_KEY")
- [Initialize your custom model](initialize-your-custom-model.md): my_custom_model = CustomOpenAICompatibleModel(
- [Initialize the Hallucination metric with the custom model](initialize-the-hallucination-metric-with-the-custom-model.md): hallucination_metric = Hallucination(
- [Example usage:](example-usage.md): evaluation = hallucination_metric.score(
- [Advanced configuration](advanced-configuration.md): Fine-tune Opik metrics with async scoring, evaluator temperatures, and logprob handling
- [Advanced configuration](advanced-configuration-2.md): Opik’s metrics expose several power-user controls so you can tailor evaluations to your workflows. This guide covers ...
- [Custom metric](custom-metric.md): Describes how to create your own metric to use with Opik's evaluation platform
- [Custom Metric](custom-metric-2.md): Opik allows you to define your own custom metrics, which is especially important when the metrics you need are not al...
- [Custom conversation metric](custom-conversation-metric.md): Learn how to create custom metrics for evaluating multi-turn conversations
- [Custom Conversation (Multi-turn) Metrics](custom-conversation-multi-turn-metrics.md): Conversation metrics evaluate multi-turn conversations rather than single input-output pairs. These metrics are parti...
- [Type definition](type-definition.md): ConversationDict = Dict[Literal["role", "content"], str]
- [Example conversation](example-conversation.md): conversation = [
- [Initialize the metric with your preferred judge model](initialize-the-metric-with-your-preferred-judge-model.md): quality_metric = ConversationQualityMetric(
- [Evaluate threads in your project](evaluate-threads-in-your-project.md): results = evaluate_threads(
- [Initialize your metrics](initialize-your-metrics.md): conversation_length_metric = ConversationLengthMetric()
- [Evaluate threads in your project](evaluate-threads-in-your-project-2.md): results = evaluate_threads(
- [Task span metrics](task-span-metrics.md): Learn how to create task span metrics for evaluating the detailed execution information of your LLM tasks
- [Task Span Metrics](task-span-metrics-2.md): Task span metrics are a powerful type of evaluation metric in Opik that can analyze the detailed execution informatio...
- [Mix regular and task span metrics](mix-regular-and-task-span-metrics.md): equals_metric = Equals()
- [Example metric under test](example-metric-under-test.md): class ExecutionTimeMetric:
- [Good: Analyzing execution patterns](good-analyzing-execution-patterns.md): def _analyze_caching_efficiency_recursively(self, span: SpanModel, cache_stats: Dict[str, Any] = None) -> Dict[str, A...
- [Comprehensive evaluation approach](comprehensive-evaluation-approach.md): scoring_metrics = [
- [Best practices for evaluating agents](best-practices-for-evaluating-agents.md): Learn best practices for evaluating AI agents, ensuring reliability and scalability throughout their lifecycle with O...
- [This is the dataset with the examples of good tool selection](this-is-the-dataset-with-the-examples-of-good-tool-selection.md): dataset = client.get_dataset(name="Your_Dataset")
- [Define the items](define-the-items.md): items = [
- [Insert the dataset items](insert-the-dataset-items.md): dataset.insert(items)
- [This function defines how each item in the dataset will be evaluated.](this-function-defines-how-each-item-in-the-dataset-will-be-evaluated.md)
- [- It sends the `input` as a message to the agent (`agent_executor`).](it-sends-the-input-as-a-message-to-the-agent-agent-executor.md)
- [- It captures the agent's actual tool calls from its outputs.](it-captures-the-agents-actual-tool-calls-from-its-outputs.md)
- [- It packages the original input, the agent's outputs, the detected tool calls, and the expected tool calls.](it-packages-the-original-input-the-agents-outputs-the-detected-tool-calls-and-th.md)
- [This structured output is what the evaluation platform will use to compare expected vs actual behavior using the custom metric(s) you define.](this-structured-output-is-what-the-evaluation-platform-will-use-to-compare-expec.md): def evaluation_task(dataset_item):
- [This is the custom metric we have defined](this-is-the-custom-metric-we-have-defined.md): metrics = [ToolSelectionQuality()]
- [It loops over each dataset item and applies the `evaluation_task` function to generate outputs.](it-loops-over-each-dataset-item-and-applies-the-evaluation-task-function-to-gene.md)
- [It then applies the custom `ToolSelectionQuality` metric (or any provided metrics) to score each result.](it-then-applies-the-custom-toolselectionquality-metric-or-any-provided-metrics-t.md)
- [It logs the evaluation results to Opik under the specified experiment name ("AgentToolSelectionExperiment").](it-logs-the-evaluation-results-to-opik-under-the-specified-experiment-name-agent.md)
- [This allows tracking, comparing, and analyzing your agent's tool selection quality over time in Opik.](this-allows-tracking-comparing-and-analyzing-your-agents-tool-selection-quality-.md): eval_results = evaluate(
- [Evaluate threads](evaluate-threads.md): Evaluate and optimize conversation threads in Opik using the evaluate_threads function in the Python SDK for enhanced...
- [Initialize the evaluation metrics](initialize-the-evaluation-metrics.md): conversation_coherence_metric = ConversationalCoherenceMetric()
- [Run the threads evaluation](run-the-threads-evaluation.md): results = evaluate_threads(
- [Cookbook - Evaluate hallucination metric](cookbook-evaluate-hallucination-metric.md): Learn to evaluate the Hallucination metric using the LLM Evaluation SDK and improve your model's performance with Opik.
- [Evaluating Opik's Hallucination Metric](evaluating-opiks-hallucination-metric.md): For this guide we will be evaluating the Hallucination metric included in the LLM Evaluation SDK which will showcase ...
- [Create dataset](create-dataset.md): import opik
- [Create dataset](create-dataset-2.md): dataset = client.get_or_create_dataset(name="HaluEval", description="HaluEval dataset")
- [Insert items into dataset](insert-items-into-dataset.md): df = pd.read_parquet(
- [Define the evaluation task](define-the-evaluation-task-2.md): def evaluation_task(x: Dict):
- [Get the dataset](get-the-dataset.md): client = Opik()
- [Define the scoring metric](define-the-scoring-metric.md): check_hallucinated_metric = Equals(name="Correct hallucination score")
- [Add the prompt template as an experiment configuration](add-the-prompt-template-as-an-experiment-configuration.md): experiment_config = {
- [Cookbook - Evaluate moderation metric](cookbook-evaluate-moderation-metric.md): Evaluate the Moderation metric in the LLM Evaluation SDK to enhance your moderation capabilities using Opik's platform.
- [Evaluating Opik's Moderation Metric](evaluating-opiks-moderation-metric.md): For this guide we will be evaluating the Moderation metric included in the LLM Evaluation SDK which will showcase bot...
- [Create dataset](create-dataset-3.md): import opik
- [Create dataset](create-dataset-4.md): dataset = client.get_or_create_dataset(
- [Insert items into dataset](insert-items-into-dataset-2.md): url = "
- [Define the evaluation task](define-the-evaluation-task-3.md): def evaluation_task(x: Dict):
- [Get the dataset](get-the-dataset-2.md): client = Opik()
- [Define the scoring metric](define-the-scoring-metric-2.md): moderation_metric = Equals(name="Correct moderation score")
- [Add the prompt template as an experiment configuration](add-the-prompt-template-as-an-experiment-configuration-2.md): experiment_config = {
- [Prompt management](prompt-management.md): Learn to utilize Opik's prompt library for versioning, reusing, and managing your prompts centrally across projects.
- [Create a chat prompt](create-a-chat-prompt.md): chat_prompt = opik.ChatPrompt(
- [Attempting to create a text prompt with the same name will raise an error](attempting-to-create-a-text-prompt-with-the-same-name-will-raise-an-error.md): try:
- [Prompt Playground](prompt-playground.md): Experiment with LLM prompts in Opik's playground to evaluate performance and enhance your prompt engineering skills.
- [Prompt Generator and Improver](prompt-generator-and-improver.md): Enhance your prompt quality using Opik's AI tools for efficient prompt generation and refinement based on expert engi...
- [Opik's MCP server](opiks-mcp-server.md): Configure Opik's MCP server with Cursor IDE to manage prompts and analyze traces efficiently.
- [Pytest integration](pytest-integration.md): Monitor your LLM applications' performance by using Opik's Pytest integration to track test results and ensure reliab...
- [Production monitoring](production-monitoring.md): Monitor your production LLM applications with Opik. Evaluate feedback scores, trace counts, and token usage daily and...
- [Online Evaluation rules](online-evaluation-rules.md): Evaluate production LLM performance with Opik's automated metrics to efficiently score traces and identify issues.
- [Gateway](gateway.md): Build a centralized LLM gateway with Opik to streamline access to multiple LLM providers using a consistent API format.
- [Guardrails](guardrails.md): Monitor your LLM calls with guardrails in Opik to prevent risks and ensure safe application outputs.
- [How it works](how-it-works.md): Conceptually, we need to determine the presence of a series of risks for each input and
- [Types of guardrails](types-of-guardrails.md): Providers like OpenAI or Anthropic have built-in guardrails for risks like harmful or
- [Brand mention detection](brand-mention-detection.md): competitor_brands = [
- [Getting started](getting-started.md): You can start the guardrails backend by running:
- [Anonymizers](anonymizers.md): Protect sensitive information in your LLM applications with Opik's Anonymizers, ensuring compliance and preventing ac...
- [How it works](how-it-works-2.md): Anonymizers work by processing all data that flows through Opik's tracing system - including inputs, outputs, and met...
- [Types of Anonymizers](types-of-anonymizers.md): The most common type of anonymizer uses pattern-matching rules to identify and replace sensitive information. Rules c...
- [Dictionary format](dictionary-format.md): email_rule = {"regex": r"\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b", "replace": "[EMAIL]"}
- [Tuple format](tuple-format.md): phone_rule = (r"\b\d{3}-\d{3}-\d{4}\b", "[PHONE]")
- [Create anonymizer with multiple rules](create-anonymizer-with-multiple-rules.md): anonymizer = create_anonymizer([email_rule, phone_rule])
- [Register globally](register-globally.md): opik.hooks.add_anonymizer(anonymizer)
- [Create anonymizer with function rules](create-anonymizer-with-function-rules.md): anonymizer = create_anonymizer([mask_api_keys, anonymize_with_hash])
- [Mix of dictionary, tuple, and function rules](mix-of-dictionary-tuple-and-function-rules.md): mixed_rules = [
- [Register the field-aware anonymizer](register-the-field-aware-anonymizer.md): opik.hooks.add_anonymizer(FieldAwareAnonymizer())
- [Register the custom anonymizer](register-the-custom-anonymizer.md): opik.hooks.add_anonymizer(AdvancedPIIAnonymizer())
- [Usage Examples](usage-examples.md): Here's a complete example showing how to set up anonymization for a simple LLM application:
- [Define PII anonymization rules](define-pii-anonymization-rules.md): pii_rules = [
- [Create and register anonymizer](create-and-register-anonymizer.md): anonymizer = create_anonymizer(pii_rules)
- [Now all traced functions will automatically anonymize PII](now-all-traced-functions-will-automatically-anonymize-pii.md): @opik.track
- [Example usage - PII will be automatically anonymized in traces](example-usage-pii-will-be-automatically-anonymized-in-traces.md): result = process_customer_data("John Doe, email: john@example.com, phone: 555-123-4567")
- [Set up multi-layer anonymization](set-up-multi-layer-anonymization.md): opik.hooks.clear_anonymizers() # Clear any existing anonymizers
- [Layer 1: Basic PII patterns](layer-1-basic-pii-patterns.md): basic_rules = [
- [Layer 2: Compliance-specific anonymization](layer-2-compliance-specific-anonymization.md): opik.hooks.add_anonymizer(ComplianceAnonymizer(compliance_level="standard"))
- [Layer 3: Custom business logic](layer-3-custom-business-logic.md): def remove_internal_identifiers(text):
- [Register the Presidio-based anonymizer globally](register-the-presidio-based-anonymizer-globally.md): opik.hooks.add_anonymizer(PresidioPIIAnonymizer())
- [Set up anonymization](set-up-anonymization.md): pii_rules = [
- [Enable OpenAI tracking with automatic anonymization](enable-openai-tracking-with-automatic-anonymization.md): client = track_openai(openai.OpenAI())
- [PII in prompts will be automatically anonymized in traces](pii-in-prompts-will-be-automatically-anonymized-in-traces.md): response = client.chat.completions.create(
- [Configure anonymization - mix regex and callable function](configure-anonymization-mix-regex-and-callable-function.md): def mask_credit_cards(text: str) -> str:
- [Set up LangChain with Opik tracing](set-up-langchain-with-opik-tracing.md): llm = ChatOpenAI(callbacks=[OpikTracer()])
- [All inputs and outputs will be automatically anonymized](all-inputs-and-outputs-will-be-automatically-anonymized.md): messages = [HumanMessage(content="Contact sarah@example.com about card 4532-1234-5678-9010")]
- [Configuration Options](configuration-options.md): Control how deeply nested data structures are processed:
- [Default max_depth is 10](default-max-depth-is-10.md): anonymizer = create_anonymizer(rules, max_depth=5)
- [Clear existing anonymizers](clear-existing-anonymizers.md): opik.hooks.clear_anonymizers()
- [Add multiple anonymizers in order](add-multiple-anonymizers-in-order.md): opik.hooks.add_anonymizer(create_anonymizer([
- [Check if any anonymizers are registered](check-if-any-anonymizers-are-registered.md): if opik.hooks.has_anonymizers():
- [Best Practices](best-practices.md): Rules are applied in the order they're defined. More specific patterns should come before general ones:
- [Pre-compile regex for better performance](pre-compile-regex-for-better-performance.md): EMAIL_PATTERN = re.compile(r"\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b")
- [Define your rules](define-your-rules.md): rules = [
- [Test with sample data](test-with-sample-data.md): test_data = "Contact John at john.doe@company.com or call 555-123-4567"
- [Test with nested data](test-with-nested-data.md): test_nested = {
- [Troubleshooting](troubleshooting.md): **Anonymizer not working:**
- [Security Considerations](security-considerations.md): * **Test thoroughly**: Always test anonymization rules with representative data
- [Alerts](alerts.md): Configure automated webhook notifications to stay informed about events in your Opik workspace, from trace errors to ...
- [In-memory storage (use a database in production)](in-memory-storage-use-a-database-in-production.md): alert_history = []
- [In your webhook handler](in-your-webhook-handler.md): api_key = request.headers.get('X-API-Key')
- [Route to different handlers based on environment](route-to-different-handlers-based-on-environment.md): if environment == 'production':
- [Dashboards](dashboards.md): Create customizable dashboards to monitor quality, cost, and performance of your LLM projects and visualize experimen...
- [Feedback Definitions](feedback-definitions.md): Configure custom feedback types for evaluating LLM outputs
- [AI Providers](ai-providers.md): Configure connections to Large Language Model providers
- [Contribution Overview](contribution-overview.md): Contribute to Opik by submitting bug reports, suggesting features, or improving documentation to enhance the project ...
- [Contributing to Opik](contributing-to-opik.md): We're excited that you're interested in contributing to Opik! There are many ways to contribute, from writing code to...
- [Documentation](documentation.md): Learn how to contribute effectively to Opik's documentation, including guidelines and structure for seamless collabor...
- [Contributing to Documentation](contributing-to-documentation.md): This guide will help you get started with contributing to Opik's documentation.
- [Local Development Setup](local-development-setup.md): Comprehensive guide for setting up and running Opik locally for development
- [Check Docker](check-docker.md): docker --version
- [Check Node.js and npm](check-nodejs-and-npm.md): node --version
- [Check Python](check-python.md): python --version
- [Build and start all services (recommended for first time)](build-and-start-all-services-recommended-for-first-time.md): ./opik.sh --build
- [Start without rebuilding (if no code changes)](start-without-rebuilding-if-no-code-changes.md): ./opik.sh
- [Enable port mapping (useful for debugging)](enable-port-mapping-useful-for-debugging.md): ./opik.sh --build --port-mapping
- [Enable debug logging](enable-debug-logging.md): ./opik.sh --build --debug
- [Infrastructure only (MySQL, Redis, ClickHouse, ZooKeeper, MinIO)](infrastructure-only-mysql-redis-clickhouse-zookeeper-minio.md): ./opik.sh --infra --port-mapping
- [Infrastructure + Backend services](infrastructure-backend-services.md): ./opik.sh --backend --port-mapping
- [All services EXCEPT backend (for local backend development)](all-services-except-backend-for-local-backend-development.md): ./opik.sh --local-be --port-mapping
- [Add guardrails services](add-guardrails-services.md): ./opik.sh --build --guardrails
- [Rebuild specific service](rebuild-specific-service.md): docker compose -f deployment/docker-compose/docker-compose.yaml build backend
- [Full restart (stop, build, start) - DEFAULT](full-restart-stop-build-start-default.md): scripts/dev-runner.sh
- [Start without rebuilding (faster if no dependency changes)](start-without-rebuilding-faster-if-no-dependency-changes.md): scripts/dev-runner.sh --start
- [View logs](view-logs.md): scripts/dev-runner.sh --logs
- [Enable verbose logging](enable-verbose-logging.md): scripts/dev-runner.sh --restart --debug
- [Or set environment variable](or-set-environment-variable.md): DEBUG_MODE=true scripts/dev-runner.sh --restart
- [Change from:](change-from.md): url_override =
- [To:](to.md): url_override =
- [Full restart (stop, build backend, start)](full-restart-stop-build-backend-start.md): scripts/dev-runner.sh --be-only-restart
- [Start without rebuilding](start-without-rebuilding.md): scripts/dev-runner.sh --be-only-start
- [Stop services](stop-services.md): scripts/dev-runner.sh --be-only-stop
- [Check status](check-status.md): scripts/dev-runner.sh --be-only-verify
- [Use URL: http://localhost:5173](use-url-httplocalhost5173.md): Or with environment variables:
- [Start only infrastructure services](start-only-infrastructure-services.md): ./opik.sh --infra --port-mapping
- [Verify infrastructure is running](verify-infrastructure-is-running.md): ./opik.sh --infra --verify
- [Stop infrastructure](stop-infrastructure.md): ./opik.sh --infra --stop
- [Different profiles](different-profiles.md): .\opik.ps1 --infra --port-mapping
- [Manage services](manage-services.md): .\opik.ps1 --verify
- [Specific commands](specific-commands.md): scripts\dev-runner.ps1 --restart
- [BE-only mode](be-only-mode.md): scripts\dev-runner.ps1 --be-only-restart
- [Debug mode](debug-mode.md): scripts\dev-runner.ps1 --restart --debug
- [Build backend only](build-backend-only.md): scripts/dev-runner.sh --build-be
- [Build frontend only](build-frontend-only.md): scripts/dev-runner.sh --build-fe
- [Lint frontend](lint-frontend.md): scripts/dev-runner.sh --lint-fe
- [Run migrations only](run-migrations-only.md): scripts/dev-runner.sh --migrate
- [4. Run ClickHouse migrations](4-run-clickhouse-migrations.md): If migrations fail, you may need to clean up:
- [Stop all services](stop-all-services.md): scripts/dev-runner.sh --stop # or ./opik.sh --stop
- [Remove Opik Docker volumes (WARNING: DATA LOSS - removes Opik databases)](remove-opik-docker-volumes-warning-data-loss-removes-opik-databases.md): ./opik.sh --clean
- [Restart](restart.md): scripts/dev-runner.sh --restart
- [Show recent logs (last 20 lines)](show-recent-logs-last-20-lines.md): scripts/dev-runner.sh --logs
- [Follow logs in real-time](follow-logs-in-real-time.md): tail -f /tmp/opik-backend.log
- [On Windows](on-windows.md): Get-Content -Wait $env:TEMP\opik-backend.log
- [View all Opik containers](view-all-opik-containers.md): docker ps --filter "name=opik-"
- [View logs from Docker services](view-logs-from-docker-services.md): docker logs -f opik-backend-1
- [Execute commands in containers](execute-commands-in-containers.md): docker exec -it opik-mysql-1 mysql -u root -p
- [Restart a specific Docker service](restart-a-specific-docker-service.md): docker restart opik-backend-1
- [Check port conflicts](check-port-conflicts.md): lsof -i :5173 # Frontend
- [On Windows](on-windows-2.md): Get-NetTCPConnection -LocalPort 5173
- [Clean backend build](clean-backend-build.md): cd apps/opik-backend
- [Clean frontend build](clean-frontend-build.md): cd apps/opik-frontend
- [Check MySQL is accessible](check-mysql-is-accessible.md): docker exec -it opik-mysql-1 mysql -u root -p
- [Check ClickHouse is accessible](check-clickhouse-is-accessible.md): docker exec -it opik-clickhouse-1 clickhouse-client
- [Or via HTTP](or-via-http.md): echo 'SELECT version()' | curl -H 'X-ClickHouse-User: opik' -H 'X-ClickHouse-Key: opik' ' -d @-
- [Kill stuck backend process](kill-stuck-backend-process.md): pkill -f "opik-backend.*jar"
- [Kill stuck frontend process](kill-stuck-frontend-process.md): pkill -f "vite.*opik-frontend"
- [On Windows](on-windows-3.md): Get-Process | Where-Object {$_.Path -like "*opik-backend*"} | Stop-Process -Force
- [Complete cleanup and restart](complete-cleanup-and-restart.md): scripts/dev-runner.sh --stop
- [1. Start BE-only mode (fastest for backend work)](1-start-be-only-mode-fastest-for-backend-work.md): scripts/dev-runner.sh --be-only-restart
- [3. Rebuild and restart backend](3-rebuild-and-restart-backend.md): scripts/dev-runner.sh --build-be
- [4. Test changes via UI at http://localhost:5173](4-test-changes-via-ui-at-httplocalhost5173.md)
- [1. Start local process mode](1-start-local-process-mode.md): scripts/dev-runner.sh --restart
- [3. View changes at http://localhost:5174](3-view-changes-at-httplocalhost5174.md)
- [1. Start local process mode](1-start-local-process-mode-2.md): scripts/dev-runner.sh --restart
- [Backend changes require rebuild:](backend-changes-require-rebuild.md): scripts/dev-runner.sh --build-be
- [3. Test at http://localhost:5174](3-test-at-httplocalhost5174.md)
- [1. Start infrastructure only](1-start-infrastructure-only.md): ./opik.sh --infra --port-mapping
- [2. Start backend separately if needed](2-start-backend-separately-if-needed.md): cd apps/opik-backend
- [4. Test SDK changes](4-test-sdk-changes.md): cd sdks/python
- [2. Run tests against full environment](2-run-tests-against-full-environment.md): cd tests_end_to_end
- [3. Clean up](3-clean-up.md): ./opik.sh --stop
- [For Cursor users - creates .cursor symlink to .agents/](for-cursor-users-creates-cursor-symlink-to-agents.md): make cursor
- [For Claude Code users - syncs rules to .claude/ and generates .mcp.json](for-claude-code-users-syncs-rules-to-claude-and-generates-mcpjson.md): make claude
- [Remove hooks](remove-hooks.md): make hooks-remove
- [Python SDK](python-sdk.md): Learn how to contribute to the Opik Python SDK, enabling seamless integration for developers in their Python applicat...
- [Contributing to the Opik Python SDK](contributing-to-the-opik-python-sdk.md): The Opik Python SDK is a key component of our platform, allowing developers to integrate Opik into their Python appli...
- [TypeScript SDK](typescript-sdk.md): Learn how to contribute effectively to the Opik TypeScript SDK by reviewing guidelines and project structure.
- [Contributing to the TypeScript SDK](contributing-to-the-typescript-sdk.md): This guide will help you get started with contributing to the Opik TypeScript SDK.
- [Opik Optimizer](opik-optimizer.md): Contribute effectively to the Agent Optimizer SDK and enhance model performance. Start optimizing prompts with our de...
- [Contributing to the Agent Optimizer SDK](contributing-to-the-agent-optimizer-sdk.md): This guide will help you get started with contributing to the Agent Optimizer SDK, our tool for optimizing prompts an...
- [Frontend](frontend.md): Learn how to contribute to the Opik frontend project by reviewing guidelines and project structure for effective coll...
- [Contributing to the Frontend](contributing-to-the-frontend.md): This guide will help you get started with contributing to the Opik frontend.
- [Backend](backend.md): Learn how to contribute to the Opik backend, enhancing data handling and API functionality in this Java application.
- [Contributing to the Backend](contributing-to-the-backend.md): This guide will help you get started with contributing to the Opik backend.
- [Bounty Program](bounty-program.md): Participate in the Opik Bounty Program to enhance the platform, showcase your skills, and earn rewards for your contr...
- [Opik Bounty Program](opik-bounty-program.md): Welcome to the Opik Bounty Program! We're excited to collaborate with our community to improve and expand the Opik pl...
- [Overview](overview-4.md): Explore how to log, visualize, and evaluate LLM calls and RAG pipelines using Opik's robust integrations with popular...
- [Observability for BeeAI (TypeScript) with Opik](observability-for-beeai-typescript-with-opik.md): Start here to integrate Opik into your BeeAI-based genai application for end-to-end LLM observability, unit testing, ...
- [Observability for LangChain (JavaScript) with Opik](observability-for-langchain-javascript-with-opik.md): Start here to integrate Opik into your LangChain-based genai application for end-to-end LLM observability, unit testi...
- [Observability for Mastra with Opik](observability-for-mastra-with-opik.md): Start here to integrate Opik into your Mastra-based genai application for end-to-end LLM observability, unit testing,...
- [Observability for Vercel AI SDK with Opik](observability-for-vercel-ai-sdk-with-opik.md): Start here to integrate Opik into your Vercel AI SDK-based genai application for end-to-end LLM observability, unit t...
- [Observability for Cloudflare Workers AI with Opik](observability-for-cloudflare-workers-ai-with-opik.md): Start here to integrate Opik into your Cloudflare Workers AI-based genai application for end-to-end LLM observability...
- [Observability for Google Gemini (TypeScript) with Opik](observability-for-google-gemini-typescript-with-opik.md): Start here to integrate Opik into your Google Gemini-based genai application for end-to-end LLM observability, unit t...
- [Observability for OpenAI (TypeScript) with Opik](observability-for-openai-typescript-with-opik.md): Start here to integrate Opik into your OpenAI-based genai application for end-to-end LLM observability, unit testing,...
- [Opik TypeScript SDK](opik-typescript-sdk.md): Monitor and debug your JavaScript/TypeScript apps with Opik's SDK, enabling seamless integration for LLM applications...
- [If running on Opik Cloud](if-running-on-opik-cloud.md): export OPIK_URL_OVERRIDE="
- [If running locally](if-running-locally.md): export OPIK_URL_OVERRIDE="
- [Observability for AG2 with Opik](observability-for-ag2-with-opik.md): Start here to integrate Opik into your AG2-based genai application for end-to-end LLM observability, unit testing, an...
- [1. Import our agent class](1-import-our-agent-class.md): from autogen import ConversableAgent, LLMConfig
- [2. Define our LLM configuration for OpenAI's GPT-4o mini](2-define-our-llm-configuration-for-openais-gpt-4o-mini.md)
- [uses the OPENAI_API_KEY environment variable](uses-the-openai-api-key-environment-variable.md): llm_config = LLMConfig(api_type="openai", model="gpt-4o-mini")
- [3. Create our LLM agent within the parent span context](3-create-our-llm-agent-within-the-parent-span-context.md): with llm_config:
- [Observability for Agno with Opik](observability-for-agno-with-opik.md): Start here to integrate Opik into your Agno-based genai application for end-to-end LLM observability, unit testing, a...
- [Configure the tracer provider](configure-the-tracer-provider.md): tracer_provider = TracerProvider()
- [Start instrumenting agno](start-instrumenting-agno.md): AgnoInstrumentor().instrument()
- [Create and configure the agent](create-and-configure-the-agent.md): agent = Agent(
- [Use the agent](use-the-agent.md): agent.print_response("What is the current price of Apple?")
- [Observability for BeeAI (Python) with Opik](observability-for-beeai-python-with-opik.md): Start here to integrate Opik into your BeeAI-based genai application for end-to-end LLM observability, unit testing, ...
- [Configure the OTLP exporter for Opik](configure-the-otlp-exporter-for-opik.md): otlp_exporter = OTLPSpanExporter()
- [Set up the tracer provider](set-up-the-tracer-provider.md): trace.set_tracer_provider(TracerProvider())
- [Instrument your framework](instrument-your-framework.md): BeeAIInstrumentor().instrument() # or SemanticKernelInstrumentor().instrument()
- [Initialize the language model](initialize-the-language-model.md): llm = ChatModel.from_name(
- [Create tools for the agent](create-tools-for-the-agent.md): tools = [
- [Create a ReAct agent with memory](create-a-react-agent-with-memory.md): agent = ReActAgent(llm=llm, tools=tools, memory=TokenMemory(llm))
- [Run the agent](run-the-agent.md): async def main():
- [Run the example](run-the-example.md): if __name__ == "__main__":
- [Observability for AutoGen with Opik](observability-for-autogen-with-opik.md): Start here to integrate Opik into your AutoGen-based genai application for end-to-end LLM observability, unit testing...
- [First we will configure the OpenTelemetry](first-we-will-configure-the-opentelemetry.md): from opentelemetry import trace
- [Now we can define and call the Agent](now-we-can-define-and-call-the-agent.md): import asyncio
- [Define a model client. You can use other model client that implements](define-a-model-client-you-can-use-other-model-client-that-implements.md)
- [the `ChatCompletionClient` interface.](the-chatcompletionclient-interface.md): model_client = OpenAIChatCompletionClient(
- [Define a simple function tool that the agent can use.](define-a-simple-function-tool-that-the-agent-can-use.md)
- [For this example, we use a fake weather tool for demonstration purposes.](for-this-example-we-use-a-fake-weather-tool-for-demonstration-purposes.md): async def get_weather(city: str) -> str:
- [Define an AssistantAgent with the model, tool, system message, and reflection](define-an-assistantagent-with-the-model-tool-system-message-and-reflection.md)
- [enabled. The system message instructs the agent via natural language.](enabled-the-system-message-instructs-the-agent-via-natural-language.md): agent = AssistantAgent(
- [Run the agent and stream the messages to the console.](run-the-agent-and-stream-the-messages-to-the-console.md): async def main() -> None:
- [Observability for CrewAI with Opik](observability-for-crewai-with-opik.md): Start here to integrate Opik into your CrewAI-based genai application for end-to-end LLM observability, unit testing,...
- [Create the crew](create-the-crew.md): my_crew = YourCrewName().crew()
- [Run the crew](run-the-crew.md): result = my_crew.kickoff()
- [Define your crew (using the example from above)](define-your-crew-using-the-example-from-above.md): my_crew = YourCrewName().crew()
- [Enable tracking with the crew instance (required for v1.0.0+)](enable-tracking-with-the-crew-instance-required-for-v100.md): track_crewai(project_name="crewai-integration-demo", crew=my_crew)
- [Pass thread_id via opik_args](pass-thread-id-via-opik-args.md): args_dict = {
- [Observability for DSPy with Opik](observability-for-dspy-with-opik.md): Start here to integrate Opik into your DSPy-based genai application for end-to-end LLM observability, unit testing, a...
- [Observability for Google Agent Development Kit (Python) with Opik](observability-for-google-agent-development-kit-python-with-opik.md): Start here to integrate Opik into your Google Agent Development Kit-based genai application for end-to-end LLM observ...
- [Initialize LiteLLM with OpenAI gpt-4o](initialize-litellm-with-openai-gpt-4o.md): llm = LiteLlm(model="openai/gpt-4o")
- [Create the basic agent](create-the-basic-agent.md): basic_agent = LlmAgent(
- [Configure Opik tracer](configure-opik-tracer.md): opik_tracer = OpikTracer(
- [Instrument the agent with a single function call - this is the recommended approach](instrument-the-agent-with-a-single-function-call-this-is-the-recommended-approac.md): track_adk_agent_recursive(basic_agent, opik_tracer)
- [Configure Opik tracer (same as before)](configure-opik-tracer-same-as-before.md): opik_tracer = OpikTracer(
- [Create the agent with explicit callback configuration](create-the-agent-with-explicit-callback-configuration.md): basic_agent = LlmAgent(
- [Weather specialist agent (no Opik callbacks needed)](weather-specialist-agent-no-opik-callbacks-needed.md): weather_agent = LlmAgent(
- [Time specialist agent (no Opik callbacks needed)](time-specialist-agent-no-opik-callbacks-needed.md): time_agent = LlmAgent(
- [Travel specialist agent (no Opik callbacks needed)](travel-specialist-agent-no-opik-callbacks-needed.md): travel_agent = LlmAgent(
- [Configure Opik tracer for multi-agent example](configure-opik-tracer-for-multi-agent-example.md): multi_agent_tracer = OpikTracer(
- [Coordinator agent with sub-agents](coordinator-agent-with-sub-agents.md): coordinator_agent = LlmAgent(
- [Use track_adk_agent_recursive to instrument all agents at once](use-track-adk-agent-recursive-to-instrument-all-agents-at-once.md)
- [This automatically adds callbacks to the coordinator and ALL sub-agents](this-automatically-adds-callbacks-to-the-coordinator-and-all-sub-agents.md): from opik.integrations.adk import track_adk_agent_recursive
- [Configure Opik tracer for hybrid example](configure-opik-tracer-for-hybrid-example.md): hybrid_tracer = OpikTracer(
- [Create hybrid agent that combines both tracing approaches](create-hybrid-agent-that-combines-both-tracing-approaches.md): hybrid_agent = LlmAgent(
- [Instrument the agent with track_adk_agent_recursive](instrument-the-agent-with-track-adk-agent-recursive.md)
- [The @opik.track decorators in your tools will automatically create child spans](the-opiktrack-decorators-in-your-tools-will-automatically-create-child-spans.md): from opik.integrations.adk import track_adk_agent_recursive
- [ADK session management](adk-session-management.md): session_service = adk_sessions.InMemorySessionService()
- [All traces will be automatically grouped by session_id as thread_id](all-traces-will-be-automatically-grouped-by-session-id-as-thread-id.md): The integration automatically:
- [Ensure all traces are sent before script exits](ensure-all-traces-are-sent-before-script-exits.md): opik_tracer.flush()
- [Create prompt](create-prompt.md): system_prompt = opik.Prompt(
- [Get prompt from the Prompt library](get-prompt-from-the-prompt-library.md): client = opik.Opik()
- [Initialize LiteLLM with OpenAI gpt-4o](initialize-litellm-with-openai-gpt-4o-2.md): llm = LiteLlm(model="openai/gpt-4o")
- [Create the basic agent](create-the-basic-agent-2.md): basic_agent = LlmAgent(
- [Configure Opik tracer](configure-opik-tracer-2.md): opik_tracer = OpikTracer(
- [Instrument the agent with a single function call - this is the recommended approach](instrument-the-agent-with-a-single-function-call-this-is-the-recommended-approac-2.md): track_adk_agent_recursive(basic_agent, opik_tracer)
- [Observability for Haystack with Opik](observability-for-haystack-with-opik.md): Start here to integrate Opik into your Haystack-based genai application for end-to-end LLM observability, unit testin...
- [Add the OpikConnector component to the pipeline](add-the-opikconnector-component-to-the-pipeline.md): pipe.add_component("tracer", OpikConnector("Chat example"))
- [Continue building the pipeline](continue-building-the-pipeline.md): pipe.add_component("prompt_builder", ChatPromptBuilder())
- [Get the trace ID from the pipeline run response](get-the-trace-id-from-the-pipeline-run-response.md): trace_id = response["tracer"]["trace_id"]
- [Log the feedback score](log-the-feedback-score.md): opik_client = opik.Opik()
- [Observability for Harbor with Opik](observability-for-harbor-with-opik.md): Start here to integrate Opik into your Harbor benchmark evaluation runs for end-to-end agent observability and analysis.
- [Run a benchmark with Opik tracking](run-a-benchmark-with-opik-tracking.md): opik harbor run -d terminal-bench@head -a terminus_2 -m gpt-4.1
- [Use a configuration file](use-a-configuration-file.md): opik harbor run -c config.yaml
- [Set project name via environment variable](set-project-name-via-environment-variable.md): export OPIK_PROJECT_NAME=my-benchmark
- [Run a job (alias for jobs start)](run-a-job-alias-for-jobs-start.md): opik harbor run [HARBOR_OPTIONS]
- [Job management](job-management.md): opik harbor jobs start [HARBOR_OPTIONS]
- [Single trial](single-trial.md): opik harbor trials start -p ./my-task -a terminus_2
- [View available options](view-available-options.md): opik harbor --help
- [Set project name](set-project-name.md): export OPIK_PROJECT_NAME=swebench-claude-sonnet
- [Run SWE-bench evaluation with tracking](run-swe-bench-evaluation-with-tracking.md): opik harbor run \
- [Structured Output Tracking for Instructor with Opik](structured-output-tracking-for-instructor-with-opik.md): Start here to integrate Opik into your Instructor-based genai application for structured output tracking, schema vali...
- [We will first create the OpenAI client and add the `track_openai`](we-will-first-create-the-openai-client-and-add-the-track-openai.md)
- [method to log data to Opik](method-to-log-data-to-opik.md): openai_client = track_openai(OpenAI())
- [Patch the OpenAI client for Instructor](patch-the-openai-client-for-instructor.md): client = instructor.from_openai(openai_client)
- [Define your desired output structure](define-your-desired-output-structure.md): class UserInfo(BaseModel):
- [Add Opik tracking](add-opik-tracking.md): anthropic_client = track_anthropic(Anthropic())
- [Patch the Anthropic client for Instructor](patch-the-anthropic-client-for-instructor.md): client = instructor.from_anthropic(
- [Add Opik tracking](add-opik-tracking-2.md): gemini_client = track_genai(genai.Client())
- [Patch the GenAI client for Instructor](patch-the-genai-client-for-instructor.md): client = instructor.from_genai(
- [Observability for LangChain (Python) with Opik](observability-for-langchain-python-with-opik.md): Start here to integrate Opik into your LangChain-based genai application for end-to-end LLM observability, unit testi...
- [Initialize the tracer](initialize-the-tracer.md): opik_tracer = OpikTracer(project_name="langchain-examples")
- [Configure Opik](configure-opik-2.md): opik.configure(use_local=False)
- [Set up API keys](set-up-api-keys.md): if "OPENAI_API_KEY" not in os.environ:
- [Download and set up the Chinook database](download-and-set-up-the-chinook-database.md): import requests
- [Create synthetic questions using OpenAI](create-synthetic-questions-using-openai.md): client = OpenAI()
- [Create dataset in Opik](create-dataset-in-opik.md): opik_client = opik.Opik()
- [Create the LangChain chain with OpikTracer](create-the-langchain-chain-with-opiktracer.md): opik_tracer = OpikTracer(tags=["sql_generation"])
- [Test the chain](test-the-chain.md): response = chain.invoke({"question": "How many employees are there?"})
- [Set up evaluation](set-up-evaluation.md): valid_sql_query = ValidSQLQuery(name="valid_sql_query", db=db)
- [Run evaluation](run-evaluation.md): res = evaluate(
- [Calling Langchain object](calling-langchain-object.md): traces = opik_tracer.created_traces()
- [Getting the content of the logged traces](getting-the-content-of-the-logged-traces.md): traces = opik_tracer.created_traces()
- [... calling Langchain object](calling-langchain-object-2.md): traces = opik_tracer.created_traces()
- [All traces with the same thread_id will be grouped together](all-traces-with-the-same-thread-id-will-be-grouped-together.md): opik_tracer = OpikTracer(thread_id="user-session-123")
- [In your service that receives distributed trace headers.](in-your-service-that-receives-distributed-trace-headers.md)
- [The distributed_headers dict can be obtained in the "parent" service via `opik_context.get_distributed_trace_headers()`](the-distributed-headers-dict-can-be-obtained-in-the-parent-service-via-opik-cont.md): distributed_headers = DistributedTraceHeadersDict(
- [LangChain operations will be attached to the existing distributed trace](langchain-operations-will-be-attached-to-the-existing-distributed-trace.md): chain.invoke(input_data, config={"callbacks": [opik_tracer]})
- [Your LangGraph setup](your-langgraph-setup.md): graph = StateGraph(...)
- [Observability for LangGraph with Opik](observability-for-langgraph-with-opik.md): Start here to integrate Opik into your LangGraph-based genai application for end-to-end LLM observability, unit testi...
- [create your LangGraph graph](create-your-langgraph-graph.md): class State(BaseModel):
- [Create OpikTracer and track the graph once - no need to pass callbacks anymore!](create-opiktracer-and-track-the-graph-once-no-need-to-pass-callbacks-anymore.md)
- [The graph visualization is automatically extracted by track_langgraph](the-graph-visualization-is-automatically-extracted-by-track-langgraph.md): opik_tracer = OpikTracer(
- [Now all invocations are automatically tracked](now-all-invocations-are-automatically-tracked.md): for s in app.stream({"messages": [HumanMessage(content = "How to use LangGraph ?")]}):
- [No callbacks needed here either!](no-callbacks-needed-here-either.md): result = app.invoke({"messages": [HumanMessage(content = "How to use LangGraph ?")]})
- [create your LangGraph graph](create-your-langgraph-graph-2.md): class State(BaseModel):
- [Create the OpikTracer](create-the-opiktracer.md): opik_tracer = OpikTracer()
- [Pass the OpikTracer callback to each invocation](pass-the-opiktracer-callback-to-each-invocation.md): for s in app.stream({"messages": [HumanMessage(content = "How to use LangGraph ?")]},
- [Configure Opik](configure-opik-3.md): opik.configure(use_local=False)
- [Define the graph state](define-the-graph-state.md): class GraphState(TypedDict):
- [Create the node functions](create-the-node-functions.md): def classify(question: str) -> str:
- [Create the workflow](create-the-workflow.md): workflow = StateGraph(GraphState)
- [Add conditional routing](add-conditional-routing.md): def decide_next_node(state):
- [The graph visualization is automatically extracted by track_langgraph](the-graph-visualization-is-automatically-extracted-by-track-langgraph-2.md): opik_tracer = OpikTracer(
- [Execute the workflow - no callbacks needed!](execute-the-workflow-no-callbacks-needed.md): inputs = {"question": "Hello, how are you?"}
- [Test with a different input - still tracked automatically](test-with-a-different-input-still-tracked-automatically.md): inputs = {"question": "What is machine learning?"}
- [Build and execute graph](build-and-execute-graph.md): graph = StateGraph(dict)
- [Synchronous execution - tracked functions work automatically](synchronous-execution-tracked-functions-work-automatically.md): result = app.invoke({"value": 21})
- [Build and execute graph](build-and-execute-graph-2.md): graph = StateGraph(dict)
- [Asynchronous execution - requires explicit trace context propagation](asynchronous-execution-requires-explicit-trace-context-propagation.md): result = await app.ainvoke({"value": 21})
- [Build and execute graph](build-and-execute-graph-3.md): graph = StateGraph(dict)
- [create your LangGraph graph](create-your-langgraph-graph-3.md): class State(BaseModel):
- [Note: check_same_thread=False is OK as the implementation uses a lock](note-check-same-threadfalse-is-ok-as-the-implementation-uses-a-lock.md)
- [to ensure thread safety.](to-ensure-thread-safety.md): conn = sqlite3.connect("checkpoints.sqlite", check_same_thread=False)
- [Create the OpikTracer and track the graph](create-the-opiktracer-and-track-the-graph.md): opik_tracer = OpikTracer()
- [Initialize the state](initialize-the-state.md): state = State(**app.get_state(config).values) or State(messages=[])
- [Add the user message](add-the-user-message.md): state.messages.append(HumanMessage(content="Hello, my name is Bob, how are you doing ?"))
- [state.messages.append(HumanMessage(content="What is my name ?"))](statemessagesappendhumanmessagecontentwhat-is-my-name.md): result = app.invoke(state, config=config)
- [Calling LangGraph stream or invoke functions](calling-langgraph-stream-or-invoke-functions.md): traces = opik_tracer.created_traces()
- [Observability for LiveKit with Opik](observability-for-livekit-with-opik.md): Start here to integrate Opik into your LiveKit-based genai application for end-to-end LLM observability, unit testing...
- [For local development, you can use these placeholder values](for-local-development-you-can-use-these-placeholder-values.md)
- [or get real credentials from https://cloud.livekit.io](or-get-real-credentials-from-httpscloudlivekitio.md): LIVEKIT_URL=wss://[your-livekit-project-url]
- [Deepgram API](deepgram-api.md): DEEPGRAM_API_KEY=[your-deepgram-api-key]
- [You'll also need OpenAI API key for the LLM and TTS](youll-also-need-openai-api-key-for-the-llm-and-tts.md): OPENAI_API_KEY=[your-openai-api-key]
- [The OTEl endpoint configuration](the-otel-endpoint-configuration.md): Then, run the application with following command:
- [Observability for LlamaIndex with Opik](observability-for-llamaindex-with-opik.md): Start here to integrate Opik into your LlamaIndex-based genai application for end-to-end LLM observability, unit test...
- [Basic setup](basic-setup.md): opik_callback = LlamaIndexCallbackHandler()
- [Or with optional parameters](or-with-optional-parameters.md): opik_callback = LlamaIndexCallbackHandler(
- [Set project name for better organization](set-project-name-for-better-organization.md): os.environ["OPIK_PROJECT_NAME"] = "llamaindex-integration-demo"
- [Create directory if it doesn't exist](create-directory-if-it-doesnt-exist.md): os.makedirs('./data/paul_graham/', exist_ok=True)
- [Download the file using requests](download-the-file-using-requests.md): url = '
- [Configure Opik integration](configure-opik-integration.md): set_global_handler("opik")
- [Call the tracked function](call-the-tracked-function.md): result = my_llm_application("What is the capital of France?")
- [Configure Opik integration](configure-opik-integration-2.md): set_global_handler("opik")
- [Create a manual trace](create-a-manual-trace.md): with opik.start_as_current_trace(name="user_query_processing"):
- [Configure Opik integration for LLM calls within steps](configure-opik-integration-for-llm-calls-within-steps.md): set_global_handler("opik")
- [Create workflow instance](create-workflow-instance.md): workflow = MyRAGWorkflow()
- [Use start_as_current_span to track workflow execution](use-start-as-current-span-to-track-workflow-execution.md): with opik.start_as_current_span(
- [Configure Opik integration](configure-opik-integration-3.md): set_global_handler("opik")
- [Configure OpenAI LLM with stream_options to include usage information](configure-openai-llm-with-stream-options-to-include-usage-information.md): llm = OpenAI(
- [Token usage will now be tracked in streaming responses](token-usage-will-now-be-tracked-in-streaming-responses.md): response = llm.stream_chat(messages)
- [Observability for Microsoft Agent Framework (Python) with Opik](observability-for-microsoft-agent-framework-python-with-opik.md): Start here to integrate Opik into your Microsoft Agent Framework-based genai application for end-to-end LLM observabi...
- [Observability for OpenAI Agents with Opik](observability-for-openai-agents-with-opik.md): Start here to integrate Opik into your OpenAI Agents-based genai application for end-to-end LLM observability, unit t...
- [Set project name for better organization](set-project-name-for-better-organization-2.md): os.environ["OPIK_PROJECT_NAME"] = "openai-agents-demo"
- [Create and use the advanced agent](create-and-use-the-advanced-agent.md): advanced_agent = create_advanced_agent()
- [Example queries](example-queries.md): queries = [
- [Observability for Pipecat with Opik](observability-for-pipecat-with-opik.md): Start here to integrate Opik into your Pipecat-based real-time voice agent application for end-to-end LLM observabili...
- [Initialize OpenTelemetry with the http exporter](initialize-opentelemetry-with-the-http-exporter.md): from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
- [Configured automatically from .env](configured-automatically-from-env.md): exporter = OTLPSpanExporter()
- [Enable tracing in your PipelineTask](enable-tracing-in-your-pipelinetask.md): task = PipelineTask(
- [Observability for Pydantic AI with Opik](observability-for-pydantic-ai-with-opik.md): Start here to integrate Opik into your Pydantic AI-based genai application for end-to-end LLM observability, unit tes...
- [Enable async support in Jupyter notebooks](enable-async-support-in-jupyter-notebooks.md): nest_asyncio.apply()
- [Create a simple agent](create-a-simple-agent.md): agent = Agent(
- [Run the agent](run-the-agent-2.md): result = agent.run_sync('Where does "hello world" come from?')
- [Observability for Semantic Kernel (Python) with Opik](observability-for-semantic-kernel-python-with-opik.md): Start here to integrate Opik into your Semantic Kernel-based genai application for end-to-end LLM observability, unit...
- [Option 1: Include sensitive data (prompts and completions)](option-1-include-sensitive-data-prompts-and-completions.md): os.environ["SEMANTICKERNEL_EXPERIMENTAL_GENAI_ENABLE_OTEL_DIAGNOSTICS_SENSITIVE"] = (
- [Option 2: Hide sensitive data (prompts and completions)](option-2-hide-sensitive-data-prompts-and-completions.md)
- [os.environ["SEMANTICKERNEL_EXPERIMENTAL_GENAI_ENABLE_OTEL_DIAGNOSTICS"] = "true"](osenvironsemantickernel-experimental-genai-enable-otel-diagnostics-true.md): from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
- [This must be done before any other telemetry calls](this-must-be-done-before-any-other-telemetry-calls.md): set_up_tracing()
- [Observability for Smolagents with Opik](observability-for-smolagents-with-opik.md): Start here to integrate Opik into your Smolagents-based genai application for end-to-end LLM observability, unit test...
- [Import telemetry components](import-telemetry-components.md): from opentelemetry.sdk.trace import TracerProvider
- [Import SmolAgents instrumentation](import-smolagents-instrumentation.md): from openinference.instrumentation.smolagents import SmolagentsInstrumentor
- [Import OTLP exporter](import-otlp-exporter.md): from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
- [Set up the tracer provider with span processor](set-up-the-tracer-provider-with-span-processor.md): trace_provider = TracerProvider()
- [Instrument SmolAgents](instrument-smolagents.md): SmolagentsInstrumentor().instrument(tracer_provider=trace_provider)
- [Now use SmolAgents as usual](now-use-smolagents-as-usual.md): from smolagents import CodeAgent, WebSearchTool, OpenAIServerModel
- [Observability for Strands Agents with Opik](observability-for-strands-agents-with-opik.md): Start here to integrate Opik into your Strands Agents-based genai application for end-to-end LLM observability, unit ...
- [Define the system prompt for the agent](define-the-system-prompt-for-the-agent.md): system_prompt = """You are \"Restaurant Helper\", a restaurant assistant helping customers reserving tables in
- [Configure the Bedrock model to be used by the agent](configure-the-bedrock-model-to-be-used-by-the-agent.md): model = BedrockModel(
- [Configure the agent](configure-the-agent.md): agent = Agent(
- [Observability for VoltAgent with Opik](observability-for-voltagent-with-opik.md): Start here to integrate Opik into your VoltAgent-based genai application for end-to-end LLM observability, unit testi...
- [Observability for Anthropic with Opik](observability-for-anthropic-with-opik.md): Start here to integrate Opik into your Anthropic-based genai application for end-to-end LLM observability, unit testi...
- [Execute the multi-step pipeline](execute-the-multi-step-pipeline.md): generate_opik_story()
- [Observability for AWS Bedrock with Opik](observability-for-aws-bedrock-with-opik.md): Start here to integrate Opik into your AWS Bedrock-based genai application for end-to-end LLM observability, unit tes...
- [Set project name via environment variable](set-project-name-via-environment-variable-2.md): os.environ["OPIK_PROJECT_NAME"] = "bedrock-integration-demo"
- [Set project name via environment variable](set-project-name-via-environment-variable-3.md): os.environ["OPIK_PROJECT_NAME"] = "bedrock-integration-demo"
- [Initialize and track the Bedrock client](initialize-and-track-the-bedrock-client.md): bedrock_client = boto3.client("bedrock-runtime", region_name="us-east-1")
- [Set project name via environment variable](set-project-name-via-environment-variable-4.md): os.environ["OPIK_PROJECT_NAME"] = "bedrock-integration-demo"
- [Initialize and track the Bedrock client](initialize-and-track-the-bedrock-client-2.md): bedrock_client = boto3.client("bedrock-runtime", region_name="us-east-1")
- [Example usage](example-usage-2.md): system_prompt = """You are an app that creates playlists for a radio station
- [Initialize and track the Bedrock client](initialize-and-track-the-bedrock-client-3.md): bedrock_client = boto3.client("bedrock-runtime", region_name="us-east-1")
- [Execute the multi-step pipeline](execute-the-multi-step-pipeline-2.md): generate_opik_story()
- [Observability for BytePlus with Opik](observability-for-byteplus-with-opik.md): Start here to integrate Opik into your BytePlus-based genai application for end-to-end LLM observability, unit testin...
- [Initialize the OpenAI client with BytePlus base URL](initialize-the-openai-client-with-byteplus-base-url.md): client = OpenAI(
- [Call the tracked function](call-the-tracked-function-2.md): result = analyze_data_with_ai("Analyze this business data...")
- [Observability for Cohere with Opik](observability-for-cohere-with-opik.md): Start here to integrate Opik into your Cohere-based genai application for end-to-end LLM observability, unit testing,...
- [Observability for DeepSeek with Opik](observability-for-deepseek-with-opik.md): Start here to integrate Opik into your DeepSeek-based genai application for end-to-end LLM observability, unit testin...
- [Observability for Fireworks AI with Opik](observability-for-fireworks-ai-with-opik.md): Start here to integrate Opik into your Fireworks AI-based genai application for end-to-end LLM observability, unit te...
- [Create your OpenAI client](create-your-openai-client.md): client = OpenAI(
- [Wrap the client with Opik tracking](wrap-the-client-with-opik-tracking.md): tracked_client = track_openai(client, project_name="your-project-name")
- [Make a chat completion call](make-a-chat-completion-call.md): response = tracked_client.chat.completions.create(
- [Use the function](use-the-function.md): result = analyze_text_with_fireworks("Open source AI models are becoming increasingly powerful.")
- [Fireworks AI Configuration](fireworks-ai-configuration.md): export FIREWORKS_API_KEY="your-fireworks-api-key"
- [Opik Configuration](opik-configuration-2.md): export OPIK_PROJECT_NAME="your-project-name"
- [Observability for Google Gemini (Python) with Opik](observability-for-google-gemini-python-with-opik.md): Start here to integrate Opik into your Google Gemini-based genai application for end-to-end LLM observability, unit t...
- [Configure for VertexAI](configure-for-vertexai.md): PROJECT_ID = "your-project-id"
- [Set project name for organization](set-project-name-for-organization.md): os.environ["OPIK_PROJECT_NAME"] = "vertexai-integration-demo"
- [Use the wrapped client](use-the-wrapped-client.md): response = vertexai_client.models.generate_content(
- [Execute the multi-step pipeline](execute-the-multi-step-pipeline-3.md): generate_opik_story()
- [Configure for VertexAI (required for Veo)](configure-for-vertexai-required-for-veo.md): client = genai.Client(
- [Generate videos](generate-videos.md): generate_video("A golden retriever playing in the snow", number_of_videos=2)
- [Observability for Groq with Opik](observability-for-groq-with-opik.md): Start here to integrate Opik into your Groq-based genai application for end-to-end LLM observability, unit testing, a...
- [Execute the multi-step pipeline](execute-the-multi-step-pipeline-4.md): generate_opik_story()
- [Observability for Mistral AI with Opik](observability-for-mistral-ai-with-opik.md): Start here to integrate Opik into your Mistral AI-based genai application for end-to-end LLM observability, unit test...
- [Observability for Novita AI with Opik](observability-for-novita-ai-with-opik.md): Start here to integrate Opik into your Novita AI-based genai application for end-to-end LLM observability, unit testi...
- [Configure LiteLLM for Opik](configure-litellm-for-opik.md): from litellm.integrations.opik.opik import OpikLogger
- [Define your evaluation task](define-your-evaluation-task.md): def evaluation_task(x):
- [Create the Hallucination metric](create-the-hallucination-metric.md): hallucination_metric = Hallucination()
- [Run the evaluation](run-the-evaluation-3.md): evaluation_results = evaluate(
- [Novita AI Configuration](novita-ai-configuration.md): export NOVITA_API_KEY="your-novita-api-key"
- [Opik Configuration](opik-configuration-3.md): export OPIK_PROJECT_NAME="your-project-name"
- [Observability for Ollama with Opik](observability-for-ollama-with-opik.md): Start here to integrate Opik into your Ollama-based genai application for end-to-end LLM observability, unit testing,...
- [Create an OpenAI client](create-an-openai-client.md): client = OpenAI(
- [Log all traces made to with the OpenAI client to Opik](log-all-traces-made-to-with-the-openai-client-to-opik.md): client = track_openai(client)
- [call the local ollama model using the OpenAI client](call-the-local-ollama-model-using-the-openai-client.md): chat_completion = client.chat.completions.create(
- [Create the Opik tracer](create-the-opik-tracer.md): opik_tracer = OpikTracer(tags=["langchain", "ollama"])
- [Create the Ollama model and configure it to use the Opik tracer](create-the-ollama-model-and-configure-it-to-use-the-opik-tracer.md): llm = ChatOllama(
- [Call the Ollama model](call-the-ollama-model.md): messages = [
- [Observability for OpenAI (Python) with Opik](observability-for-openai-python-with-opik.md): Start here to integrate Opik into your OpenAI-based genai application for end-to-end LLM observability, unit testing,...
- [Execute the multi-step pipeline](execute-the-multi-step-pipeline-5.md): generate_opik_story()
- [gets the API Key from environment variable AZURE_OPENAI_API_KEY](gets-the-api-key-from-environment-variable-azure-openai-api-key.md): azure_client = AzureOpenAI(
- [Generate a video](generate-a-video.md): generate_video("A golden retriever playing in the snow")
- [Observability for Predibase with Opik](observability-for-predibase-with-opik.md): Start here to integrate Opik into your Predibase-based genai application for end-to-end LLM observability, unit testi...
- [Create the Opik tracer](create-the-opik-tracer-2.md): opik_tracer = OpikTracer(tags=["predibase", "langchain"])
- [Create Predibase model](create-predibase-model.md): model = Predibase(
- [Test the model with Opik tracing](test-the-model-with-opik-tracing.md): response = model.invoke(
- [Synopsis chain](synopsis-chain.md): template = """You are a playwright. Given the title of play, it is your job to write a synopsis for that title.
- [Review chain](review-chain.md): template = """You are a play critic from the New York Times. Given the synopsis of play, it is your job to write a re...
- [Overall chain](overall-chain.md): overall_chain = SimpleSequentialChain(
- [Run the chain with Opik tracing](run-the-chain-with-opik-tracing.md): review = overall_chain.run("Tragedy at sunset on the beach", callbacks=[opik_tracer])
- [Flush traces to ensure all data is logged](flush-traces-to-ensure-all-data-is-logged.md): opik_tracer.flush()
- [Configure the Opik tracer](configure-the-opik-tracer.md): fine_tuned_model = fine_tuned_model.with_config({"callbacks": [opik_tracer]})
- [Invoke the fine-tuned model](invoke-the-fine-tuned-model.md): response = fine_tuned_model.invoke(
- [Final flush to ensure all traces are logged](final-flush-to-ensure-all-traces-are-logged.md): opik_tracer.flush()
- [Observability for Together AI with Opik](observability-for-together-ai-with-opik.md): Start here to integrate Opik into your Together AI-based genai application for end-to-end LLM observability, unit tes...
- [Observability for IBM watsonx with Opik](observability-for-ibm-watsonx-with-opik.md): Start here to integrate Opik into your IBM watsonx-based genai application for end-to-end LLM observability, unit tes...
- [(required) either one of the following:](required-either-one-of-the-following.md): os.environ["WATSONX_API_KEY"] = "" # IBM cloud API key
- [optional - can also be passed as params to completion() or embedding()](optional-can-also-be-passed-as-params-to-completion-or-embedding.md)
- [os.environ["WATSONX_PROJECT_ID"] = "" # Project ID of your WatsonX instance](osenvironwatsonx-project-id-project-id-of-your-watsonx-instance.md)
- [os.environ["WATSONX_DEPLOYMENT_SPACE_ID"] = "" # ID of your deployment space to use deployed models](osenvironwatsonx-deployment-space-id-id-of-your-deployment-space-to-use-deployed.md): In order to log the LLM calls to Opik, you will need to create the OpikLogger callback. Once the OpikLogger callback ...
- [Execute the multi-step pipeline](execute-the-multi-step-pipeline-6.md): generate_opik_story()
- [Observability for xAI Grok with Opik](observability-for-xai-grok-with-opik.md): Start here to integrate Opik into your xAI Grok-based genai application for end-to-end LLM observability, unit testin...
- [Configure LiteLLM for Opik](configure-litellm-for-opik-2.md): from litellm.integrations.opik.opik import OpikLogger
- [More creative responses](more-creative-responses.md): response = completion(
- [More factual responses](more-factual-responses.md): response = completion(
- [Define your evaluation task](define-your-evaluation-task-2.md): def evaluation_task(x):
- [Create the Hallucination metric](create-the-hallucination-metric-2.md): hallucination_metric = Hallucination()
- [Run the evaluation](run-the-evaluation-4.md): evaluation_results = evaluate(
- [xAI Configuration](xai-configuration.md): export XAI_API_KEY="your-xai-api-key"
- [Opik Configuration](opik-configuration-4.md): export OPIK_PROJECT_NAME="your-project-name"
- [Observability for Gretel with Opik](observability-for-gretel-with-opik.md): Start here to integrate Opik into your Gretel-based genai application for end-to-end LLM observability, unit testing,...
- [Set project name for organization](set-project-name-for-organization-2.md): os.environ["OPIK_PROJECT_NAME"] = "gretel-integration-demo"
- [Initialize Data Designer](initialize-data-designer.md): gretel_navigator = Gretel()
- [Add topic column (categorical sampler)](add-topic-column-categorical-sampler.md): dd.add_column(
- [Add difficulty column](add-difficulty-column.md): dd.add_column(
- [Add question column (LLM-generated)](add-question-column-llm-generated.md): dd.add_column(
- [Add answer column (LLM-generated)](add-answer-column-llm-generated.md): dd.add_column(
- [Generate the dataset](generate-the-dataset.md): workflow_run = dd.create(num_records=20, wait_until_done=True)
- [Convert to Opik format](convert-to-opik-format.md): opik_data = convert_to_opik_format(synthetic_df)
- [Initialize Opik client](initialize-opik-client.md): opik_client = opik.Opik()
- [Create the dataset](create-the-dataset.md): dataset_name = "gretel-ai-qa-dataset"
- [Insert the data](insert-the-data.md): dataset.insert(opik_data)
- [Create sample Q&A data (needs 200+ records for holdout)](create-sample-qa-data-needs-200-records-for-holdout.md): sample_questions = [
- [Generate synthetic version](generate-synthetic-version.md): synthetic_dataset = gretel.safe_synthetic_dataset \
- [Wait for completion and get results](wait-for-completion-and-get-results.md): synthetic_dataset.wait_until_done()
- [Convert to Opik format](convert-to-opik-format-2.md): opik_data_safe = convert_to_opik_format(synthetic_df_safe)
- [Create dataset in Opik](create-dataset-in-opik-2.md): dataset_safe = opik_client.get_or_create_dataset(
- [Evaluate on your dataset](evaluate-on-your-dataset.md): for item in opik_data[:5]: # Evaluate first 5 items
- [Define your evaluation task](define-your-evaluation-task-3.md): def evaluation_task(x):
- [Create the Hallucination metric](create-the-hallucination-metric-3.md): hallucination_metric = Hallucination()
- [Run the evaluation](run-the-evaluation-5.md): evaluation_results = evaluate(
- [Gretel Configuration](gretel-configuration.md): export GRETEL_API_KEY="your-gretel-api-key"
- [Opik Configuration](opik-configuration-5.md): export OPIK_PROJECT_NAME="your-project-name"
- [Observability for Hugging Face Datasets with Opik](observability-for-hugging-face-datasets-with-opik.md): Start here to integrate Opik with Hugging Face Datasets for end-to-end LLM observability, unit testing, and optimizat...
- [Set project name for organization](set-project-name-for-organization-3.md): os.environ["OPIK_PROJECT_NAME"] = "huggingface-datasets-integration-demo"
- [Initialize the converter](initialize-the-converter.md): opik_client = Opik()
- [Load a dataset from Hugging Face](load-a-dataset-from-hugging-face.md): dataset = converter.load_hf_dataset(
- [Convert to Opik format](convert-to-opik-format-3.md): opik_data = converter.convert_to_opik_format(
- [Evaluate on your dataset](evaluate-on-your-dataset-2.md): for item in opik_data[:5]: # Evaluate first 5 items
- [Load SQuAD dataset](load-squad-dataset.md): squad_dataset = converter.load_hf_dataset(
- [Convert to Opik format](convert-to-opik-format-4.md): squad_opik = converter.convert_to_opik_format(
- [Load GLUE SST-2 dataset](load-glue-sst-2-dataset.md): sst2_dataset = converter.load_hf_dataset(
- [Convert to Opik format](convert-to-opik-format-5.md): sst2_opik = converter.convert_to_opik_format(
- [Load Common Crawl dataset](load-common-crawl-dataset.md): cc_dataset = converter.load_hf_dataset(
- [Convert to Opik format](convert-to-opik-format-6.md): cc_opik = converter.convert_to_opik_format(
- [Define your evaluation task](define-your-evaluation-task-4.md): def evaluation_task(x):
- [Create the Hallucination metric](create-the-hallucination-metric-4.md): hallucination_metric = Hallucination()
- [Run the evaluation](run-the-evaluation-6.md): evaluation_results = evaluate(
- [Hugging Face Configuration (optional, for private datasets)](hugging-face-configuration-optional-for-private-datasets.md): export HUGGINGFACE_HUB_TOKEN="your-huggingface-token"
- [Opik Configuration](opik-configuration-6.md): export OPIK_PROJECT_NAME="your-project-name"
- [Evaluate LLM Applications with Ragas Metrics in Opik](evaluate-llm-applications-with-ragas-metrics-in-opik.md): Use Ragas evaluation metrics to assess your LLM application quality and automatically track results in Opik for compr...
- [Import the required dependencies](import-the-required-dependencies.md): from ragas.metrics import AnswerRelevancy
- [Initialize the Ragas metric](initialize-the-ragas-metric.md): llm = LangchainLLMWrapper(ChatOpenAI())
- [Wrap the Ragas metric with RagasMetricWrapper for Opik integration](wrap-the-ragas-metric-with-ragasmetricwrapper-for-opik-integration.md): answer_relevancy_metric = RagasMetricWrapper(
- [Create a small dataset](create-a-small-dataset.md): fiqa_eval = load_dataset("explodinggradients/fiqa", "ragas_eval")
- [Reformat the dataset to match the schema expected by the Ragas evaluate function](reformat-the-dataset-to-match-the-schema-expected-by-the-ragas-evaluate-function.md): hf_dataset = fiqa_eval["baseline"].select(range(3))
- [Create an evaluation task](create-an-evaluation-task.md): def evaluation_task(x):
- [Use the RagasMetricWrapper directly with Opik's evaluate function](use-the-ragasmetricwrapper-directly-with-opiks-evaluate-function.md): opik.evaluation.evaluate(
- [Reformat the dataset to match the schema expected by the Ragas evaluate function](reformat-the-dataset-to-match-the-schema-expected-by-the-ragas-evaluate-function-2.md): dataset = fiqa_eval["baseline"].select(range(3))
- [Initialize the Ragas metric](initialize-the-ragas-metric-2.md): llm = LangchainLLMWrapper(ChatOpenAI())
- [Create the answer relevancy scoring metric](create-the-answer-relevancy-scoring-metric.md): answer_relevancy = RagasMetricWrapper(
- [Observability for Opik LLM Gateway with Opik](observability-for-opik-llm-gateway-with-opik.md): Start here to integrate Opik LLM Gateway into your genai application for centralized LLM provider access using a cons...
- [Observability for Kong AI Gateway with Opik](observability-for-kong-ai-gateway-with-opik.md): Start here to integrate Kong AI Gateway with Opik for production-ready LLM gateway capabilities with enterprise featu...
- [Observability for AISuite with Opik](observability-for-aisuite-with-opik.md): Start here to integrate Opik into your AISuite-based genai application for end-to-end LLM observability, unit testing...
- [Execute the multi-step pipeline](execute-the-multi-step-pipeline-7.md): generate_opik_story()
- [Observability for Anannas AI with Opik](observability-for-anannas-ai-with-opik.md): Start here to integrate Opik into your Anannas AI-based genai application for end-to-end LLM observability, unit test...
- [Create an OpenAI client with Anannas AI's base URL](create-an-openai-client-with-anannas-ais-base-url.md): client = OpenAI(
- [Wrap the client with Opik tracking](wrap-the-client-with-opik-tracking-2.md): client = track_openai(client, project_name="anannas-integration-demo")
- [Make a chat completion request](make-a-chat-completion-request.md): response = client.chat.completions.create(
- [Print the assistant's reply](print-the-assistants-reply.md): print(response.choices[0].message.content)
- [Create and wrap the OpenAI client with Anannas AI's base URL](create-and-wrap-the-openai-client-with-anannas-ais-base-url.md): client = OpenAI(
- [Example usage](example-usage-3.md): text_to_analyze = "Anannas AI provides a unified gateway to access hundreds of LLM models with built-in observability...
- [Observability for Helicone with Opik](observability-for-helicone-with-opik.md): Learn how to integrate Helicone with Opik to log and monitor your LLM traffic using the standard Helicone integration...
- [Create an OpenAI client with Helicone's base URL](create-an-openai-client-with-helicones-base-url.md): client = OpenAI(
- [Wrap the client with Opik tracking](wrap-the-client-with-opik-tracking-3.md): client = track_openai(client, project_name="helicone-integration-demo")
- [Make a chat completion request](make-a-chat-completion-request-2.md): response = client.chat.completions.create(
- [Print the assistant's reply](print-the-assistants-reply-2.md): print(response.choices[0].message.content)
- [Create and wrap the OpenAI client with Helicone's base URL](create-and-wrap-the-openai-client-with-helicones-base-url.md): client = OpenAI(
- [Example usage](example-usage-4.md): result = generate_and_refine("Explain quantum computing in simple terms.")
- [Observability for LiteLLM with Opik](observability-for-litellm-with-opik.md): Start here to integrate Opik into your LiteLLM-based genai application for end-to-end LLM observability, unit testing...
- [Set project name for better organization](set-project-name-for-better-organization-3.md): os.environ["OPIK_PROJECT_NAME"] = "litellm-integration-demo"
- [Observability for OpenRouter with Opik](observability-for-openrouter-with-opik.md): Start here to integrate Opik into your OpenRouter-based genai application for end-to-end LLM observability, unit test...
- [Initialize the OpenAI client with OpenRouter base URL](initialize-the-openai-client-with-openrouter-base-url.md): client = OpenAI(
- [Optional headers for OpenRouter leaderboard](optional-headers-for-openrouter-leaderboard.md): headers = {
- [Observability for Portkey with Opik](observability-for-portkey-with-opik.md): Start here to integrate Portkey with Opik for enterprise-grade LLM gateway capabilities with advanced routing and fal...
- [Wrap the client with Opik tracking](wrap-the-client-with-opik-tracking-4.md): client = track_openai(client, project_name="portkey-integration-demo")
- [Make a chat completion request](make-a-chat-completion-request-3.md): response = client.chat.completions.create(
- [Print the assistant's reply](print-the-assistants-reply-3.md): print(response.choices[0].message.content)
- [Create an OpenAI client configured for Portkey](create-an-openai-client-configured-for-portkey.md): client = OpenAI(
- [Wrap the client with Opik tracking](wrap-the-client-with-opik-tracking-5.md): client = track_openai(client, project_name="portkey-integration-demo")
- [Example usage](example-usage-5.md): result = generate_and_refine("Explain quantum computing in simple terms.")
- [Observability for TrueFoundry with Opik](observability-for-truefoundry-with-opik.md): Start here to integrate TrueFoundry with Opik for enterprise MLOps platform capabilities with LLM gateway features.
- [Create an OpenAI client with TrueFoundry's base URL](create-an-openai-client-with-truefoundrys-base-url.md): client = OpenAI(
- [Wrap the client with Opik tracking](wrap-the-client-with-opik-tracking-6.md): client = track_openai(client, project_name="truefoundry-integration-demo")
- [Make a chat completion request](make-a-chat-completion-request-4.md): response = client.chat.completions.create(
- [Print the assistant's reply](print-the-assistants-reply-4.md): print(response.choices[0].message.content)
- [Create and wrap the OpenAI client with TrueFoundry's base URL](create-and-wrap-the-openai-client-with-truefoundrys-base-url.md): client = OpenAI(
- [Example usage](example-usage-6.md): result = generate_and_refine("Explain quantum computing in simple terms.")
- [Observability for Vercel AI Gateway with Opik](observability-for-vercel-ai-gateway-with-opik.md): Start here to integrate Vercel AI Gateway with Opik for unified access to multiple AI providers with edge-optimized p...
- [Create an OpenAI client with Vercel AI Gateway's base URL](create-an-openai-client-with-vercel-ai-gateways-base-url.md): client = OpenAI(
- [Wrap the client with Opik tracking](wrap-the-client-with-opik-tracking-7.md): client = track_openai(client, project_name="vercel-ai-gateway-demo")
- [Make a chat completion request](make-a-chat-completion-request-5.md): response = client.chat.completions.create(
- [Print the assistant's reply](print-the-assistants-reply-5.md): print(response.choices[0].message.content)
- [Create and wrap the OpenAI client with Vercel AI Gateway's base URL](create-and-wrap-the-openai-client-with-vercel-ai-gateways-base-url.md): client = OpenAI(
- [Example usage](example-usage-7.md): result = generate_and_refine("Explain quantum computing in simple terms.")
- [Validation Tracking for Guardrails AI with Opik](validation-tracking-for-guardrails-ai-with-opik.md): Start here to integrate Opik into your Guardrails AI-based genai application for validation tracking, compliance moni...
- [Overview](overview-5.md): Build powerful applications with Opik's Python SDK, enabling seamless integration and functionality for your projects.
- [OpenTelemetry Python SDK](opentelemetry-python-sdk.md): Learn to instrument your Python applications with OpenTelemetry SDK to effectively send trace data to Opik for better...
- [Using the OpenTelemetry Python SDK](using-the-opentelemetry-python-sdk.md): This guide shows you how to directly instrument your Python applications with the OpenTelemetry SDK to send trace dat...
- [Dependencies: opentelemetry-exporter-otlp](dependencies-opentelemetry-exporter-otlp.md): import os
- [For comet.com](for-cometcom.md): os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "
- [Configure the tracer provider](configure-the-tracer-provider-2.md): resource = Resource.create({
- [Create a tracer provider](create-a-tracer-provider.md): tracer_provider = TracerProvider(resource=resource)
- [Set up the OTLP HTTP exporter](set-up-the-otlp-http-exporter.md): otlp_exporter = OTLPSpanExporter()
- [Add the exporter to the tracer provider](add-the-exporter-to-the-tracer-provider.md): tracer_provider.add_span_processor(BatchSpanProcessor(otlp_exporter))
- [Set the tracer provider](set-the-tracer-provider.md): trace.set_tracer_provider(tracer_provider)
- [Get a tracer](get-a-tracer.md): tracer = trace.get_tracer("example-tracer")
- [Observability for Spring AI (Java) with Opik](observability-for-spring-ai-java-with-opik.md): Start here to integrate Opik into your Spring AI-based genai application for end-to-end LLM observability, unit testi...
- [Enable OpenTelemetry tracing](enable-opentelemetry-tracing.md): management:
- [Disable metrics and logs exporters via OpenTelemetry to avoid putting an extra load on the OpenTelemetry collector](disable-metrics-and-logs-exporters-via-opentelemetry-to-avoid-putting-an-extra-l.md): otel:
- [Observability for Microsoft Agent Framework (.NET) with Opik](observability-for-microsoft-agent-framework-net-with-opik.md): Start here to integrate Opik into your Microsoft Agent Framework-based .NET application for end-to-end LLM observabil...
- [Agent Framework (2 packages)](agent-framework-2-packages.md): dotnet add package Microsoft.Agents.AI --prerelease
- [Hosting (1 package)](hosting-1-package.md): dotnet add package Microsoft.Extensions.Hosting
- [OpenTelemetry (3 packages)](opentelemetry-3-packages.md): dotnet add package OpenTelemetry.Exporter.OpenTelemetryProtocol
- [OpenTelemetry Ruby SDK](opentelemetry-ruby-sdk.md): Configure OpenTelemetry SDK in your Ruby app to send telemetry data to Opik efficiently and enhance your observability.
- [... other gems](other-gems.md): gem 'opentelemetry-sdk'
- [opentelemetry_config.rb](opentelemetry-configrb.md): require 'opentelemetry/sdk'
- [app.rb](apprb.md): require_relative './opentelemetry_config'
- [Create a trace and spans via OpenTelemetry instrumentation](create-a-trace-and-spans-via-opentelemetry-instrumentation.md): TRACER.in_span('JSONPlaceholder API Call', kind: :client, attributes: {
- [Observability for Cursor with Opik](observability-for-cursor-with-opik.md): Start here to integrate Opik into your Cursor-based genai application for end-to-end LLM observability, unit testing,...
- [Observability for Dify with Opik](observability-for-dify-with-opik.md): Start here to integrate Opik into your Dify-based genai application for end-to-end LLM observability, unit testing, a...
- [Observability for Flowise with Opik](observability-for-flowise-with-opik.md): Start here to integrate Opik into your Flowise-based genai application for end-to-end LLM observability, unit testing...
- [Observability for Langflow with Opik](observability-for-langflow-with-opik.md): Start here to integrate Opik into your Langflow applications for end-to-end LLM observability.
- [Observability for n8n with Opik](observability-for-n8n-with-opik.md): Start here to integrate Opik into your n8n-based workflow automation for end-to-end LLM observability.
- [Clone and navigate to the example](clone-and-navigate-to-the-example.md): git clone
- [Set your Opik API key (get one free at https://www.comet.com/signup)](set-your-opik-api-key-get-one-free-at-httpswwwcometcomsignup.md): export OPIK_API_KEY=your_api_key_here
- [Build and run](build-and-run.md): docker-compose up --build
- [Install globally](install-globally.md): npm install -g n8n-observability
- [Only trace specific nodes](only-trace-specific-nodes.md): export N8N_OTEL_NODE_INCLUDE="OpenAI,HTTP Request"
- [Exclude noisy nodes](exclude-noisy-nodes.md): export N8N_OTEL_NODE_EXCLUDE="Wait,Set"
- [Disable I/O capture for privacy](disable-io-capture-for-privacy.md): export N8N_OTEL_CAPTURE_INPUT=false
- [Observability for OpenWebUI with Opik](observability-for-openwebui-with-opik.md): Start here to integrate Opik into your OpenWebUI-based AI application for end-to-end LLM observability and monitoring.
- [OpenTelemetry](opentelemetry.md): Integrate OpenTelemetry SDKs with Opik to enhance your ML/AI apps with distributed tracing and streamline monitoring.
- [Get Started with OpenTelemetry](get-started-with-opentelemetry.md): Opik provides native support for OpenTelemetry (OTel), allowing you to instrument
- [For Comet-hosted installations](for-comet-hosted-installations.md): OPIK_ENDPOINT = "
- [Initialize the trace provider](initialize-the-trace-provider.md): provider = TracerProvider()
- [Agent Optimization](agent-optimization.md): Learn about Opik's automated LLM prompt and agent optimization SDK. Discover MetaPrompt, Few-shot Bayesian, and Evolu...
- [Optimization Studio](optimization-studio.md): Run prompt optimizations from the Opik UI with datasets, metrics, and visual progress tracking.
- [Quickstart](quickstart-2.md): Install the Agent Optimizer SDK, run your first optimization, and inspect the results in under 10 minutes.
- [Optimizer Introduction Cookbook](optimizer-introduction-cookbook.md): Learn how to use Opik Agent Optimizer with the HotPotQA dataset through an interactive notebook, covering setup, conf...
- [Optimizer Frequently Asked Questions](optimizer-frequently-asked-questions.md): Find answers to common questions about Opik Agent Optimizer, including optimizer selection, configuration, usage, and...
- [Changelog](changelog.md): Release notes focused on Agent Optimizer features.
- [Known Issues](known-issues.md): Known runtime issues when running Opik Optimizer with current dependencies.
- [Opik Agent Optimizer Core Concepts](opik-agent-optimizer-core-concepts.md): Learn about the core concepts of Opik Agent Optimizer, including key terms, evaluation processes, optimization workfl...
- [Configuring LLM Providers](configuring-llm-providers.md): Learn how to configure different LLM providers like OpenAI, Anthropic, Gemini, Azure, and Ollama for use with the Opi...
- [ChatPrompt model - this is the model your prompt runs on](chatprompt-model-this-is-the-model-your-prompt-runs-on.md): prompt = ChatPrompt(
- [Optimizer model - this is the model that improves your prompt](optimizer-model-this-is-the-model-that-improves-your-prompt.md): optimizer = MetaPromptOptimizer(
- [Examples of the LiteLLM model format](examples-of-the-litellm-model-format.md): model="openai/gpt-4o" # OpenAI
- [Configure model parameters for the ChatPrompt](configure-model-parameters-for-the-chatprompt.md): prompt = ChatPrompt(
- [Configure model parameters for the Optimizer](configure-model-parameters-for-the-optimizer.md): optimizer = MetaPromptOptimizer(
- [Use a cost-effective model for prompt evaluation](use-a-cost-effective-model-for-prompt-evaluation.md): prompt = ChatPrompt(
- [Use a powerful model for optimization reasoning](use-a-powerful-model-for-optimization-reasoning.md): optimizer = MetaPromptOptimizer(
- [Define datasets](define-datasets.md): Design, version, and validate datasets used for optimization runs.
- [Create training dataset (70-80% of your data)](create-training-dataset-70-80-of-your-data.md): training_dataset = client.get_or_create_dataset(name="agent-opt-train")
- [Create validation dataset (20-30% of your data)](create-validation-dataset-20-30-of-your-data.md): validation_dataset = client.get_or_create_dataset(name="agent-opt-val")
- [Use both during optimization](use-both-during-optimization.md): result = optimizer.optimize_prompt(
- [After optimization, test on unseen data](after-optimization-test-on-unseen-data.md): test_dataset = client.get_dataset(name="agent-opt-test")
- [Define metrics](define-metrics.md): Create reliable metrics and composite objectives for Agent Optimizer runs.
- [Metric that needs task_span](metric-that-needs-task-span.md): def cost_in_cents(dataset_item, llm_output, task_span):
- [Metric that needs task_span](metric-that-needs-task-span-2.md): def duration_seconds(dataset_item, llm_output, task_span):
- [Regular metric without task_span](regular-metric-without-task-span.md): def answer_relevance(dataset_item, llm_output):
- [Use negative weights to minimize cost and duration](use-negative-weights-to-minimize-cost-and-duration.md): objective = MultiMetricObjective(
- [Optimize prompts](optimize-prompts.md): Pick the right optimizer, run experiments, and ship better prompts.
- [Optimize agents](optimize-agents.md): Connect the Agent Optimizer to Agent Frameworks.
- [Define both prompts in the workflow](define-both-prompts-in-the-workflow.md): prompts = {
- [Returns list of message dicts: [{"role": "system", "content": "..."}, ...]](returns-list-of-message-dicts-role-system-content.md): For multi-prompt workflows, pass additional context when calling`get_messages()`:
- [Pass intermediate results to subsequent prompts](pass-intermediate-results-to-subsequent-prompts.md): context = {**dataset_item, "intermediate_result": some_value}
- [Define multi-prompt workflow](define-multi-prompt-workflow.md): prompts = {
- [Optimize multimodal prompts](optimize-multimodal-prompts.md): Run optimizations for prompts that combine text, images, and other modalities.
- [Dashboard results](dashboard-results.md): Analyze optimization trials, failure modes, and improvements inside Opik.
- [result is the OptimizationResult returned by your optimizer call](result-is-the-optimizationresult-returned-by-your-optimizer-call.md): optimized_prompt = result.prompt
- [Optimization algorithms overview](optimization-algorithms-overview.md): Compare Agent Optimizer algorithms and pick the right one for your workload.
- [Optimizer benchmarks](optimizer-benchmarks.md): Compare algorithm performance across common datasets and learn how to run the benchmark suite locally.
- [MetaPrompt Optimizer](metaprompt-optimizer.md): Learn how to use the MetaPrompt Optimizer to refine and improve your LLM prompts through systematic analysis and iter...
- [Initialize optimizer](initialize-optimizer.md): optimizer = MetaPromptOptimizer(
- [Prepare dataset](prepare-dataset.md): dataset = datasets.hotpot(count=300)
- [Define metric and task configuration (see docs for more options)](define-metric-and-task-configuration-see-docs-for-more-options.md): def levenshtein_ratio(dataset_item, llm_output):
- [Run optimization](run-optimization.md): results = optimizer.optimize_prompt(
- [Access results](access-results.md): results.display()
- [HRPO (Hierarchical Reflective Prompt Optimizer)](hrpo-hierarchical-reflective-prompt-optimizer.md): Learn how to use HRPO (Hierarchical Reflective Prompt Optimizer) to improve prompts through systematic root cause ana...
- [1. Define your evaluation dataset](1-define-your-evaluation-dataset.md): dataset = datasets.hotpot(count=300) # or use your own dataset
- [2. Configure the evaluation metric (MUST return reasons!)](2-configure-the-evaluation-metric-must-return-reasons.md): def answer_quality_metric(dataset_item, llm_output):
- [3. Define your initial prompt](3-define-your-initial-prompt.md): initial_prompt = ChatPrompt(
- [4. Initialize HRPO](4-initialize-hrpo.md): optimizer = HRPO(
- [5. Run the optimization](5-run-the-optimization.md): optimization_result = optimizer.optimize_prompt(
- [6. View the results](6-view-the-results.md): optimization_result.display()
- [Few-Shot Bayesian Optimizer](few-shot-bayesian-optimizer.md): Learn how to use the Few-Shot Bayesian Optimizer to find optimal few-shot examples for your chat-based prompts using ...
- [Initialize optimizer](initialize-optimizer-2.md): optimizer = FewShotBayesianOptimizer(
- [Prepare dataset](prepare-dataset-2.md): dataset = datasets.hotpot(count=300)
- [Define metric and prompt (see docs for more options)](define-metric-and-prompt-see-docs-for-more-options.md): def levenshtein_ratio(dataset_item, llm_output):
- [Run optimization](run-optimization-2.md): results = optimizer.optimize_prompt(
- [Access results](access-results-2.md): results.display()
- [Evolutionary Optimizer: Genetic Algorithms](evolutionary-optimizer-genetic-algorithms.md): Learn how to use the Evolutionary Optimizer to discover optimal prompts through genetic algorithms, with support for ...
- [1. Define your evaluation dataset](1-define-your-evaluation-dataset-2.md): dataset = datasets.tiny_test() # Replace with your actual dataset
- [2. Configure the evaluation metric](2-configure-the-evaluation-metric.md): def levenshtein_ratio(dataset_item, llm_output):
- [3. Define your base prompt and task configuration](3-define-your-base-prompt-and-task-configuration.md): initial_prompt = ChatPrompt(
- [4. Initialize the EvolutionaryOptimizer](4-initialize-the-evolutionaryoptimizer.md): optimizer = EvolutionaryOptimizer(
- [5. Run the optimization](5-run-the-optimization-2.md): optimization_result = optimizer.optimize_prompt(
- [6. View the results](6-view-the-results-2.md): optimization_result.display()
- [GEPA Optimizer](gepa-optimizer.md): Use the external GEPA package through Opik's`GepaOptimizer`to optimize a single system prompt for single-turn tasks...
- [Parameter Optimizer: Bayesian Parameter Tuning](parameter-optimizer-bayesian-parameter-tuning.md): Learn how to use the Parameter Optimizer to find optimal LLM call parameters (temperature, top_p, etc.) using Bayesia...
- [Initialize optimizer](initialize-optimizer-3.md): optimizer = ParameterOptimizer(
- [Prepare dataset](prepare-dataset-3.md): dataset = datasets.hotpot(count=300)
- [Define metric](define-metric.md): def levenshtein_ratio(dataset_item, llm_output):
- [Define prompt (this stays unchanged)](define-prompt-this-stays-unchanged.md): prompt = ChatPrompt(
- [Define parameter search space](define-parameter-search-space.md): parameter_space = ParameterSearchSpace(
- [Run optimization](run-optimization-3.md): results = optimizer.optimize_parameter(
- [Access results](access-results-3.md): results.display()
- [Optimize more parameters including model selection](optimize-more-parameters-including-model-selection.md): parameter_space = ParameterSearchSpace(
- [Run with more trials and custom Optuna sampler](run-with-more-trials-and-custom-optuna-sampler.md): import optuna
- [Tool Optimization (MCP & Function Calling)](tool-optimization-mcp-function-calling.md): Learn how to optimize prompts that leverage MCP (Model Context Protocol) tools and function calling capabilities usin...
- [GEPA example: optimizing an agent with function calling](gepa-example-optimizing-an-agent-with-function-calling.md): prompt = ChatPrompt(
- [GEPA optimizes the agent's prompt, not the tools themselves](gepa-optimizes-the-agents-prompt-not-the-tools-themselves.md): optimizer = GepaOptimizer(model="gpt-4o-mini")
- [See scripts/archive/litellm_metaprompt_context7_mcp_example.py for working examples](see-scriptsarchivelitellm-metaprompt-context7-mcp-examplepy-for-working-examples.md): from opik_optimizer import MetaPromptOptimizer
- [MCP tools are configured through mcp.json manifests](mcp-tools-are-configured-through-mcpjson-manifests.md)
- [The MetaPrompt Optimizer can optimize MCP tool descriptions and usage](the-metaprompt-optimizer-can-optimize-mcp-tool-descriptions-and-usage.md): optimizer = MetaPromptOptimizer(model="gpt-4")
- [Check the scripts/ directory for working examples](check-the-scripts-directory-for-working-examples.md): MCP Tool Optimization:
- [Define a research assistant prompt with tools](define-a-research-assistant-prompt-with-tools.md): research_prompt = ChatPrompt(
- [Initialize the optimizer](initialize-the-optimizer.md): optimizer = MetaPromptOptimizer(
- [Define evaluation metric](define-evaluation-metric.md): def research_quality_metric(dataset_item, llm_output):
- [Run optimization](run-optimization-4.md): result = optimizer.optimize_prompt(
- [Synthetic Data Optimizer Cookbook](synthetic-data-optimizer-cookbook.md): Learn how to generate synthetic Q&A data from Opik traces and optimize prompts using the MetaPromptOptimizer through ...
- [ARC-AGI Optimization Tutorial](arc-agi-optimization-tutorial.md): End-to-end ARC-AGI optimization walkthrough with Opik Agent Optimizer.
- [Multimodal Agent Optimization Tutorial](multimodal-agent-optimization-tutorial.md): Multimodal agent optimization tutorial inspired by a self-driving car example.
- [Extending Optimizers](extending-optimizers.md): Learn how to extend Opik's optimization framework with custom algorithms and contribute to the project's development.
- [Returns a parsed Pydantic object, not raw text](returns-a-parsed-pydantic-object-not-raw-text.md): analysis = call_model(
- [Custom metrics](custom-metrics.md): Build specialized metrics, integrate external models, and reuse them across optimizations.
- [Custom Optimizer Prompts](custom-optimizer-prompts.md): Learn how to override and customize the internal prompt templates used by Opik optimizers to add domain constraints, ...
- [Simple dict override](simple-dict-override.md): optimizer = MetaPromptOptimizer(
- [Get the current (possibly overridden) prompt](get-the-current-possibly-overridden-prompt.md): current = optimizer.get_prompt("reasoning_system")
- [Get the original default (before any overrides)](get-the-original-default-before-any-overrides.md): default = optimizer.prompts.get_default("reasoning_system")
- [Original: "Generate {num_prompts} variations of the prompt."](original-generate-num-prompts-variations-of-the-prompt.md)
- [Your override should keep {num_prompts}:](your-override-should-keep-num-prompts.md): prompt_overrides = {
- [This will raise KeyError - "reasoing_system" is misspelled](this-will-raise-keyerror-reasoing-system-is-misspelled.md): optimizer = MetaPromptOptimizer(
- [KeyError: "Unknown prompt keys: ['reasoing_system'].](keyerror-unknown-prompt-keys-reasoing-system.md)
- [Available: ['candidate_generation', 'reasoning_system', ...]"](available-candidate-generation-reasoning-system.md): **Tips for effective prompt customization:**
- [Create optimizer with customizations](create-optimizer-with-customizations.md): optimizer = MetaPromptOptimizer(
- [Verify customizations applied](verify-customizations-applied.md): print("Customized reasoning prompt:")
- [Sampling controls](sampling-controls.md): Use n_samples, n_samples_minibatch, and the n parameter to trade cost for stability and exploration.
- [Generate 3 candidates per evaluation, select best](generate-3-candidates-per-evaluation-select-best.md): prompt = ChatPrompt(
- [What happens internally:](what-happens-internally.md): candidates = ["output_1", "output_2", "output_3"]
- [Multiple Completions (n parameter)](multiple-completions-n-parameter.md): Learn how to use the n parameter to generate multiple candidate outputs per evaluation, reducing variance and improvi...
- [Generate 3 candidates per evaluation, select best](generate-3-candidates-per-evaluation-select-best-2.md): prompt = ChatPrompt(
- [What happens internally:](what-happens-internally-2.md): candidates = ["output_1", "output_2", "output_3"]
- [Create prompt with n=3 for variety](create-prompt-with-n3-for-variety.md): prompt = ChatPrompt(
- [Define metric](define-metric-2.md): def extraction_accuracy(dataset_item, llm_output):
- [Optimize - each trial evaluates 3 candidates, picks best](optimize-each-trial-evaluates-3-candidates-picks-best.md): optimizer = MetaPromptOptimizer(model="gpt-4o")
- [Chaining optimizers](chaining-optimizers.md): Run multiple optimizers in sequence to balance exploration and fine-tuning.
- [Reuse the optimized prompt from the first stage](reuse-the-optimized-prompt-from-the-first-stage.md): optimized_prompt = prompt.with_messages(meta_result.prompt)
- [Opik Agent Optimizer API Reference](opik-agent-optimizer-api-reference.md): Technical SDK reference guide
- [❌ Deprecated](deprecated.md): optimizer = FewShotBayesianOptimizer(
- [✅ Correct](correct.md): optimizer = FewShotBayesianOptimizer(
- [Overview](overview-6.md): Explore how to self-host Opik on your infrastructure with full feature access, including tracing and evaluation, with...
- [Configure the Python SDK to point to the local Opik platform](configure-the-python-sdk-to-point-to-the-local-opik-platform.md): export OPIK_BASE_URL=
- [Local deployment](local-deployment.md): Configure your local Opik deployment using Docker Compose for quick setup. Ideal for testing but not for production e...
- [Navigate to the opik/deployment/docker-compose directory](navigate-to-the-opikdeploymentdocker-compose-directory.md): cd opik/deployment/docker-compose
- [Start full Opik platform (equivalent to ./opik.sh)](start-full-opik-platform-equivalent-to-opiksh.md): docker compose --profile opik up --detach
- [Start only infrastructure services (equivalent to ./opik.sh --infra)](start-only-infrastructure-services-equivalent-to-opiksh-infra.md): docker compose up --detach
- [Start infrastructure + backend services (equivalent to ./opik.sh --backend)](start-infrastructure-backend-services-equivalent-to-opiksh-backend.md): docker compose --profile backend up --detach
- [Or manually remove containers and volumes](or-manually-remove-containers-and-volumes.md): cd deployment/docker-compose
- [Clone the Opik repository](clone-the-opik-repository.md): git clone
- [Build the Opik platform from source](build-the-opik-platform-from-source.md): ./opik.sh --build
- [Kubernetes deployment](kubernetes-deployment.md): Learn to configure and deploy Opik on Kubernetes using our Helm chart for robust, production-ready environments.
- [Prerequisites](prerequisites.md): In order to install Opik on a Kubernetes cluster, you will need to have the following tools installed:
- [Installation](installation.md): You can install Opik using the helm chart maintained by the Opik team by running the following commands:
- [Configuration](configuration.md): You can find a full list of the configuration options in the [helm chart documentation](https://comet-ml.github.io/op...
- [Advanced deployment options](advanced-deployment-options.md): component:
- [Delete your installation](delete-your-installation.md): Before deleting opik installation with helm, make sure to remove finalizer on the clickhouse resource:
- [Version Compatibility](version-compatibility.md): It's important to ensure that your Python SDK version matches your Kubernetes deployment version to avoid compatibili...
- [Troubleshooting](troubleshooting-2.md): If you get this error when running helm
- [Migrating from Bitnami Charts and Images](migrating-from-bitnami-charts-and-images.md): Learn how to migrate from Bitnami charts and images to Opik's official Helm chart
- [Migrating from Bitnami Charts and Images](migrating-from-bitnami-charts-and-images-2.md): Opik's Helm chart has migrated from Bitnami charts and images to official Docker images and alternative Helm charts. ...
- [Platform Architecture](platform-architecture.md): Learn how Opik's architecture utilizes multiple services for backend, frontend, and data management, optimizing perfo...
- [Scaling Opik](scaling-opik.md): Learn best practices and configurations for running Opik in production, ensuring resilience and scalability for missi...
- [Scaling Opik](scaling-opik-2.md): Opik is built to power mission-critical workloads at scale. Whether you're running a small proof of concept or a high...
- [Advanced clickhouse backup](advanced-clickhouse-backup.md): Learn to implement SQL-based and dedicated backup tools for ClickHouse in Opik's Kubernetes setup to ensure data prot...
- [ClickHouse Backup Guide](clickhouse-backup-guide.md): This guide covers the two backup options available for ClickHouse in Opik's Kubernetes deployment:
- [Connect to ClickHouse](connect-to-clickhouse.md): kubectl exec -it deployment/clickhouse-opik-clickhouse -- clickhouse-client
- [Restore from S3 backup](restore-from-s3-backup.md): RESTORE ALL FROM S3(' 'ACCESS_KEY', 'SECRET_KEY');
- [Port-forward to access the backup server](port-forward-to-access-the-backup-server.md): kubectl port-forward svc/clickhouse-opik-clickhouse 7171:7171
- [Create a backup](create-a-backup.md): curl -X POST " +%Y%m%d-%H%M%S)"
- [List available backups](list-available-backups.md): curl "
- [Upload backup to S3](upload-backup-to-s3.md): curl -X POST "
- [Download backup from S3](download-backup-from-s3.md): curl -X POST "
- [Restore backup](restore-backup.md): curl -X POST "
- [Check backup job logs](check-backup-job-logs.md): kubectl logs -l app=clickhouse-backup
- [Check CronJob status](check-cronjob-status.md): kubectl get cronjobs
- [Test S3 connectivity](test-s3-connectivity.md): kubectl exec -it deployment/clickhouse-opik-clickhouse -- \
- [Check backup server logs](check-backup-server-logs.md): kubectl logs -l app=clickhouse-backup-server
- [Test API connectivity](test-api-connectivity.md): kubectl port-forward svc/clickhouse-opik-clickhouse 7171:7171
- [Example Prometheus alerts](example-prometheus-alerts.md): - alert: ClickHouseBackupFailed
- [Troubleshooting](troubleshooting-3.md): Troubleshooting guide for common issues when running self-hosted Opik deployments.
- [Connect to Zookeeper pod](connect-to-zookeeper-pod.md): kubectl exec -it cometml-production-opik-zookeeper-0 -- zkCli.sh -server localhost:2181
- [Delete all ClickHouse table paths](delete-all-clickhouse-table-paths.md): deleteall /clickhouse/tables
- [Connect to the first ClickHouse replica](connect-to-the-first-clickhouse-replica.md): kubectl exec -it chi-opik-clickhouse-cluster-0-0-0 -- clickhouse-client
- [Connect to Zookeeper CLI](connect-to-zookeeper-cli.md): kubectl exec -it cometml-production-opik-zookeeper-0 -- zkCli.sh -server localhost:2181
- [List tables (example path - adjust for your database name)](list-tables-example-path-adjust-for-your-database-name.md): ls /clickhouse/tables/0//
- [Connect to first replica](connect-to-first-replica.md): kubectl exec -it chi-opik-clickhouse-cluster-0-0-0 -- clickhouse-client
- [Connect to second replica (if running multiple replicas)](connect-to-second-replica-if-running-multiple-replicas.md): kubectl exec -it chi-opik-clickhouse-cluster-0-1-0 -- clickhouse-client
- [Connect to Zookeeper pod](connect-to-zookeeper-pod-2.md): kubectl exec -it cometml-production-opik-zookeeper-0 -- bash
- [Run Zookeeper client commands](run-zookeeper-client-commands.md): zkCli.sh -server localhost:2181
- [List tables in Zookeeper](list-tables-in-zookeeper.md): kubectl exec -it cometml-production-opik-zookeeper-0 -- \
- [Remove a specific table from Zookeeper](remove-a-specific-table-from-zookeeper.md): kubectl exec -it cometml-production-opik-zookeeper-0 -- \
- [Anonymous usage statistics](anonymous-usage-statistics.md): Learn how to enable or disable anonymous usage statistics reporting in Opik to improve platform performance and user ...
- [Large CSV uploads](large-csv-uploads.md): Configure Opik to accept CSV uploads up to 2GB, enabling efficient dataset creation and asynchronous processing.
- [Enabling Large CSV Uploads](enabling-large-csv-uploads.md): By default, Opik supports CSV file uploads up to 20MB for dataset creation. For self-hosted deployments that need to ...
- [values.yaml](valuesyaml.md): component:
- [Kubernetes](kubernetes.md): kubectl logs -n opik deployment/opik-backend -f | grep CSV
- [Docker Compose](docker-compose.md): docker-compose logs -f backend | grep CSV
- [Dataset versioning migration](dataset-versioning-migration.md): Learn about the automated dataset versioning migration process when upgrading to Opik 1.9.92 or higher, including con...
- [Dataset Versioning Migration](dataset-versioning-migration-2.md): Starting with Opik version`1.9.92`, the platform includes an **automated migration process** to support enhanced dat...
- [Disable after successful migration](disable-after-successful-migration.md): DATASET_VERSION_ITEMS_TOTAL_MIGRATION_ENABLED: false
- [Kubernetes](kubernetes-2.md): kubectl logs -n opik deployment/opik-backend -f | grep -i "migration"
- [Docker Compose](docker-compose-2.md): docker-compose logs -f backend | grep -i "migration"
- [Docker Compose](docker-compose-3.md): services:
- [Kubernetes/Helm](kuberneteshelm.md): component:
- [Kubernetes](kubernetes-3.md): kubectl logs -n opik deployment/opik-backend -f | grep -i "migration"
- [Docker Compose](docker-compose-4.md): docker-compose logs -f backend | grep -i "migration"
- [Overview](overview-7.md): Explore comprehensive SDK reference documentation with Opik to effectively implement and utilize all available methods.
- [REST API Client](rest-api-client.md): Access all Opik platform functionality with the Python SDK's REST API client for advanced operations and custom integ...
- [Access the REST API client](access-the-rest-api-client.md): rest_client = client.rest_client
- [Now you can use any of the available client methods](now-you-can-use-any-of-the-available-client-methods.md): traces = rest_client.traces.search_traces(project_name="my-project")
- [Get a specific trace by ID](get-a-specific-trace-by-id.md): trace = client.rest_client.traces.get_trace_by_id("trace-id")
- [Search traces with advanced filters](search-traces-with-advanced-filters.md): traces = client.rest_client.traces.search_traces(
- [Add feedback to a trace](add-feedback-to-a-trace.md): client.rest_client.traces.add_trace_feedback_score(
- [List all datasets with pagination](list-all-datasets-with-pagination.md): datasets = client.rest_client.datasets.find_datasets(
- [Create a new dataset](create-a-new-dataset.md): dataset = client.rest_client.datasets.create_dataset(
- [Add items to the dataset](add-items-to-the-dataset.md): items = [
- [Create an experiment linked to a dataset](create-an-experiment-linked-to-a-dataset.md): experiment = client.rest_client.experiments.create_experiment(
- [Add experiment results](add-experiment-results.md): client.rest_client.experiments.create_experiment_items(
- [Example paginated response](example-paginated-response.md): response = client.rest_client.datasets.find_datasets(page=0, size=10)
- [Access the response data](access-the-response-data.md): datasets = response.content # List of dataset objects
- [Opik TS - Interactive Setup Tool](opik-ts-interactive-setup-tool.md): A guided CLI tool for quickly integrating Opik SDK into your Node.js/TypeScript projects with automatic configuration...
- [Opik Configuration](opik-configuration-7.md): OPIK_API_KEY=your-api-key-here
- [Use --force-install to bypass peer dependency checks](use-force-install-to-bypass-peer-dependency-checks.md): npx opik-ts configure --force-install
- [In your CI/CD pipeline](in-your-cicd-pipeline.md): npx opik-ts configure --default --force-install
- [Prompts](prompts.md): Learn to version, store, and format LLM prompt templates using the Opik TypeScript SDK for seamless integration with ...
- [Opik Query Language (OQL)](opik-query-language-oql.md): Learn to use OQL for powerful data filtering in Opik with SQL-like syntax and expressive conditions for precise data ...
- [Evaluation](evaluation.md): Evaluate your LLM applications effectively using Opik's TypeScript SDK and the powerful evaluate function for diverse...
- [Quick Start](quick-start.md): Evaluate your AI models effectively. Learn to create datasets and analyze results with Opik's TypeScript SDK in just ...
- [Datasets](datasets.md): Learn to create and manage datasets in Opik for effective evaluation, leveraging TypeScript's type safety and dedupli...
- [Evaluate Function](evaluate-function.md): Evaluate LLM tasks against datasets using customizable metrics with the evaluate function in Opik for precise assessm...
- [evaluatePrompt Function](evaluateprompt-function.md): Evaluate prompt templates against datasets effortlessly with Opik. Streamline formatting, generate responses, and ass...
- [Models](models.md): Configure models for evaluation using the TypeScript SDK with Opik, enabling seamless integration with various provid...
- [Google](google.md): export GOOGLE_API_KEY="..."
- [Evaluation Metrics](evaluation-metrics.md): Evaluate AI model performance with Opik metrics to ensure accurate assessments and informed decisions.
- [Experiments](experiments.md): Evaluate and compare LLM applications with Opik by linking execution data to datasets for structured analysis and per...
- [Overview](overview-8.md): Explore the Opik REST API to integrate seamlessly with Open-Source and Opik Cloud platforms. Engage with our communit...
- [Get datasets information for BI events](get-datasets-information-for-bi-events.md): GET
- [Get experiments information for BI events](get-experiments-information-for-bi-events.md): GET
- [Get spans information for BI events](get-spans-information-for-bi-events.md): GET
- [Get spans count on previous day for all available workspaces](get-spans-count-on-previous-day-for-all-available-workspaces.md): GET
- [Get traces information for BI events](get-traces-information-for-bi-events.md): GET
- [Get traces count on previous day for all available workspaces](get-traces-count-on-previous-day-for-all-available-workspaces.md): GET
- [Find alerts](find-alerts.md): GET
- [Create alert](create-alert.md): POST
- [Delete alert batch](delete-alert-batch.md): POST
- [Get Alert by id](get-alert-by-id.md): GET
- [Update alert](update-alert.md): PUT
- [Get webhook payload examples](get-webhook-payload-examples.md): GET
- [Test alert webhook](test-alert-webhook.md): POST
- [Add items to annotation queue](add-items-to-annotation-queue.md): POST
- [Find annotation queues](find-annotation-queues.md): GET
- [Create annotation queue](create-annotation-queue.md): POST
- [Create annotation queue batch](create-annotation-queue-batch.md): POST
- [Delete annotation queue batch](delete-annotation-queue-batch.md): POST
- [Get annotation queue by id](get-annotation-queue-by-id.md): GET
- [Update annotation queue](update-annotation-queue.md): PATCH
- [Remove items from annotation queue](remove-items-from-annotation-queue.md): POST
- [Attachments list for entity](attachments-list-for-entity.md): GET
- [Complete multipart attachment upload](complete-multipart-attachment-upload.md): POST
- [Delete attachments](delete-attachments.md): POST
- [Download attachment from MinIO](download-attachment-from-minio.md): GET
- [Start multipart attachment upload](start-multipart-attachment-upload.md): POST
- [Upload attachment to MinIO](upload-attachment-to-minio.md): PUT
- [Check user access to workspace](check-user-access-to-workspace.md): POST
- [User's default workspace name](users-default-workspace-name.md): GET
- [Find project Evaluators](find-project-evaluators.md): GET
- [Create automation rule evaluator](create-automation-rule-evaluator.md): POST
- [Delete automation rule evaluators](delete-automation-rule-evaluators.md): POST
- [Get automation rule evaluator by id](get-automation-rule-evaluator-by-id.md): GET
- [Update Automation Rule Evaluator by id](update-automation-rule-evaluator-by-id.md): PATCH
- [Get automation rule evaluator logs by id](get-automation-rule-evaluator-logs-by-id.md): GET
- [Create chat completions](create-chat-completions.md): POST
- [Find dashboards](find-dashboards.md): GET
- [Create dashboard](create-dashboard.md): POST
- [Get dashboard by id](get-dashboard-by-id.md): GET
- [Delete dashboard](delete-dashboard.md): DELETE
- [Update dashboard](update-dashboard.md): PATCH
- [Delete dashboards](delete-dashboards.md): POST
- [Apply changes to dataset items](apply-changes-to-dataset-items.md): POST
- [Batch update dataset items](batch-update-dataset-items.md): PATCH
- [Find datasets](find-datasets.md): GET
- [Create dataset](create-dataset-5.md): POST
- [Create/update dataset items](createupdate-dataset-items.md): PUT
- [Create dataset items from CSV file](create-dataset-items-from-csv-file.md): POST
- [Create dataset items from spans](create-dataset-items-from-spans.md): POST
- [Create dataset items from traces](create-dataset-items-from-traces.md): POST
- [Get dataset by id](get-dataset-by-id.md): GET
- [Update dataset by id](update-dataset-by-id.md): PUT
- [Delete dataset by id](delete-dataset-by-id.md): DELETE
- [Delete dataset by name](delete-dataset-by-name.md): POST
- [Delete dataset items](delete-dataset-items.md): POST
- [Delete datasets](delete-datasets.md): POST
- [Download dataset export file](download-dataset-export-file.md): GET
- [Expand dataset with synthetic samples](expand-dataset-with-synthetic-samples.md): POST
- [Find dataset items with experiment items](find-dataset-items-with-experiment-items.md): GET
- [Get dataset by name](get-dataset-by-name.md): POST
- [Get experiment items stats for dataset](get-experiment-items-stats-for-dataset.md): GET
- [Get dataset export job status](get-dataset-export-job-status.md): GET
- [Get all dataset export jobs](get-all-dataset-export-jobs.md): GET
- [Get dataset item by id](get-dataset-item-by-id.md): GET
- [Partially update dataset item by id](partially-update-dataset-item-by-id.md): PATCH
- [Get dataset items](get-dataset-items.md): GET
- [Get dataset items output columns](get-dataset-items-output-columns.md): GET
- [Mark dataset export job as viewed](mark-dataset-export-job-as-viewed.md): PUT
- [Start dataset CSV export](start-dataset-csv-export.md): POST
- [Stream dataset items](stream-dataset-items.md): POST
- [Compare latest version with draft](compare-latest-version-with-draft.md): GET
- [Create version tag](create-version-tag.md): POST
- [Delete version tag](delete-version-tag.md): DELETE
- [List dataset versions](list-dataset-versions.md): GET
- [Restore dataset to a previous version](restore-dataset-to-a-previous-version.md): POST
- [Update dataset version](update-dataset-version.md): PATCH
- [Batch update experiments](batch-update-experiments.md): PATCH
- [Find experiments](find-experiments.md): GET
- [Create experiment](create-experiment.md): POST
- [Create experiment items](create-experiment-items.md): POST
- [Delete experiment items](delete-experiment-items.md): POST
- [Delete experiments by id](delete-experiments-by-id.md): POST
- [Record experiment items in bulk](record-experiment-items-in-bulk.md): PUT
- [Find Feedback Score names](find-feedback-score-names.md): GET
- [Find experiment groups](find-experiment-groups.md): GET
- [Find experiment groups with aggregations](find-experiment-groups-with-aggregations.md): GET
- [Finish experiments](finish-experiments.md): POST
- [Get experiment by id](get-experiment-by-id.md): GET
- [Update experiment by id](update-experiment-by-id.md): PATCH
- [Get experiment item by id](get-experiment-item-by-id.md): GET
- [Stream experiment items](stream-experiment-items.md): POST
- [Stream experiments](stream-experiments.md): POST
- [Find Feedback definitions](find-feedback-definitions.md): GET
- [Create feedback definition](create-feedback-definition.md): POST
- [Get feedback definition by id](get-feedback-definition-by-id.md): GET
- [Update feedback definition by id](update-feedback-definition-by-id.md): PUT
- [Delete feedback definition by id](delete-feedback-definition-by-id.md): DELETE
- [Delete feedback definitions](delete-feedback-definitions.md): POST
- [Create guardrails for traces in a batch](create-guardrails-for-traces-in-a-batch.md): POST
- [Delete LLM Provider's ApiKeys](delete-llm-providers-apikeys.md): POST
- [Find LLM Provider's ApiKeys](find-llm-providers-apikeys.md): GET
- [Store LLM Provider's ApiKey](store-llm-providers-apikey.md): POST
- [Get LLM Provider's ApiKey by id](get-llm-providers-apikey-by-id.md): GET
- [Update LLM Provider's ApiKey](update-llm-providers-apikey.md): PATCH
- [Manually evaluate spans](manually-evaluate-spans.md): POST
- [Manually evaluate threads](manually-evaluate-threads.md): POST
- [Manually evaluate traces](manually-evaluate-traces.md): POST
- [List available Ollama models](list-available-ollama-models.md): POST
- [Test connection to Ollama instance](test-connection-to-ollama-instance.md): POST
- [Receive Protobuf Traces](receive-protobuf-traces.md): POST
- [Cancel Studio optimizations](cancel-studio-optimizations.md): GET
- [Find optimizations](find-optimizations.md): GET
- [Create optimization](create-optimization.md): POST
- [Upsert optimization](upsert-optimization.md): PUT
- [Delete optimizations by id](delete-optimizations-by-id.md): POST
- [Get optimization by id](get-optimization-by-id.md): GET
- [Update optimization by id](update-optimization-by-id.md): PUT
- [Get Studio optimization logs](get-studio-optimization-logs.md): GET
- [Find projects](find-projects.md): GET
- [Create project](create-project.md): POST
- [Get project by id](get-project-by-id.md): GET
- [Delete project by id](delete-project-by-id.md): DELETE
- [Update project by id](update-project-by-id.md): PATCH
- [Delete projects](delete-projects.md): POST
- [Find Feedback Score names By Project Ids](find-feedback-score-names-by-project-ids.md): GET
- [Find Token Usage names](find-token-usage-names.md): GET
- [Get Project Metrics](get-project-metrics.md): POST
- [Get Project Stats](get-project-stats.md): GET
- [Retrieve project](retrieve-project.md): POST
- [Get prompts](get-prompts.md): GET
- [Create prompt](create-prompt-2.md): POST
- [Create prompt version](create-prompt-version.md): POST
- [Update prompt versions](update-prompt-versions.md): PATCH
- [Get prompt by id](get-prompt-by-id.md): GET
- [Update prompt](update-prompt.md): PUT
- [Delete prompt](delete-prompt.md): DELETE
- [Delete prompts](delete-prompts.md): POST
- [Get prompt version by id](get-prompt-version-by-id.md): GET
- [Get prompt versions](get-prompt-versions.md): GET
- [Restore prompt version](restore-prompt-version.md): POST
- [Retrieve prompt version](retrieve-prompt-version.md): POST
- [Get Service Toggles](get-service-toggles.md): GET
- [Add span comment](add-span-comment.md): POST
- [Add span feedback score](add-span-feedback-score.md): PUT
- [Create spans](create-spans.md): POST
- [Batch update spans](batch-update-spans.md): PATCH
- [Get spans by project_name or project_id and optionally by trace_id and/or type](get-spans-by-project-name-or-project-id-and-optionally-by-trace-id-andor-type.md): GET
- [Create span](create-span.md): POST
- [Get span by id](get-span-by-id.md): GET
- [Delete span by id](delete-span-by-id.md): DELETE
- [Update span by id](update-span-by-id.md): PATCH
- [Delete span comments](delete-span-comments.md): POST
- [Delete span feedback score](delete-span-feedback-score.md): POST
- [Find Feedback Score names](find-feedback-score-names-2.md): GET
- [Get span comment](get-span-comment.md): GET
- [Get span stats](get-span-stats.md): GET
- [Batch feedback scoring for spans](batch-feedback-scoring-for-spans.md): PUT
- [Search spans](search-spans.md): POST
- [Update span comment by id](update-span-comment-by-id.md): PATCH
- [Add thread comment](add-thread-comment.md): POST
- [Add trace comment](add-trace-comment.md): POST
- [Add trace feedback score](add-trace-feedback-score.md): PUT
- [Create traces](create-traces.md): POST
- [Batch update traces](batch-update-traces.md): PATCH
- [Batch update threads](batch-update-threads.md): PATCH
- [Close trace thread(s)](close-trace-threads.md): PUT
- [Get traces by project_name or project_id](get-traces-by-project-name-or-project-id.md): GET
- [Create trace](create-trace.md): POST
- [Get trace by id](get-trace-by-id.md): GET
- [Delete trace by id](delete-trace-by-id.md): DELETE
- [Update trace by id](update-trace-by-id.md): PATCH
- [Delete thread comments](delete-thread-comments.md): POST
- [Delete thread feedback scores](delete-thread-feedback-scores.md): POST
- [Delete trace comments](delete-trace-comments.md): POST
- [Delete trace feedback score](delete-trace-feedback-score.md): POST
- [Delete trace threads](delete-trace-threads.md): POST
- [Delete traces](delete-traces.md): POST
- [Find Feedback Score names](find-feedback-score-names-3.md): GET
- [Find Trace Threads Feedback Score names](find-trace-threads-feedback-score-names.md): GET
- [Get trace stats](get-trace-stats.md): GET
- [Get thread comment](get-thread-comment.md): GET
- [Get trace thread stats](get-trace-thread-stats.md): GET
- [Get trace comment](get-trace-comment.md): GET
- [Get trace thread](get-trace-thread.md): POST
- [Get trace threads](get-trace-threads.md): GET
- [Open trace thread](open-trace-thread.md): PUT
- [Batch feedback scoring for threads](batch-feedback-scoring-for-threads.md): PUT
- [Batch feedback scoring for traces](batch-feedback-scoring-for-traces.md): PUT
- [Search trace threads](search-trace-threads.md): POST
- [Search traces](search-traces.md): POST
- [Update thread](update-thread.md): PATCH
- [Update thread comment by id](update-thread-comment-by-id.md): PATCH
- [Update trace comment by id](update-trace-comment-by-id.md): PATCH
- [Get welcome wizard tracking status](get-welcome-wizard-tracking-status.md): GET
- [Submit welcome wizard](submit-welcome-wizard.md): POST
- [Get costs summary](get-costs-summary.md): POST
- [Get workspace configuration](get-workspace-configuration.md): GET
- [Upsert workspace configuration](upsert-workspace-configuration.md): PUT
- [Delete workspace configuration](delete-workspace-configuration.md): DELETE
- [Get cost daily data](get-cost-daily-data.md): POST
- [Get metric daily data](get-metric-daily-data.md): POST
- [Get metrics summary](get-metrics-summary.md): POST
- [Create dataset redirect url](create-dataset-redirect-url.md): GET
- [Create experiment redirect url](create-experiment-redirect-url.md): GET
- [Create optimization redirect url](create-optimization-redirect-url.md): GET
- [Create project redirect url](create-project-redirect-url.md): GET
- [Overview](overview-9.md): Master Opik's features through structured video tutorials, enhancing your observability and monitoring capabilities f...
- [Welcome to Opik University](welcome-to-opik-university.md): Opik University is your home for learning how to get the most out of Opik. Here you'll find a collection of video tut...
- [Opik Overview](opik-overview.md): Explore Opik's capabilities to enhance your workflow and maximize productivity with practical insights and an engagin...
- [Next steps / Set expectations](next-steps-set-expectations.md): Learn to set clear expectations for your project with Opik, ensuring alignment and understanding among all stakeholders.
- [Log Traces](log-traces-2.md): Monitor and analyze log traces effectively with Opik to improve system observability and troubleshoot issues efficien...
- [Annotate Traces](annotate-traces.md): Learn to effectively annotate traces in Opik to improve observability and gain deeper insights into your applications.
- [Evaluation Concepts and Overview](evaluation-concepts-and-overview.md): Explore evaluation concepts in Opik to enhance your understanding and implementation strategies for effective assessm...
- [Create Evaluation Datasets](create-evaluation-datasets.md): Build and manage evaluation datasets effectively using Opik's tools to enhance your analysis and decision-making proc...
- [Define Evaluation Metrics](define-evaluation-metrics.md): Learn to define and implement effective evaluation metrics in Opik for better performance analysis and decision-making.
- [Evaluate your LLM Application](evaluate-your-llm-application.md): Evaluate your LLM application effectively with Opik's comprehensive evaluation tools and insights for optimized perfo...
- [No-code LLM Evaluation Workflow](no-code-llm-evaluation-workflow.md): Evaluate LLM performance effortlessly with Opik's no-code workflow, streamlining your machine learning evaluation pro...
- [Prompt Management](prompt-management-2.md): Learn to effectively manage and optimize prompts in Opik to enhance your AI-driven projects and streamline workflows.
- [Prompt Playground](prompt-playground-2.md): Learn to craft effective prompts for AI interactions in Opik's Prompt Playground. Enhance your prompt engineering ski...
- [PyTest Integration](pytest-integration-2.md): Learn to seamlessly integrate PyTest in Opik to enhance your testing workflows and ensure reliable software performance.
- [Online Evaluation Rules](online-evaluation-rules-2.md): Monitor and evaluate production processes effectively with Opik's comprehensive online evaluation rules.