# Ray

> You can post questions or issues or feedback through the following channels:

## Pages

- [_Help](includes-help.md): You can post questions or issues or feedback through the following channels:
- [_Latest_Contribution_Doc](includes-latest-contribution-doc.md): .. admonition:: Check your version!
- [New_Api_Stack](includes-rllib-new-api-stack.md): ..
- [We_Are_Hiring](includes-rllib-we-are-hiring.md): .. TODO: we comment out the hiring message, as it's too much with the RL conf announcement.
- [Cli](cluster-cli.md): .. _ray-cluster-cli:
- [Faq](cluster-faq.md): .. _cluster-FAQ:
- [Getting Started](cluster-getting-started.md): .. _cluster-index:
- [Key Concepts](cluster-key-concepts.md): Key Concepts
- [Gpu](cluster-kubernetes-user-guides-gpu.md): .. _kuberay-gpu:
- [Package Overview](cluster-package-overview.md): .. _cluster-api-ref:
- [Reference](cluster-running-applications-autoscaling-reference.md): .. _ref-autoscaler-sdk:
- [Cli](cluster-running-applications-job-submission-cli.md): .. _ray-job-submission-cli-ref:
- [Jobs Package Ref](cluster-running-applications-job-submission-jobs-package-ref.md): .. _ray-job-submission-sdk-ref:
- [Quickstart](cluster-running-applications-job-submission-quickstart.md): .. _jobs-quickstart:
- [Ray Client](cluster-running-applications-job-submission-ray-client.md): .. _ray-client-ref:
- [Rest](cluster-running-applications-job-submission-rest.md): .. _ray-job-rest-api:
- [Sdk](cluster-running-applications-job-submission-sdk.md): .. _ray-job-sdk:
- [Usage Stats](cluster-usage-stats.md): .. _ref-usage-stats:
- [Getting Started](cluster-vms-getting-started.md): .. _vm-cluster-quick-start:
- [Ray Cluster Cli](cluster-vms-references-ray-cluster-cli.md): .. _cluster-commands:
- [Ray Cluster Configuration](cluster-vms-references-ray-cluster-configuration.md): .. _cluster-config:
- [Index](cluster-vms-user-guides-community.md): .. _ref-cluster-setup:
- [Lsf](cluster-vms-user-guides-community-lsf.md): .. _ray-LSF-deploy:
- [Slurm Basic](cluster-vms-user-guides-community-slurm-basic.md): :orphan:
- [Slurm Launch](cluster-vms-user-guides-community-slurm-launch.md): :orphan:
- [Slurm Template](cluster-vms-user-guides-community-slurm-template.md): :orphan:
- [Slurm](cluster-vms-user-guides-community-slurm.md): .. _ray-slurm-deploy:
- [Spark](cluster-vms-user-guides-community-spark.md): .. _ray-Spark-deploy:
- [Yarn](cluster-vms-user-guides-community-yarn.md): .. _ray-yarn-deploy:
- [Configuring Autoscaling](cluster-vms-user-guides-configuring-autoscaling.md): .. _vms-autoscaling:
- [Large Cluster Best Practices](cluster-vms-user-guides-large-cluster-best-practices.md): .. _vms-large-cluster:
- [Index](cluster-vms-user-guides-launching-clusters.md): .. _launching-vm-clusters:
- [Aggregating Data](data-aggregating-data.md): .. _aggregations:
- [_Autogen](data-api-autogen.md): :orphan:
- [Aggregate](data-api-aggregate.md): .. _aggregations_api_ref:
- [Api](data-api-api.md): .. _data-api:
- [Data_Context](data-api-data-context.md): .. _data-context-api:
- [Data_Iterator](data-api-data-iterator.md): .. _dataset-iterator-api:
- [Dataset](data-api-dataset.md): .. _dataset-api:
- [Datatype](data-api-datatype.md): .. _datatype-api:
- [Execution_Options](data-api-execution-options.md): .. _execution-options-api:
- [Expressions](data-api-expressions.md): .. _expressions-api:
- [From_Other_Data_Libs](data-api-from-other-data-libs.md): .. _api-guide-for-users-from-other-data-libs:
- [Grouped_Data](data-api-grouped-data.md): .. _grouped-dataset-api:
- [Input_Output](data-api-input-output.md): .. _input-output:
- [Llm](data-api-llm.md): .. _llm-ref:
- [Preprocessor](data-api-preprocessor.md): .. _preprocessor-ref:
- [Batch_Inference](data-batch-inference.md): .. _batch_inference_home:
- [Comparing Ray Data to other systems](data-comparisons.md): Comparing Ray Data to other systems
- [Contributing](data-contributing-contributing.md): ========================
- [Custom Datasource Example](data-custom-datasource-example.md): .. _custom_datasource:
- [Data Internals](data-data-internals.md): .. _datasets_scheduling:
- [Data](data-data.md): .. _data:
- [Execution Configurations](data-execution-configurations.md): .. _execution_configurations:
- [Inspecting Data](data-inspecting-data.md): .. _inspecting-data:
- [Iterating Over Data](data-iterating-over-data.md): .. _iterating-over-data:
- [Joining Data](data-joining-data.md): .. _joining-data:
- [Key Concepts](data-key-concepts.md): .. _data_key_concepts:
- [Loading Data](data-loading-data.md): .. _loading_data:
- [Monitoring Your Workload](data-monitoring-your-workload.md): .. _monitoring-your-workload:
- [Performance Tips](data-performance-tips.md): .. _data_performance_tips:
- [Quickstart](data-quickstart.md): .. _data_quickstart:
- [Saving Data](data-saving-data.md): .. _saving-data:
- [Shuffling Data](data-shuffling-data.md): .. _shuffling_data:
- [Transforming Data](data-transforming-data.md): .. _transforming_data:
- [User Guide](data-user-guide.md): .. _data_user_guide:
- [Working With Images](data-working-with-images.md): .. _working_with_images:
- [Working With Llms](data-working-with-llms.md): .. _working-with-llms:
- [Working With Pytorch](data-working-with-pytorch.md): .. _working_with_pytorch:
- [Working With Tensors](data-working-with-tensors.md): .. _working_with_tensors:
- [Working with Text](data-working-with-text.md): Working with Text
- [Index](index.md): :html_theme.sidebar_secondary.remove:
- [Deploying Ray for ML platforms](ray-air-deployment.md): Deploying Ray for ML platforms
- [Getting Started](ray-air-getting-started.md): .. _ray-for-ml-infra:
- [Api Policy](ray-contribute-api-policy.md): .. _api-policy:
- [CI Testing Workflow on PRs](ray-contribute-ci.md): CI Testing Workflow on PRs
- [Debugging for Ray Developers](ray-contribute-debugging.md): Debugging for Ray Developers
- [Development](ray-contribute-development.md): .. _building-ray:
- [Fake Autoscaler](ray-contribute-fake-autoscaler.md): .. _fake-multinode:
- [Getting Involved](ray-contribute-getting-involved.md): .. include:: /_includes/_latest_contribution_doc.rst
- [Developer Guides](ray-contribute.md): Developer Guides
- [Involvement](ray-contribute-involvement.md): Ray is more than a framework for distributed applications but also an active community of developers,
- [Profiling](ray-contribute-profiling.md): .. _ray-core-internal-profiling:
- [Stability](ray-contribute-stability.md): .. _api-stability:
- [Tips for testing Ray programs](ray-contribute-testing-tips.md): Tips for testing Ray programs
- [Whitepaper](ray-contribute-whitepaper.md): .. _whitepaper:
- [Writing Code Snippets](ray-contribute-writing-code-snippets.md): .. _writing-code-snippets_ref:
- [Accelerator Types](ray-core-accelerator-types.md): :orphan:
- [Utility Classes](ray-core-actors-actor-utils.md): Utility Classes
- [AsyncIO / Concurrency for Actors](ray-core-actors-async-api.md): AsyncIO / Concurrency for Actors
- [Limiting Concurrency Per-Method with Concurrency Groups](ray-core-actors-concurrency-group-api.md): Limiting Concurrency Per-Method with Concurrency Groups
- [Named Actors](ray-core-actors-named-actors.md): Named Actors
- [Out-of-band Communication](ray-core-actors-out-of-band-communication.md): Out-of-band Communication
- [Task Orders](ray-core-actors-task-orders.md): .. _actor-task-order:
- [Terminating Actors](ray-core-actors-terminating-actors.md): Terminating Actors
- [Actors](ray-core-actors.md): .. _ray-remote-classes:
- [Advanced topics](ray-core-advanced-topics.md): Advanced topics
- [Ray Core CLI](ray-core-api-cli.md): Ray Core CLI
- [Core API](ray-core-api-core.md): Core API
- [Ray Direct Transport (RDT) API](ray-core-api-direct-transport.md): Ray Direct Transport (RDT) API
- [Exceptions](ray-core-api-exceptions.md): .. _ray-core-exceptions:
- [Ray Core API](ray-core-api.md): Ray Core API
- [Runtime Env API](ray-core-api-runtime-env.md): Runtime Env API
- [Scheduling API](ray-core-api-scheduling.md): Scheduling API
- [Utility](ray-core-api-utility.md): Utility
- [Compiled Graph API](ray-core-compiled-graph-compiled-graph-api.md): Compiled Graph API
- [Overlap](ray-core-compiled-graph-overlap.md): .. _compiled-graph-overlap:
- [Profiling](ray-core-compiled-graph-profiling.md): Profiling
- [Quickstart](ray-core-compiled-graph-quickstart.md): Quickstart
- [Ray Compiled Graph](ray-core-compiled-graph-ray-compiled-graph.md): .. _ray-compiled-graph:
- [Troubleshooting](ray-core-compiled-graph-troubleshooting.md): Troubleshooting
- [Alternatively, this command automatically encode and create the secret for the CA key pair.](ray-core-configure.md): .. _configuring-ray:
- [Cross Language](ray-core-cross-language.md): .. _cross_language:
- [Direct Transport](ray-core-direct-transport.md): .. _direct-transport:
- [Monte_Carlo_Pi](ray-core-examples-monte-carlo-pi.md): .. _monte-carlo-pi:
- [Overview](ray-core-examples-overview.md): .. _ray-core-examples-tutorial:
- [Fault Tolerance](ray-core-fault-tolerance.md): .. _fault-tolerance:
- [Actors](ray-core-fault-tolerance-actors.md): .. _fault-tolerance-actors:
- [Gcs](ray-core-fault-tolerance-gcs.md): .. _fault-tolerance-gcs:
- [Nodes](ray-core-fault-tolerance-nodes.md): .. _fault-tolerance-nodes:
- [Objects](ray-core-fault-tolerance-objects.md): .. _fault-tolerance-objects:
- [Tasks](ray-core-fault-tolerance-tasks.md): .. _fault-tolerance-tasks:
- [Handling Dependencies](ray-core-handling-dependencies.md): .. _handling_dependencies:
- [Autoscaler V2](ray-core-internals-autoscaler-v2.md): .. _autoscaler-v2:
- [Metric Exporter](ray-core-internals-metric-exporter.md): .. _metric-exporter:
- [Ray Event Exporter](ray-core-internals-ray-event-exporter.md): .. _ray-event-exporter:
- [Rpc Fault Tolerance](ray-core-internals-rpc-fault-tolerance.md): .. _rpc-fault-tolerance:
- [Task Lifecycle](ray-core-internals-task-lifecycle.md): .. _task-lifecycle:
- [Token Authentication](ray-core-internals-token-authentication.md): .. _token-authentication:
- [Internals](ray-core-internals.md): .. _ray-core-internals:
- [Key Concepts](ray-core-key-concepts.md): .. _core-key-concepts:
- [Miscellaneous Topics](ray-core-miscellaneous.md): Miscellaneous Topics
- [Namespaces](ray-core-namespaces.md): .. _namespaces-guide:
- [Object Spilling](ray-core-objects-object-spilling.md): Object Spilling
- [Serialization](ray-core-objects-serialization.md): .. _serialization-guide:
- [Objects](ray-core-objects.md): .. _objects-in-ray:
- [Pattern: Using an actor to synchronize other tasks and actors](ray-core-patterns-actor-sync.md): Pattern: Using an actor to synchronize other tasks and actors
- [Anti-pattern: Closure capturing large objects harms performance](ray-core-patterns-closure-capture-large-objects.md): Anti-pattern: Closure capturing large objects harms performance
- [Pattern: Using asyncio to run actor methods concurrently](ray-core-patterns-concurrent-operations-async-actor.md): Pattern: Using asyncio to run actor methods concurrently
- [Fork New Processes](ray-core-patterns-fork-new-processes.md): .. _forking-ray-processes-antipattern:
- [Generators](ray-core-patterns-generators.md): .. _generator-pattern:
- [Anti-pattern: Using global variables to share state between tasks and actors](ray-core-patterns-global-variables.md): Anti-pattern: Using global variables to share state between tasks and actors
- [Index](ray-core-patterns.md): .. _core-patterns:
- [Limit Pending Tasks](ray-core-patterns-limit-pending-tasks.md): .. _core-patterns-limit-pending-tasks:
- [Limit Running Tasks](ray-core-patterns-limit-running-tasks.md): .. _core-patterns-limit-running-tasks:
- [Nested Ray Get](ray-core-patterns-nested-ray-get.md): .. _nested-ray-get:
- [Nested Tasks](ray-core-patterns-nested-tasks.md): .. _nested-tasks:
- [Out Of Band Object Ref Serialization](ray-core-patterns-out-of-band-object-ref-serialization.md): .. _ray-out-of-band-object-ref-serialization:
- [Pass Large Arg By Value](ray-core-patterns-pass-large-arg-by-value.md): .. _ray-pass-large-arg-by-value:
- [Pattern: Using pipelining to increase throughput](ray-core-patterns-pipelining.md): Pattern: Using pipelining to increase throughput
- [Ray Get Loop](ray-core-patterns-ray-get-loop.md): .. _ray-get-loop:
- [Anti-pattern: Processing results in submission order using ray.get increases runtime](ray-core-patterns-ray-get-submission-order.md): Anti-pattern: Processing results in submission order using ray.get increases runtime
- [Ray Get Too Many Objects](ray-core-patterns-ray-get-too-many-objects.md): .. _ray-get-too-many-objects:
- [Anti-pattern: Redefining the same remote function or class harms performance](ray-core-patterns-redefine-task-actor-loop.md): Anti-pattern: Redefining the same remote function or class harms performance
- [Anti-pattern: Returning ray.put() ObjectRefs from a task harms performance and fault tolerance](ray-core-patterns-return-ray-put.md): Anti-pattern: Returning ray.put() ObjectRefs from a task harms performance and fault tolerance
- [Anti-pattern: Over-parallelizing with too fine-grained tasks harms speedup](ray-core-patterns-too-fine-grained-tasks.md): Anti-pattern: Over-parallelizing with too fine-grained tasks harms speedup
- [Pattern: Using a supervisor actor to manage a tree of actors](ray-core-patterns-tree-of-actors.md): Pattern: Using a supervisor actor to manage a tree of actors
- [Unnecessary Ray Get](ray-core-patterns-unnecessary-ray-get.md): .. _unnecessary-ray-get:
- [Ray Dag](ray-core-ray-dag.md): .. _ray-dag-guide:
- [Ray Generator](ray-core-ray-generator.md): .. _generators:
- [Accelerators](ray-core-scheduling-accelerators.md): .. _gpu-support:
- [Index](ray-core-scheduling.md): .. _ray-scheduling:
- [Memory Management](ray-core-scheduling-memory-management.md): .. _memory:
- [Placement Groups](ray-core-scheduling-placement-group.md): Placement Groups
- [Ray Oom Prevention](ray-core-scheduling-ray-oom-prevention.md): .. _ray-oom-prevention:
- [Resources](ray-core-scheduling-resources.md): .. _core-resources:
- [Starting Ray](ray-core-starting-ray.md): .. _start-ray:
- [Dynamic_Generators](ray-core-tasks-dynamic-generators.md): .. _dynamic_generators:
- [Nested Remote Functions](ray-core-tasks-nested-tasks.md): Nested Remote Functions
- [Tasks](ray-core-tasks.md): .. _ray-remote-functions:
- [Tips for first-time users](ray-core-tips-for-first-time.md): Tips for first-time users
- [User Guide](ray-core-user-guide.md): .. _core-use-guide:
- [Lifetimes of a User-Spawn Process](ray-core-user-spawn-processes.md): Lifetimes of a User-Spawn Process
- [Working with Jupyter Notebooks & JupyterLab](ray-core-using-ray-with-jupyter.md): Working with Jupyter Notebooks & JupyterLab
- [Walkthrough](ray-core-walkthrough.md): .. _core-walkthrough:
- [Dask On Ray](ray-more-libs-dask-on-ray.md): .. _dask-on-ray:
- [More Ray ML Libraries](ray-more-libs.md): More Ray ML Libraries
- [Joblib](ray-more-libs-joblib.md): .. _ray-joblib:
- [Mars On Ray](ray-more-libs-mars-on-ray.md): .. _mars-on-ray:
- [Index](ray-more-libs-modin.md): .. _modin-on-ray:
- [Multiprocessing](ray-more-libs-multiprocessing.md): .. _ray-multiprocessing:
- [Ray Collective](ray-more-libs-ray-collective.md): ..
- [Raydp](ray-more-libs-raydp.md): .. _spark-on-ray:
- [Getting Started](ray-observability-getting-started.md): .. _observability-getting-started:
- [Key Concepts](ray-observability-key-concepts.md): .. _observability-key-concepts:
- [Ray Distributed Debugger](ray-observability-ray-distributed-debugger.md): .. _ray-distributed-debugger:
- [Api](ray-observability-reference-api.md): .. _state-api-ref:
- [Cli](ray-observability-reference-cli.md): .. _state-api-cli-ref:
- [System Metrics](ray-observability-reference-system-metrics.md): .. _system-metrics:
- [Add App Metrics](ray-observability-user-guides-add-app-metrics.md): .. _application-level-metrics:
- [Cli Sdk](ray-observability-user-guides-cli-sdk.md): .. _observability-programmatic:
- [Debug Failures](ray-observability-user-guides-debug-apps-debug-failures.md): .. _observability-debug-failures:
- [Debug Hangs](ray-observability-user-guides-debug-apps-debug-hangs.md): .. _observability-debug-hangs:
- [Debug Memory](ray-observability-user-guides-debug-apps-debug-memory.md): .. _ray-core-mem-profiling:
- [General Debugging](ray-observability-user-guides-debug-apps-general-debugging.md): .. _observability-general-debugging:
- [Optimize Performance](ray-observability-user-guides-debug-apps-optimize-performance.md): .. _observability-optimize-performance:
- [Ray Debugging](ray-observability-user-guides-debug-apps-ray-debugging.md): .. _ray-debugger:
- [Ray Event Export](ray-observability-user-guides-ray-event-export.md): .. _ray-event-export:
- [Ray Tracing](ray-observability-user-guides-ray-tracing.md): .. _ray-tracing:
- [Index](ray-overview-examples.md): .. _ref-overview-examples:
- [Examples](ray-overview-examples-2.md): :orphan:
- [Installation](ray-overview-installation.md): .. _installation:
- [Ray Libraries](ray-overview-ray-libraries.md): .. _ray-oss-list:
- [Use Cases](ray-overview-use-cases.md): .. _ref-use-cases:
- [Faq](ray-references-faq.md): :orphan:
- [Glossary](ray-references-glossary.md): .. _ray_glossary:
- [Algorithm Config](rllib-algorithm-config.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Checkpoints](rllib-checkpoints.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Connector V2](rllib-connector-v2.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Env To Module Connector](rllib-env-to-module-connector.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [External Envs](rllib-external-envs.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Getting Started](rllib-getting-started.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Hierarchical Envs](rllib-hierarchical-envs.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Index](rllib.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Key Concepts](rllib-key-concepts.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Learner Connector](rllib-learner-connector.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Metrics Logger](rllib-metrics-logger.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Multi Agent Envs](rllib-multi-agent-envs.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [New Api Stack Migration Guide](rllib-new-api-stack-migration-guide.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Algorithm Config](rllib-package-ref-algorithm-config.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Algorithm](rllib-package-ref-algorithm.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Callback](rllib-package-ref-callback.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Connector V2](rllib-package-ref-connector-v2.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Distributions](rllib-package-ref-distributions.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Env](rllib-package-ref-env.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Index](rllib-package-ref.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Learner](rllib-package-ref-learner.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Offline](rllib-package-ref-offline.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Replay Buffers](rllib-package-ref-replay-buffers.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Rl_Modules](rllib-package-ref-rl-modules.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Utils](rllib-package-ref-utils.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Rl Modules](rllib-rl-modules.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Rllib Advanced Api](rllib-rllib-advanced-api.md): .. include:: /_includes/rllib/new_api_stack.rst
- [Rllib Algorithms](rllib-rllib-algorithms.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Rllib Callback](rllib-rllib-callback.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Rllib Dev](rllib-rllib-dev.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Rllib Env](rllib-rllib-env.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Rllib Examples](rllib-rllib-examples.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Rllib Fault Tolerance](rllib-rllib-fault-tolerance.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Rllib Learner](rllib-rllib-learner.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Rllib Offline](rllib-rllib-offline.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Rllib Replay Buffers](rllib-rllib-replay-buffers.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Scaling Guide](rllib-scaling-guide.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Single Agent Episode](rllib-single-agent-episode.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [User Guides](rllib-user-guides.md): .. include:: /_includes/rllib/we_are_hiring.rst
- [Api](train-api-api.md): .. _train-api:
- [Deprecated](train-api-deprecated.md): :orphan:
- [Benchmarks](train-benchmarks.md): .. _train-benchmarks:
- [Configure scale and GPUs](train-common-torch-configure-run.md): Configure scale and GPUs
- [Torch Configure Train_Func](train-common-torch-configure-train-func.md): First, update your training code to support distributed training.
- [Deepspeed](train-deepspeed.md): .. _train-deepspeed:
- [Fault Tolerance Deprecated Api](train-deprecated-user-guides-fault-tolerance-deprecated-api.md): :orphan:
- [Hyperparameter Optimization Deprecated](train-deprecated-user-guides-hyperparameter-optimization-deprecated.md): :orphan:
- [Distributed Tensorflow Keras](train-distributed-tensorflow-keras.md): .. _train-tensorflow-overview:
- [Accelerate_Example](train-examples-accelerate-accelerate-example.md): :orphan:
- [Llama3](train-examples-aws-trainium-llama3.md): :orphan:
- [Deepspeed_Example](train-examples-deepspeed-deepspeed-example.md): :orphan:
- [Horovod_Example](train-examples-horovod-horovod-example.md): :orphan:
- [Dreambooth_Finetuning](train-examples-pytorch-dreambooth-finetuning.md): :orphan:
- [Torch_Fashion_Mnist_Example](train-examples-pytorch-torch-fashion-mnist-example.md): :orphan:
- [Torch_Regression_Example](train-examples-pytorch-torch-regression-example.md): :orphan:
- [Tensorflow_Mnist_Example](train-examples-tf-tensorflow-mnist-example.md): :orphan:
- [Tensorflow_Regression_Example](train-examples-tf-tensorflow-regression-example.md): :orphan:
- [Transformers_Torch_Trainer_Basic](train-examples-transformers-transformers-torch-trainer-basic.md): :orphan:
- [Getting Started Jax](train-getting-started-jax.md): .. _train-jax:
- [Getting Started Lightgbm](train-getting-started-lightgbm.md): .. _train-lightgbm:
- [Getting Started Pytorch Lightning](train-getting-started-pytorch-lightning.md): .. _train-pytorch-lightning:
- [Getting Started Pytorch](train-getting-started-pytorch.md): .. _train-pytorch:
- [Getting Started Transformers](train-getting-started-transformers.md): .. _train-pytorch-transformers:
- [Getting Started Xgboost](train-getting-started-xgboost.md): .. _train-xgboost:
- [Horovod](train-horovod.md): .. _train-horovod:
- [Huggingface Accelerate](train-huggingface-accelerate.md): .. _train-hf-accelerate:
- [More Frameworks](train-more-frameworks.md): .. _train-more-frameworks:
- [Overview](train-overview.md): .. _train-key-concepts:
- [Train](train-train.md): .. _train-docs:
- [_Collate_Utils](train-user-guides-collate-utils.md): :orphan:
- [Asynchronous Validation](train-user-guides-asynchronous-validation.md): .. _train-validating-checkpoints:
- [Checkpoints](train-user-guides-checkpoints.md): .. _train-checkpointing:
- [Data Loading Preprocessing](train-user-guides-data-loading-preprocessing.md): .. _data-ingest-torch:
- [Experiment Tracking](train-user-guides-experiment-tracking.md): .. _train-experiment-tracking-native:
- [Fault Tolerance](train-user-guides-fault-tolerance.md): .. _train-fault-tolerance:
- [Hyperparameter Optimization](train-user-guides-hyperparameter-optimization.md): .. _train-tune:
- [Local_Mode](train-user-guides-local-mode.md): .. _train-local-mode:
- [Monitor Your Application](train-user-guides-monitor-your-application.md): .. _train-metrics:
- [Monitoring Logging](train-user-guides-monitoring-logging.md): .. _train-monitoring-and-logging:
- [Persistent Storage](train-user-guides-persistent-storage.md): .. _persistent-storage-guide:
- [Reproducibility](train-user-guides-reproducibility.md): .. _train-reproducibility:
- [Results](train-user-guides-results.md): .. _train-inspect-results:
- [Scaling Collation Functions](train-user-guides-scaling-collation-functions.md): .. _train-scaling-collation-functions:
- [Using Gpus](train-user-guides-using-gpus.md): .. _train_scaling_config:
- [User Guides](train-user-guides.md): .. _train-user-guides:
- [Api](tune-api-api.md): .. _tune-api-ref:
- [Callbacks](tune-api-callbacks.md): .. _tune-callbacks-docs:
- [Tune CLI (Experimental)](tune-api-cli.md): Tune CLI (Experimental)
- [Env](tune-api-env.md): .. _tune-env-vars:
- [Tune Execution (tune.Tuner)](tune-api-execution.md): Tune Execution (tune.Tuner)
- [Integration](tune-api-integration.md): .. _tune-integration:
- [Tune Internals](tune-api-internals.md): Tune Internals
- [Logging](tune-api-logging.md): .. _loggers-docstring:
- [Reporters](tune-api-reporters.md): .. _tune-reporter-doc:
- [Result_Grid](tune-api-result-grid.md): .. _air-results-ref:
- [Schedulers](tune-api-schedulers.md): .. _tune-schedulers:
- [Search_Space](tune-api-search-space.md): .. _tune-search-space:
- [Stoppers](tune-api-stoppers.md): .. _tune-stoppers:
- [Suggestion](tune-api-suggestion.md): .. _tune-search-alg:
- [Syncing in Tune](tune-api-syncing.md): Syncing in Tune
- [Trainable](tune-api-trainable.md): .. _trainable-docs:
- [Async_Hyperband_Example](tune-examples-includes-async-hyperband-example.md): :orphan:
- [Ax_Example](tune-examples-includes-ax-example.md): :orphan:
- [Bayesopt_Example](tune-examples-includes-bayesopt-example.md): :orphan:
- [Bohb_Example](tune-examples-includes-bohb-example.md): :orphan:
- [Custom_Func_Checkpointing](tune-examples-includes-custom-func-checkpointing.md): :orphan:
- [Hyperband_Example](tune-examples-includes-hyperband-example.md): :orphan:
- [Hyperband_Function_Example](tune-examples-includes-hyperband-function-example.md): :orphan:
- [Hyperopt_Conditional_Search_Space_Example](tune-examples-includes-hyperopt-conditional-search-space-example.md): :orphan:
- [Logging_Example](tune-examples-includes-logging-example.md): :orphan:
- [Mlflow_Ptl_Example](tune-examples-includes-mlflow-ptl-example.md): :orphan:
- [Mnist_Ptl_Mini](tune-examples-includes-mnist-ptl-mini.md): :orphan:
- [Mnist_Pytorch](tune-examples-includes-mnist-pytorch.md): :orphan:
- [Mnist_Pytorch_Trainable](tune-examples-includes-mnist-pytorch-trainable.md): :orphan:
- [Nevergrad_Example](tune-examples-includes-nevergrad-example.md): :orphan:
- [Pb2_Example](tune-examples-includes-pb2-example.md): :orphan:
- [Pb2_Ppo_Example](tune-examples-includes-pb2-ppo-example.md): :orphan:
- [Pbt_Convnet_Function_Example](tune-examples-includes-pbt-convnet-function-example.md): :orphan:
- [Pbt_Example](tune-examples-includes-pbt-example.md): :orphan:
- [Pbt_Function](tune-examples-includes-pbt-function.md): :orphan:
- [Pbt_Memnn_Example](tune-examples-includes-pbt-memnn-example.md): :orphan:
- [Pbt_Tune_Cifar10_With_Keras](tune-examples-includes-pbt-tune-cifar10-with-keras.md): :orphan:
- [Tf_Mnist_Example](tune-examples-includes-tf-mnist-example.md): :orphan:
- [Tune_Basic_Example](tune-examples-includes-tune-basic-example.md): :orphan:
- [Xgboost_Dynamic_Resources_Example](tune-examples-includes-xgboost-dynamic-resources-example.md): :orphan:
- [Index](tune-examples.md): .. _tune-examples-ref:
- [Pbt_Visualization_Utils](tune-examples-pbt-visualization-pbt-visualization-utils.md): :orphan:
- [Faq](tune-faq.md): .. _tune-faq:
- [Getting Started](tune-getting-started.md): .. _tune-tutorial:
- [Index](tune.md): .. _tune-main:
- [Key Concepts](tune-key-concepts.md): .. _tune-60-seconds:
- [Overview](tune-tutorials-overview.md): .. _tune-guides:
- [Tune Distributed](tune-tutorials-tune-distributed.md): .. _tune-distributed-ref:
- [Tune Fault Tolerance](tune-tutorials-tune-fault-tolerance.md): .. _tune-fault-tolerance-ref:
- [How does Tune work?](tune-tutorials-tune-lifecycle.md): How does Tune work?
- [A Guide To Callbacks & Metrics in Tune](tune-tutorials-tune-metrics.md): A Guide To Callbacks & Metrics in Tune
- [Logging and Outputs in Tune](tune-tutorials-tune-output.md): Logging and Outputs in Tune
- [Tune Resources](tune-tutorials-tune-resources.md): .. _tune-parallelism:
- [Tune Run](tune-tutorials-tune-run.md): .. _tune-parallel-experiments-guide:
- [Tune Scalability](tune-tutorials-tune-scalability.md): :orphan:
- [Tune Search Spaces](tune-tutorials-tune-search-spaces.md): .. _tune-search-space-tutorial:
- [Tune Stopping](tune-tutorials-tune-stopping.md): .. _tune-stopping-guide:
- [Tune Storage](tune-tutorials-tune-storage.md): .. _tune-storage-options:
- [Tune Trial Checkpoints](tune-tutorials-tune-trial-checkpoints.md): .. _tune-trial-checkpoint:
- [Configuring and Managing Ray Dashboard](cluster-configure-manage-dashboard.md): (observability-configure-manage-dashboard)=
- [KubeRay memory and scalability benchmark](cluster-kubernetes-benchmarks-memory-scalability-benchmark.md): (kuberay-mem-scalability)=
- [KubeRay Benchmarks](cluster-kubernetes-benchmarks.md): (kuberay-benchmarks)=
- [Deploying Ray Clusters via ArgoCD](cluster-kubernetes-examples-argocd.md): (deploying-on-argocd-example)=
- [Distributed checkpointing with KubeRay and GCSFuse](cluster-kubernetes-examples-distributed-checkpointing-with-gcsfuse.md): (kuberay-distributed-checkpointing-gcsfuse)=
- [Train a PyTorch model on Fashion MNIST with CPUs on Kubernetes](cluster-kubernetes-examples-mnist-training-example.md): (kuberay-mnist-training-example)=
- [Serve a MobileNet image classifier on Kubernetes](cluster-kubernetes-examples-mobilenet-rayservice.md): (kuberay-mobilenet-rayservice-example)=
- [Use Modin with Ray on Kubernetes](cluster-kubernetes-examples-modin-example.md): (kuberay-modin-example)=
- [RayJob Batch Inference Example](cluster-kubernetes-examples-rayjob-batch-inference-example.md): (kuberay-batch-inference-example)=
- [Gang Scheduling with RayJob and Kueue](cluster-kubernetes-examples-rayjob-kueue-gang-scheduling.md): (kuberay-kueue-gang-scheduling-example)=
- [Priority Scheduling with RayJob and Kueue](cluster-kubernetes-examples-rayjob-kueue-priority-scheduling.md): (kuberay-kueue-priority-scheduling-example)=
- [Serve Deepseek R1 using Ray Serve LLM](cluster-kubernetes-examples-rayserve-deepseek-example.md): (kuberay-rayservice-deepseek-example)=
- [Serve a Large Language Model using Ray Serve LLM on Kubernetes](cluster-kubernetes-examples-rayserve-llm-example.md): (kuberay-rayservice-llm-example)=
- [Serve a StableDiffusion text-to-image model on Kubernetes](cluster-kubernetes-examples-stable-diffusion-rayservice.md): (kuberay-stable-diffusion-rayservice-example)=
- [Serve a text summarizer on Kubernetes](cluster-kubernetes-examples-text-summarizer-rayservice.md): (kuberay-text-summarizer-rayservice-example)=
- [Serve a Stable Diffusion model on GKE with TPUs](cluster-kubernetes-examples-tpu-serve-stable-diffusion.md): (kuberay-tpu-stable-diffusion-example)=
- [Reinforcement Learning with Human Feedback (RLHF) for LLMs with verl on KubeRay](cluster-kubernetes-examples-verl-post-training.md): (kuberay-verl)=
- [Examples](cluster-kubernetes-examples.md): (kuberay-examples)=
- [KubeRay Operator Installation](cluster-kubernetes-getting-started-kuberay-operator-installation.md): (kuberay-operator-deploy)=
- [RayCluster Quickstart](cluster-kubernetes-getting-started-raycluster-quick-start.md): (kuberay-raycluster-quickstart)=
- [RayJob Quickstart](cluster-kubernetes-getting-started-rayjob-quick-start.md): (kuberay-rayjob-quickstart)=
- [RayService Quickstart](cluster-kubernetes-getting-started-rayservice-quick-start.md): (kuberay-rayservice-quickstart)=
- [Getting Started with KubeRay](cluster-kubernetes-getting-started.md): (kuberay-quickstart)=
- [Ray on Kubernetes](cluster-kubernetes.md): :hidden:
- [Ingress](cluster-kubernetes-k8s-ecosystem-ingress.md): (kuberay-ingress)=
- [mTLS and L7 observability with Istio](cluster-kubernetes-k8s-ecosystem-istio.md): (kuberay-istio)=
- [Gang scheduling, queue priority, and GPU sharing for RayClusters using KAI Scheduler](cluster-kubernetes-k8s-ecosystem-kai-scheduler.md): (kuberay-kai-scheduler)=
- [Gang scheduling, Priority scheduling, and Autoscaling for KubeRay CRDs with Kueue](cluster-kubernetes-k8s-ecosystem-kueue.md): (kuberay-kueue)=
- [KubeRay metrics references](cluster-kubernetes-k8s-ecosystem-metrics-references.md): (kuberay-metrics-references)=
- [Using Prometheus and Grafana](cluster-kubernetes-k8s-ecosystem-prometheus-grafana.md): (kuberay-prometheus-grafana)=
- [Profiling with py-spy](cluster-kubernetes-k8s-ecosystem-pyspy.md): (kuberay-pyspy-integration)=
- [KubeRay integration with scheduler plugins](cluster-kubernetes-k8s-ecosystem-scheduler-plugins.md): (kuberay-scheduler-plugins)=
- [KubeRay integration with Volcano](cluster-kubernetes-k8s-ecosystem-volcano.md): (kuberay-volcano)=
- [KubeRay integration with Apache YuniKorn](cluster-kubernetes-k8s-ecosystem-yunikorn.md): (kuberay-yunikorn)=
- [KubeRay Ecosystem](cluster-kubernetes-k8s-ecosystem.md): (kuberay-ecosystem-integration)=
- [API Reference](cluster-kubernetes-references.md): (kuberay-api-reference)=
- [RayService troubleshooting](cluster-kubernetes-troubleshooting-rayservice-troubleshooting.md): (kuberay-raysvc-troubleshoot)=
- [Troubleshooting guide](cluster-kubernetes-troubleshooting-troubleshooting.md): (kuberay-troubleshooting-guides)=
- [KubeRay Troubleshooting](cluster-kubernetes-troubleshooting.md): (kuberay-troubleshooting)=
- [Start an Aliyun ACK cluster with GPUs for KubeRay](cluster-kubernetes-user-guides-ack-gpu-cluster.md): (kuberay-ack-gpu-cluster-setup)=
- [Start Amazon EKS Cluster with GPUs for KubeRay](cluster-kubernetes-user-guides-aws-eks-gpu-cluster.md): (kuberay-eks-gpu-cluster-setup)=
- [Start Azure AKS Cluster with GPUs for KubeRay](cluster-kubernetes-user-guides-azure-aks-gpu-cluster.md): (kuberay-aks-gpu-cluster-setup)=
- [RayCluster Configuration](cluster-kubernetes-user-guides-config.md): (kuberay-config)=
- [KubeRay Autoscaling](cluster-kubernetes-user-guides-configuring-autoscaling.md): (kuberay-autoscaling)=
- [Start Google Cloud GKE Cluster with GPUs for KubeRay](cluster-kubernetes-user-guides-gcp-gke-gpu-cluster.md): (kuberay-gke-gpu-cluster-setup)=
- [Start Google Cloud GKE Cluster with TPUs for KubeRay](cluster-kubernetes-user-guides-gcp-gke-tpu-cluster.md): (kuberay-gke-tpu-cluster-setup)=
- [Configuring KubeRay to use Google Cloud Storage Buckets in GKE](cluster-kubernetes-user-guides-gke-gcs-bucket.md): (kuberay-gke-bucket)=
- [Helm Chart RBAC](cluster-kubernetes-user-guides-helm-chart-rbac.md): (kuberay-helm-chart-rbac)=
- [(Advanced) Understanding the Ray Autoscaler in the Context of Kubernetes](cluster-kubernetes-user-guides-k8s-autoscaler.md): (ray-k8s-autoscaler-comparison)=
- [Managed Kubernetes services](cluster-kubernetes-user-guides-k8s-cluster-setup.md): (kuberay-k8s-setup)=
- [Use kubectl plugin (beta)](cluster-kubernetes-user-guides-kubectl-plugin.md): (kubectl-plugin)=
- [Configure Ray clusters to use token authentication](cluster-kubernetes-user-guides-kuberay-auth.md): (kuberay-auth)=
- [Use KubeRay dashboard (experimental)](cluster-kubernetes-user-guides-kuberay-dashboard.md): (kuberay-dashboard)=
- [GCS fault tolerance in KubeRay](cluster-kubernetes-user-guides-kuberay-gcs-ft.md): (kuberay-gcs-ft)=
- [Tuning Redis for a Persistent Fault Tolerant GCS](cluster-kubernetes-user-guides-kuberay-gcs-persistent-ft.md): (kuberay-gcs-persistent-ft)=
- [KubeRay label-based scheduling](cluster-kubernetes-user-guides-label-based-scheduling.md): (kuberay-label-scheduling)=
- [KubeRay Observability](cluster-kubernetes-user-guides-observability.md): (kuberay-observability)=
- [Persist KubeRay custom resource logs](cluster-kubernetes-user-guides-persist-kuberay-custom-resource-logs.md): (persist-kuberay-custom-resource-logs)=
- [Persist KubeRay Operator Logs](cluster-kubernetes-user-guides-persist-kuberay-operator-logs.md): (persist-kuberay-operator-logs)=
- [Specify container commands for Ray head/worker Pods](cluster-kubernetes-user-guides-pod-command.md): (kuberay-pod-command)=
- [RayService high availability](cluster-kubernetes-user-guides-rayservice-high-availability.md): (kuberay-rayservice-ha)=
- [RayService Zero-Downtime Incremental Upgrades](cluster-kubernetes-user-guides-rayservice-incremental-upgrade.md): (kuberay-rayservice-incremental-upgrade)=
- [RayService worker Pods aren't ready](cluster-kubernetes-user-guides-rayservice-no-ray-serve-replica.md): (kuberay-rayservice-no-ray-serve-replica)=
- [Deploy Ray Serve Applications](cluster-kubernetes-user-guides-rayservice.md): (kuberay-rayservice)=
- [Reducing image pull latency on Kubernetes](cluster-kubernetes-user-guides-reduce-image-pull-latency.md): (reduce-image-pull-latency)=
- [Best Practices for Storage and Dependencies](cluster-kubernetes-user-guides-storage.md): (kuberay-storage)=
- [TLS Authentication](cluster-kubernetes-user-guides-tls.md): (kuberay-tls)=
- [Use TPUs with KubeRay](cluster-kubernetes-user-guides-tpu.md): (kuberay-tpu)=
- [KubeRay upgrade guide](cluster-kubernetes-user-guides-upgrade-guide.md): (kuberay-upgrade-guide)=
- [Using `uv` for Python package management in KubeRay](cluster-kubernetes-user-guides-uv.md): (kuberay-uv)=
- [User Guides](cluster-kubernetes-user-guides.md): (kuberay-guides)=
- [Collecting and monitoring metrics](cluster-metrics.md): (collect-metrics)=
- [Application guide](cluster-running-applications.md): This section introduces the main differences in running a Ray application on your laptop vs on a Ray Cluster.
- [Ray Jobs Overview](cluster-running-applications-job-submission.md): (jobs-overview)=
- [Examples](cluster-vms-examples.md): (vm-cluster-examples)=
- [Ray Train XGBoostTrainer on VMs](cluster-vms-examples-ml-example.md): (clusters-vm-ml-example)=
- [Ray on Cloud VMs](cluster-vms.md): (cloud-vm-index)=
- [API References](cluster-vms-references.md): (vm-cluster-api-references)=
- [User Guides](cluster-vms-user-guides.md): (vm-cluster-guides)=
- [Launching Ray Clusters on AWS](cluster-vms-user-guides-launching-clusters-aws.md): This guide details the steps needed to start a Ray cluster on AWS.
- [Launching Ray Clusters on Azure](cluster-vms-user-guides-launching-clusters-azure.md): This guide details the steps needed to start a Ray cluster on Azure.
- [Launching Ray Clusters on GCP](cluster-vms-user-guides-launching-clusters-gcp.md): This guide details the steps needed to start a Ray cluster in GCP.
- [Launching an On-Premise Cluster](cluster-vms-user-guides-launching-clusters-on-premises.md): (on-prem)=
- [Launching Ray Clusters on vSphere](cluster-vms-user-guides-launching-clusters-vsphere.md): This guide details the steps needed to launch a Ray cluster in a vSphere environment.
- [Log Persistence](cluster-vms-user-guides-logging.md): (vm-logging)=
- [Ray Data Benchmarks](data-benchmark.md): This page documents benchmark results and methodologies for evaluating Ray Data performance across a variety of data ...
- [Contributing Guide](data-contributing-contributing-guide.md): If you want your changes to be reviewed and merged quickly, following a few key
- [How to write tests](data-contributing-how-to-write-tests.md): (how-to-write-tests)=
- [Contributing to the Ray Documentation](ray-contribute-docs.md): There are many ways to contribute to the Ray documentation, and we're always looking for new contributors.
- [Authenticating Remote URIs in runtime_env](ray-core-runtime-env-auth.md): (runtime-env-auth)=
- [Use labels to control scheduling](ray-core-scheduling-labels.md): (labels)=
- [Type hints in Ray](ray-core-type-hint.md): (core-type-hint)=
- [Distributed Data Processing in Data-Juicer](ray-more-libs-data-juicer-distributed-data-processing.md): Data-Juicer supports large-scale distributed data processing based on [Ray](https://github.com/ray-project/ray) and [...
- [Monitoring and Debugging](ray-observability.md): (observability)=
- [Reference](ray-observability-reference.md): (observability-reference)=
- [Configuring Logging](ray-observability-user-guides-configure-logging.md): (configure-logging)=
- [Debugging Applications](ray-observability-user-guides-debug-apps.md): (observability-debug-apps)=
- [User Guides](ray-observability-user-guides.md): (observability-user-guides)=
- [Profiling](ray-observability-user-guides-profiling.md): (profiling)=
- [Getting Started](ray-overview-getting-started.md): (gentle-intro)=
- [Overview](ray-overview.md): (overview-overview)=
- [Security](ray-security.md): (security)=
- [Ray token authentication](ray-security-token-auth.md): (token-auth)=
- [Advanced Ray Serve Autoscaling](serve-advanced-guides-advanced-autoscaling.md): (serve-advanced-autoscaling)=
- [Pass Arguments to Applications](serve-advanced-guides-app-builder-guide.md): (serve-app-builder-guide)=
- [Asyncio and concurrency best practices in Ray Serve](serve-advanced-guides-asyncio-best-practices.md): (serve-asyncio-best-practices)=
- [Use Custom Algorithm for Request Routing](serve-advanced-guides-custom-request-router.md): (custom-request-router-guide)=
- [Deploy on VM](serve-advanced-guides-deploy-vm.md): (serve-in-production-deploying)=
- [Development Workflow](serve-advanced-guides-dev-workflow.md): (serve-dev-workflow)=
- [Dynamic Request Batching](serve-advanced-guides-dyn-req-batch.md): (serve-performance-batching-requests)=
- [Set Up a gRPC Service](serve-advanced-guides-grpc-guide.md): (serve-set-up-grpc-service)=
- [Advanced Guides](serve-advanced-guides.md): (serve-advanced-guides)=
- [Updating Applications In-Place](serve-advanced-guides-inplace-updates.md): (serve-inplace-updates)=
- [Experimental Java API](serve-advanced-guides-managing-java-deployments.md): (serve-java-api)=
- [Run Multiple Applications in Different Containers](serve-advanced-guides-multi-app-container.md): (serve-container-runtime-env-guide)=
- [Troubleshoot multi-node GPU serving on KubeRay](serve-advanced-guides-multi-node-gpu-troubleshooting.md): (serve-multi-node-gpu-troubleshooting)=
- [Performance Tuning](serve-advanced-guides-performance.md): (serve-perf-tuning)=
- [Replica ranks](serve-advanced-guides-replica-ranks.md): (serve-replica-ranks)=
- [Replica scheduling](serve-advanced-guides-replica-scheduling.md): (serve-replica-scheduling)=
- [Ray Serve API](serve-api.md): (serve-api)=
- [Architecture](serve-architecture.md): (serve-architecture)=
- [Asynchronous Inference](serve-asynchronous-inference.md): (serve-asynchronous-inference)=
- [Ray Serve Autoscaling](serve-autoscaling-guide.md): (serve-autoscaling)=
- [Configure Ray Serve deployments](serve-configure-serve-deployment.md): (serve-configure-deployment)=
- [Develop and Deploy an ML Application](serve-develop-and-deploy.md): (serve-develop-and-deploy)=
- [Getting Started](serve-getting-started.md): (serve-getting-started)=
- [Set Up FastAPI and HTTP](serve-http-guide.md): (serve-set-up-fastapi-http)=
- [Ray Serve: Scalable and Programmable Serving](serve.md): (rayserve)=
- [Key Concepts](serve-key-concepts.md): (serve-key-concepts)=
- [Core components](serve-llm-architecture-core.md): (serve-llm-architecture-core)=
- [Architecture](serve-llm-architecture.md): Technical documentation for Ray Serve LLM architecture, components, and patterns.
- [Architecture overview](serve-llm-architecture-overview.md): (serve-llm-architecture-overview)=
- [Request routing](serve-llm-architecture-routing-policies.md): Ray Serve LLM provides customizable request routing to optimize request distribution across replicas for different wo...
- [Data parallel attention](serve-llm-architecture-serving-patterns-data-parallel.md): (serve-llm-architecture-data-parallel)=
- [Serving patterns](serve-llm-architecture-serving-patterns.md): Architecture documentation for distributed LLM serving patterns.
- [Prefill-decode disaggregation](serve-llm-architecture-serving-patterns-prefill-decode.md): (serve-llm-architecture-prefill-decode)=
- [Benchmarks](serve-llm-benchmarks.md): Performance in LLM serving depends heavily on your specific workload characteristics and hardware stack. From a Ray S...
- [Examples](serve-llm-examples.md): Production examples for deploying LLMs with Ray Serve.
- [Serving LLMs](serve-llm.md): (serving-llms)=
- [Quickstart examples](serve-llm-quick-start.md): (quick-start)=
- [Troubleshooting](serve-llm-troubleshooting.md): Common issues and frequently asked questions for Ray Serve LLM.
- [Cross-node parallelism](serve-llm-user-guides-cross-node-parallelism.md): (cross-node-parallelism)=
- [Data parallel attention](serve-llm-user-guides-data-parallel-attention.md): (data-parallel-attention-guide)=
- [Deployment Initialization](serve-llm-user-guides-deployment-initialization.md): (deployment-initialization-guide)=
- [Fractional GPU serving](serve-llm-user-guides-fractional-gpu.md): (fractional-gpu-guide)=
- [User guides](serve-llm-user-guides.md): How-to guides for deploying and configuring Ray Serve LLM features.
- [KV cache offloading](serve-llm-user-guides-kv-cache-offloading.md): (kv-cache-offloading-guide)=
- [Multi-LoRA deployment](serve-llm-user-guides-multi-lora.md): Deploy multiple fine-tuned LoRA adapters efficiently with Ray Serve LLM.
- [Observability and monitoring](serve-llm-user-guides-observability.md): (observability-guide)=
- [Prefill/decode disaggregation](serve-llm-user-guides-prefill-decode.md): (prefill-decode-guide)=
- [Prefix-aware routing](serve-llm-user-guides-prefix-aware-routing.md): (prefix-aware-routing-guide)=
- [vLLM compatibility](serve-llm-user-guides-vllm-compatibility.md): (vllm-compatibility-guide)=
- [Model Multiplexing](serve-model-multiplexing.md): (serve-model-multiplexing)=
- [Deploy Compositions of Models](serve-model-composition.md): (serve-model-composition)=
- [Monitor Your Application](serve-monitoring.md): (serve-monitoring)=
- [Deploy Multiple Applications](serve-multi-app.md): (serve-multi-application)=
- [Best practices in production](serve-production-guide-best-practices.md): (serve-best-practices)=
- [Serve Config Files](serve-production-guide-config.md): (serve-in-production-config-file)=
- [Custom Docker Images](serve-production-guide-docker.md): (serve-custom-docker-images)=
- [Add End-to-End Fault Tolerance](serve-production-guide-fault-tolerance.md): (serve-e2e-ft)=
- [Handle Dependencies](serve-production-guide-handling-dependencies.md): (serve-handling-dependencies)=
- [Production Guide](serve-production-guide.md): (serve-in-production)=
- [Deploy on Kubernetes](serve-production-guide-kubernetes.md): (serve-in-production-kubernetes)=
- [Resource Allocation](serve-resource-allocation.md): (serve-resource-allocation)=
- [Serve an Inference with Stable Diffusion Model on AWS NeuronCores Using FastAPI](serve-tutorials-aws-neuron-core-inference-stable-diffusion.md): This example uses a precompiled Stable Diffusion XL model and deploys on an AWS Inferentia2 (Inf2)
- [Serve an Inference Model on AWS NeuronCores Using FastAPI (Experimental)](serve-tutorials-aws-neuron-core-inference.md): (aws-neuron-core-inference-tutorial)=
- [Serve a Text Generator with Request Batching](serve-tutorials-batch.md): (serve-batch-tutorial)=
- [Scale a Gradio App with Ray Serve](serve-tutorials-gradio-integration.md): This guide shows how to scale up your [Gradio](https://gradio.app/) application using Ray Serve. Keep the internal ar...
- [Serve Llama2-7b/70b on a single or multiple Intel Gaudi Accelerator](serve-tutorials-intel-gaudi-inference.md): [Intel Gaudi AI Processors (HPUs)](https://habana.ai) are AI hardware accelerators designed by Intel Habana Labs. See...
- [Serve a Java App](serve-tutorials-java.md): (serve-java-tutorial)=
- [Building a Real-time Object Detection Service with Ray Serve](serve-tutorials-object-detection.md): (serve-object-detection-tutorial)=
- [Serve DeepSeek](serve-tutorials-serve-deepseek.md): (serve-deepseek-tutorial)=
- [Serve ML Models (Tensorflow, PyTorch, Scikit-Learn, others)](serve-tutorials-serve-ml-models.md): (serve-ml-models-tutorial)=
- [Serve a Stable Diffusion Model](serve-tutorials-stable-diffusion.md): (serve-stable-diffusion-tutorial)=
- [Serve a Chatbot with Request and Response Streaming](serve-tutorials-streaming.md): (serve-streaming-tutorial)=
- [Serve a Text Classification Model](serve-tutorials-text-classification.md): (serve-text-classification-tutorial)=
- [Serving models with Triton Server in Ray Serve](serve-tutorials-triton-server-integration.md): This guide shows how to build an application with stable diffusion model using [NVIDIA Triton Server](https://github....
- [Getting Data in and out of Tune](tune-tutorials-tune-get-data-in-and-out.md): Often, you will find yourself needing to pass data into Tune [Trainables](tune_60_seconds_trainables) (datasets, mode...