# dat1.co

> Dat1 is a serverless GPU platform for running custom generative AI models at scale, offering fast, cost-efficient inference, privacy compliance, and zero hardware management.

Dat1 provides a machine learning model hosting solution that eliminates the complexity and expense of managing hardware for large AI models. The platform automatically handles scaling, reduces operational and hardware costs by efficiently sharing GPUs, and charges only for the time your model is actively running inferenceâno costs for idle time or timeouts.

## Key Features

* **Serverless GPU Inference:** Deploy custom AI models without managing hardware or scaling logistics.
* **Pay-per-second Pricing:** Only pay for the seconds your model is processing requests, with no charges for idle time or timeouts.
* **Low Cold Start Times:** Achieves 15-20 second end-to-end cold starts for 70GB models. This includes container launch, model weight download, GPU memory load, and inference server startup.
Competitors often quote only the time it takes to initialize the container, which can be misleading (that takes less than a second with Dat1).
Dat1 also helps their clients optimize cold start and models in general even further.
More details in comparison to other platforms can be found in the [real scenario benchmark test](https://dat1.co/blog/serverless-inference-providers-compared).
* **Privacy and Compliance:** Fully GDPR and CCPA-compliant, following top cybersecurity best practices.
* **Part of Nvidia Inception:** Recognized as part of Nvidiaâs Inception program for AI startups.


## How It Works

* Upload your model weights and Python code as a deployment package using the `dat1-cli`.
* Dat1 manages all hardware logistics and distributes your model across its infrastructure.
* Scaling is handled automatically based on demand.
* You are only billed for active inference time.


## CLI and Deployment

The `dat1-cli` is the official command-line interface for interacting with the Dat1 platform to deploy and manage your models.

### Installation

To install the CLI, run the following command in your terminal:

```bash
pip install dat1-cli
```


### CLI Usage

* **Login:** Initialize the CLI with your API key.

```bash
dat1 login
```

* **Initialize Project:** Create a `dat1.yaml` configuration file in your project's root directory.

```bash
dat1 init
```

* **Deploy Model:** Upload your model to the platform.

```bash
dat1 deploy
```

* **Serve Locally:** Launch your model in a local environment for testing. This requires Docker, a CUDA-compatible GPU, and the NVIDIA Container Toolkit.

```bash
dat1 serve
```


### Configuration (`dat1.yaml`)

The `dat1.yaml` file is used to configure your model for deployment. It specifies the model name and uses glob patterns to define files that should be excluded from the upload package.

```yaml
model_name: <your model name>
exclude:
  - '**/.git/**'
  - '**/.idea/**'
  - '*.md'
  - '*.jpg'
  - .dat1.yaml
  - .DS_Store
```


### Code Examples

The platform requires a `handler.py` file in your project's root that contains a FastAPI application to handle inference requests.

#### **Basic Handler**

This example demonstrates a simple handler with a health check endpoint (`/`) and a standard inference endpoint (`/infer`).

```python
from fastapi import Request, FastAPI
from vllm import LLM, SamplingParams
import os