# Disable after successful migration
DATASET_VERSION_ITEMS_TOTAL_MIGRATION_ENABLED: false
```

### 3. Lazy Migration (Optional Data Reconciliation)

An optional safety mechanism controlled by `DATASET_VERSIONING_MIGRATION_LAZY_ENABLED` (disabled by default).

**When to enable**: Only if data was left behind due to concurrent writes during the migration. This handles the edge case where datasets were created during the migration process itself and were not migrated by the Liquibase migration.

**How it works**: When enabled, every dataset CRUD operation checks whether the dataset was migrated. If not migrated, it automatically migrates the dataset during that operation. This means you can simply access an unmigrated dataset through the UI or API, and it will be migrated on-the-fly.

**Usage**: If you know a specific dataset was not migrated, enable this flag temporarily and access that dataset. It's recommended to disable this flag after all datasets are aligned (have a first version), but this is **not mandatory** - the check only runs for the dataset being accessed, so it becomes a no-op for datasets that are already migrated (the check will find they're already migrated and do nothing).

## Expected Behavior During Migration

### First Startup with Version 1.9.92+

**Expected Timeline**: The migration typically takes **2-10 minutes**, longer than standard Liquibase migrations due to data processing requirements.

During this time, you may see transient errors in the logs similar to:

```
Causing: liquibase.exception.DatabaseException: ClickHouse exception, code: 159, host: clickhouse-opik-clickhouse, port: 8123; Read timed out [Failed SQL: (159) INSERT INTO opik_prod.dataset_item_versions
```

<Callout intent="info">
  **These errors are expected and transient.** The system automatically recovers once the migration process completes. No manual intervention is required.
</Callout>

### Migration Process Timeline

The automated migration follows this sequence:

1. **Application starts** (t=0s)
2. **Liquibase migration begins** (immediately)
   * Migrates dataset schema in ClickHouse
   * Duration: 2-10 minutes (depends on data volume)
3. **Startup delay** (default: 30 seconds after app start)
4. **MySQL counter migration begins** (after startup delay)
   * Processes dataset versions in batches
   * Updates `items_total` field in MySQL
5. **Migration completes** automatically without downtime

## Configuration Options

All configuration is done through environment variables. **The default values are appropriate for most deployments and require no changes.**

### MySQL Counter Migration (Recommended to Disable After Completion)

Controls the MySQL dataset version counter migration. **This should be disabled after the upgrade completes successfully.**

| Environment Variable                                          | Default | Description                                                                          |
| ------------------------------------------------------------- | ------- | ------------------------------------------------------------------------------------ |
| `DATASET_VERSION_ITEMS_TOTAL_MIGRATION_ENABLED`               | `true`  | Enable the MySQL counter migration on startup. **Disable after successful upgrade.** |
| `DATASET_VERSION_ITEMS_TOTAL_MIGRATION_BATCH_SIZE`            | `100`   | Number of dataset versions to process per batch                                      |
| `DATASET_VERSION_ITEMS_TOTAL_MIGRATION_LOCK_TIMEOUT_SECONDS`  | `3600`  | Lock timeout in seconds (1 hour)                                                     |
| `DATASET_VERSION_ITEMS_TOTAL_MIGRATION_STARTUP_DELAY_SECONDS` | `30`    | Delay before starting migration after startup                                        |
| `DATASET_VERSION_ITEMS_TOTAL_MIGRATION_JOB_TIMEOUT_SECONDS`   | `3600`  | Maximum time for the entire migration job (1 hour)                                   |

### Lazy Migration (Optional Data Reconciliation)

Controls optional data reconciliation for datasets that may have been missed during concurrent writes. **Only enable if you suspect data was left behind during migration.**

| Environment Variable                        | Default | Description                                                                     |
| ------------------------------------------- | ------- | ------------------------------------------------------------------------------- |
| `DATASET_VERSIONING_MIGRATION_LAZY_ENABLED` | `false` | Enable lazy migration for reconciling missed datasets (migrate on first access) |

## Post-Migration Steps

### Step 1: Verify Migration Completion

Monitor the backend logs to confirm successful completion:

```bash