Last update: 2025-12-30T14:08:31.000Z
Content:
---
title: "Service data sources · Tinybird Docs"
theme-color: "#171612"
description: "In addition to the data sources you upload, Tinybird provides other "Service data sources" that allow you to inspect what's going on in your account."
inkeep:version: "forward"
---




# Service data sources [¶](https://www.tinybird.co/docs/forward/monitoring/service-datasources#service-data-sources)

Copy as MD Tinybird provides Service data sources that you can use to inspect what's going on in your Tinybird account, diagnose issues, monitor usage, and so on.

For example, you can get real time stats about API calls or a log of every operation over your data sources. This is similar to using system tables in a database, although Service data sources contain information about the usage of the service itself.

Queries made to Service data sources are free of charge and don't count towards your usage. However, calls to API endpoints that use Service data sources do count towards API rate limits. See [Billing](../pricing/billing).

## Considerations [¶](https://www.tinybird.co/docs/forward/monitoring/service-datasources#considerations)

- You can't use Service data sources in materialized view queries.
- Pass dynamic query parameters to API endpoints to then query Service data sources.
- You can only query Organization-level Service data sources if you're an administrator. See[  Consumption overview](../administration/organizations#consumption-overview)  .

## Service data sources [¶](https://www.tinybird.co/docs/forward/monitoring/service-datasources#service-data-sources)

The following Service data sources are available.

### tinybird.pipe_stats_rt [¶](https://www.tinybird.co/docs/forward/monitoring/service-datasources#tinybird-pipe-stats-rt)

Contains information about all requests made to your [API endpoints](../work-with-data/publish-data/endpoints) in real time. This data source has a TTL of 7 days. If you need to query data older than 7 days you must use the aggregated by day data available at [tinybird.pipe_stats](https://www.tinybird.co/docs/forward/monitoring/service-datasources#tinybird-pipe-stats).

Calls made against Service data sources are not logged and don't count towards usage limits.

| Field | Type | Description |
| --- | --- | --- |
| `start_datetime` | `DateTime` | API call start date and time. |
| `pipe_id` | `String` | pipe Id as returned in our[  pipes API](/docs/api-reference/pipe-api)   ( `query_api`   in case it's a Query API request). |
| `pipe_name` | `String` | pipe name as returned in our[  pipes API](/docs/api-reference/pipe-api)   ( `query_api`   in case it's a Query API request). |
| `duration` | `Float` | API call duration, in seconds. |
| `read_bytes` | `UInt64` | API call read data, in bytes. |
| `read_rows` | `UInt64` | API call rows read. |
| `result_rows` | `UInt64` | Rows returned by the API call. |
| `url` | `String` | URL ( `token`   param is removed for security reasons). |
| `error` | `UInt8` | `1`   if query returned error, else `0`  . |
| `request_id` | `String` | API call identifier returned in `x-request-id`   header. Format is ULID string. |
| `token` | `String` | API call token identifier used. |
| `token_name` | `String` | API call token name used. |
| `status_code` | `Int32` | API call returned status code. |
| `method` | `String` | API call method POST or GET. |
| `parameters` | `Map(String, String)` | API call parameters used. |
| `release` | `String` | Semantic version of the release (deprecated). |
| `user_agent` | `Nullable(String)` | User Agent HTTP header from the request. |
| `resource_tags` | `Array(String)` | Tags associated with the pipe when the request was made. |
| `cpu_time` | `Float` | CPU time used by the query, in seconds. |
| `memory_usage` | `UInt64` | Memory consumption by the query, in bytes. High memory usage indicates the query may not be optimized efficiently. |

### tinybird.pipe_stats [¶](https://www.tinybird.co/docs/forward/monitoring/service-datasources#tinybird-pipe-stats)

Aggregates the request stats in [tinybird.pipe_stats_rt](https://www.tinybird.co/docs/forward/monitoring/service-datasources#tinybird-pipe-stats-rt) by day.

Calls made against Service data sources are not logged and don't count towards usage limits.

| Field | Type | Description |
| --- | --- | --- |
| `date` | `Date` | Request date and time. |
| `pipe_id` | `String` | pipe Id as returned in our[  pipes API](/docs/api-reference/pipe-api)  . |
| `pipe_name` | `String` | Name of the pipe. |
| `view_count` | `UInt64` | Request count. |
| `error_count` | `UInt64` | Number of requests with error. |
| `avg_duration_state` | `AggregateFunction(avg, Float32)` | Average duration state, in seconds (see[  Querying _state columns](https://www.tinybird.co/docs/forward/monitoring/service-datasources#querying-state-columns)   ). |
| `quantile_timing_state` | `AggregateFunction(quantilesTiming(0.9, 0.95, 0.99), Float64)` | 0.9, 0.95 and 0.99 quantiles state. Time, in milliseconds (see[  Querying _state columns](https://www.tinybird.co/docs/forward/monitoring/service-datasources#querying-state-columns)   ). |
| `read_bytes_sum` | `UInt64` | Total bytes read. |
| `read_rows_sum` | `UInt64` | Total rows read. |
| `resource_tags` | `Array(String)` | All the tags associated with the resource when the aggregated requests were made. |

### tinybird.bi_stats_rt [¶](https://www.tinybird.co/docs/forward/monitoring/service-datasources#tinybird-bi-stats-rt)

This data source has a TTL of 7 days. If you need to query data older than 7 days you must use the aggregated by day data available at [tinybird.bi_stats](https://www.tinybird.co/docs/forward/monitoring/service-datasources#tinybird-bi-stats).

| Field | Type | Description |
| --- | --- | --- |
| `start_datetime` | `DateTime` | Query start timestamp. |
| `query` | `String` | Executed query. |
| `query_normalized` | `String` | Normalized executed query. This is the pattern of the query, without literals. Useful to analyze usage patterns. |
| `error_code` | `Int32` | Error code, if any. `0`   on normal execution. |
| `error` | `String` | Error description, if any. Empty otherwise. |
| `duration` | `UInt64` | Query duration, in milliseconds. |
| `read_rows` | `UInt64` | Read rows. |
| `read_bytes` | `UInt64` | Read bytes. |
| `result_rows` | `UInt64` | Total rows returned. |
| `result_bytes` | `UInt64` | Total bytes returned. |

### tinybird.bi_stats [¶](https://www.tinybird.co/docs/forward/monitoring/service-datasources#tinybird-bi-stats)

Aggregates the stats in [tinybird.bi_stats_rt](https://www.tinybird.co/docs/forward/monitoring/service-datasources#tinybird-bi-stats-rt) by day.

| Field | Type | Description |
| --- | --- | --- |
| `date` | `Date` | Stats date. |
| `database` | `String` | Database identifier. |
| `query_normalized` | `String` | Normalized executed query. This is the pattern of the query, without literals. Useful to analyze usage patterns. |
| `view_count` | `UInt64` | Requests count. |
| `error_count` | `UInt64` | Error count. |
| `avg_duration_state` | `AggregateFunction(avg, Float32)` | Average duration state, in milliseconds (see[  Querying _state columns](https://www.tinybird.co/docs/forward/monitoring/service-datasources#querying-state-columns)   ). |
| `quantile_timing_state` | `AggregateFunction(quantilesTiming(0.9, 0.95, 0.99), Float64)` | 0.9, 0.95 and 0.99 quantiles state. Time, in milliseconds (see[  Querying _state columns](https://www.tinybird.co/docs/forward/monitoring/service-datasources#querying-state-columns)   ). |
| `read_bytes_sum` | `UInt64` | Total bytes read. |
| `read_rows_sum` | `UInt64` | Total rows read. |
| `avg_result_rows_state` | `AggregateFunction(avg, Float32)` | Total bytes returned state (see[  Querying _state columns](https://www.tinybird.co/docs/forward/monitoring/service-datasources#querying-state-columns)   ). |
| `avg_result_bytes_state` | `AggregateFunction(avg, Float32)` | Total rows returned state (see[  Querying _state columns](https://www.tinybird.co/docs/forward/monitoring/service-datasources#querying-state-columns)   ). |

### tinybird.block_log [¶](https://www.tinybird.co/docs/forward/monitoring/service-datasources#tinybird-block-log)

The data source contains details about how Tinybird ingests data into your data sources. You can use this Service data source to spot problematic parts of your data.

| Field | Type | Description |
| --- | --- | --- |
| `timestamp` | `DateTime` | Date and time of the block ingestion. |
| `import_id` | `String` | Id of the import operation. |
| `job_id` | `Nullable(String)` | Id of the job that ingested the block of data, if it was ingested by URL. In this case, `import_id`   and `job_id`   must have the same value. |
| `request_id` | `String` | Id of the request that performed the operation. In this case, `import_id`   and `job_id`   must have the same value. Format is ULID string. |
| `source` | `String` | Either the URL or `stream`   or `body`   keywords. |
| `block_id` | `String` | Block identifier. You can cross this with the `blocks_ids`   column from the[  tinybird.datasources_ops_log](https://www.tinybird.co/docs/forward/monitoring/service-datasources#tinybird-datasources-ops-log)   Service data source. |
| `status` | `String` | `done`   | `error`  . |
| `datasource_id` | `String` | data source consistent id. |
| `datasource_name` | `String` | data source name when the block was ingested. |
| `start_offset` | `Nullable(Int64)` | The starting byte of the block, if the ingestion was split, where this block started. |
| `end_offset` | `Nullable(Int64)` | If split, the ending byte of the block. |
| `rows` | `Nullable(Int32)` | How many rows it ingested. |
| `parser` | `Nullable(String)` | Whether the native block parser or falling back to row by row parsing is used. |
| `quarantine_lines` | `Nullable(UInt32)` | If any, how many rows went into the quarantine data source. |
| `empty_lines` | `Nullable(UInt32)` | If any, how many empty lines were skipped. |
| `bytes` | `Nullable(UInt32)` | How many bytes the block had. |
| `processing_time` | `Nullable(Float32)` | How long it took, in seconds. |
| `processing_error` | `Nullable(String)` | Detailed message in case of error. |

When Tinybird ingests data from a URL, it splits the download in several requests, resulting in different ingestion blocks. The same happens when the data upload happens with a multipart request.

### tinybird.datasources_ops_log [¶](https://www.tinybird.co/docs/forward/monitoring/service-datasources#tinybird-datasources-ops-log)

Contains all operations performed to your data sources. Tinybird tracks the following operations:

| Event | Description |
| --- | --- |
| `create` | A data source is created. |
| `append` | Append operation. |
| `append-hfi` | Append operation using the[  High-frequency Ingestion API](/docs/forward/get-data-in/events-api)  . |
| `append-kafka` | Append operation using the[  Kafka Connector](/docs/forward/get-data-in/connectors/kafka)  . |
| `replace` | A replace operation took place in the data source. |
| `delete` | A data source is deleted. |
| `delete_data` | A delete operation took place in the data source. |
| `truncate` | A truncate operation took place in the data source. |
| `rename` | The data source was renamed. |
| `populateview-queued` | A populate operation was queued for execution. |
| `populateview` | A finished populate operation (up to 8 hours after it started). |
| `copy` | A copy operation took place in the data source. |
| `alter` | An alter operation took place in the data source. |

Materializations are logged with same `event_type` and `operation_id` as the operation that triggers them. You can track the materialization pipe with `pipe_id` and `pipe_name`.

Tinybird logs all operations with the following information in this data source:

| Field | Type | Description |
| --- | --- | --- |
| `timestamp` | `DateTime` | Date and time when the operation started. |
| `event_type` | `String` | Operation being logged. |
| `operation_id` | `String` | Groups rows affected by the same operation. Useful for checking materializations triggered by an append operation. |
| `datasource_id` | `String` | Id of your data source. The data source id is consistent after renaming operations. You should use the id when you want to track name changes. |
| `datasource_name` | `String` | Name of your data source when the operation happened. |
| `result` | `String` | `ok`   | `error` |
| `elapsed_time` | `Float32` | How much time the operation took, in seconds. |
| `error` | `Nullable(String)` | Detailed error message if the result was error. |
| `import_id` | `Nullable(String)` | Id of the import operation, if data has been ingested using one of the following operations: `create`  , `append`   or `replace` |
| `job_id` | `Nullable(String)` | Id of the job that performed the operation, if any. If data has been ingested, `import_id`   and `job_id`   must have the same value. |
| `request_id` | `String` | Id of the request that performed the operation. If data has been ingested, `import_id`   and `request_id`   must have the same value. Format is ULID string. |
| `rows` | `Nullable(UInt64)` | How many rows the operations affected. This depends on `event_type`   : for the `append`   event, how many rows got inserted; for `delete`   or `truncate`   events, how many rows the data source had; for `replace`   , how many rows the data source has after the operation. |
| `rows_quarantine` | `Nullable(UInt64)` | How many rows went into the quarantine data source, if any. |
| `blocks_ids` | `Array(String)` | List of blocks ids used for the operation. See the[  tinybird.block_log](https://www.tinybird.co/docs/forward/monitoring/service-datasources#tinybird-block-log)   Service data source for more details. |
| `options` | `Nested(Names String, Values String)` | Tinybird stores key-value pairs with extra information for some operations. For the `replace`   event, Tinybird uses the `rows_before_replace`   key to track how many rows the data source had before the replacement happened, the `replace_condition`   key shows what condition was used. For `append`   and `replace`   events, Tinybird stores the data `source`   , for example the URL, or body/stream keywords. For `rename`   event, `old_name`   and `new_name`   . For `populateview`   you can find there the whole populate `job`   metadata as a JSON string. For `alter`   events, Tinybird stores `operations`   , and dependent pipes as `dependencies`   if they exist. |
| `read_bytes` | `UInt64` | Read bytes in the operation. |
| `read_rows` | `UInt64` | Read rows in the operation. |
| `written_rows` | `UInt64` | Written rows in the operation. |
| `written_bytes` | `UInt64` | Written bytes in the operation. |
| `written_rows_quarantine` | `UInt64` | Quarantined rows in the operation. |
| `written_bytes_quarantine` | `UInt64` | Quarantined bytes in the operation. |
| `pipe_id` | `String` | If present, materialization pipe id as returned in our[  pipes API](/docs/api-reference/pipe-api)  . |
| `pipe_name` | `String` | If present, materialization pipe name as returned in our[  pipes API](/docs/api-reference/pipe-api)  . |
| `release` | `String` | Semantic version of the release (deprecated). |
| `resource_tags` | `Array(String)` | Tags associated with the pipe when the request was made. |
| `cpu_time` | `Float32` | CPU time used by the operation, in seconds. |
| `memory_usage` | `UInt64` | Memory consuptiom by the operation, in bytes. |

### tinybird.datasource_ops_stats [¶](https://www.tinybird.co/docs/forward/monitoring/service-datasources#tinybird-datasource-ops-stats)

Data from `datasource_ops_log` , aggregated by day.

| Field | Type | Description |
| --- | --- | --- |
| `event_date` | `Date` | Date of the event. |
| `workspace_id` | `String` | Unique identifier for the workspace. |
| `event_type` | `String` | Name of your data source. |
| `pipe_id` | `String` | Identifier of the pipe. |
| `pipe_name` | `String` | Name of the pipe. |
| `error_count` | `UInt64` | Number of requests with an error. |
| `executions` | `UInt64` | Number of executions. |
| `avg_elapsed_time_state` | `Float32` | Average time spent in elapsed state. |
| `quantiles_state` | `Float32` | 0.9, 0.95 and 0.99 quantiles state. Time in milliseconds (see[  Querying _state columns](https://www.tinybird.co/docs/forward/monitoring/service-datasources#querying-state-columns)   ). |
| `read_bytes` | `UInt64` | Read bytes in the operation. |
| `read_rows` | `UInt64` | Read rows in the Sink operation. |
| `written_rows` | `UInt64` | Written rows in the Sink operation. |
| `read_bytes` | `UInt64` | Read bytes in the operation. |
| `written_bytes` | `UInt64` | Written bytes in the operation. |
| `written_rows_quarantine` | `UInt64` | Quarantined rows in the operation. |
| `written_bytes_quarantine` | `UInt64` | Quarantined bytes in the operation. |
| `resource_tags` | `Array(String)` | Tags associated with the pipe when the request was made. |

### tinybird.endpoint_errors [¶](https://www.tinybird.co/docs/forward/monitoring/service-datasources#tinybird-endpoint-errors)

It provides the last 30 days errors of your published endpoints. Tinybird logs all errors with additional information in this data source.

| Field | Type | Description |
| --- | --- | --- |
| `start_datetime` | `DateTime` | Date and time when the API call started. |
| `request_id` | `String` | The id of the request that performed the operation. Format is ULID string. |
| `pipe_id` | `String` | If present, pipe id as returned in our[  pipes API](/docs/api-reference/pipe-api)  . |
| `pipe_name` | `String` | If present, pipe name as returned in our[  pipes API](/docs/api-reference/pipe-api)  . |
| `params` | `Nullable(String)` | URL query params included in the request. |
| `url` | `Nullable(String)` | URL pathname. |
| `status_code` | `Nullable(Int32)` | HTTP error code. |
| `error` | `Nullable(String)` | Error message. |
| `resource_tags` | `Array(String)` | Tags associated with the pipe when the request was made. |

### tinybird.kafka_ops_log [¶](https://www.tinybird.co/docs/forward/monitoring/service-datasources#tinybird-kafka-ops-log)

Contains all operations performed to your Kafka data sources during the last 30 days.

| Field | Type | Description |
| --- | --- | --- |
| `timestamp` | `DateTime` | Date and time when the operation took place. |
| `datasource_id` | `String` | Id of your data source. The data source id is consistent after renaming operations. You should use the id when you want to track name changes. |
| `topic` | `String` | Kafka topic. |
| `partition` | `Int16` | Partition number, or `-1`   for all partitions. |
| `msg_type` | `String` | 'info' for regular messages, 'warning' for issues related to the user's Kafka cluster, deserialization or materialized views, and 'error' for other issues. |
| `lag` | `Int64` | Number of messages behind for the partition. This is the difference between the high-water mark and the last commit offset. |
| `processed_messages` | `Int32` | Messages processed for a topic and partition. |
| `processed_bytes` | `Int32` | Amount of bytes processed. |
| `committed_messages` | `Int32` | Messages ingested for a topic and partition. |
| `msg` | `String` | Information in the case of warnings or errors. Empty otherwise. |

### tinybird.datasources_storage [¶](https://www.tinybird.co/docs/forward/monitoring/service-datasources#tinybird-datasources-storage)

Contains stats about your data sources storage.

Tinybird logs maximum values per hour, the same as when it calculates storage consumption.

| Field | Type | Description |
| --- | --- | --- |
| `datasource_id` | `String` | Id of your data source. The data source id is consistent after renaming operations. You should use the id when you want to track name changes. |
| `datasource_name` | `String` | Name of your data source. |
| `timestamp` | `DateTime` | When storage was tracked. By hour. |
| `bytes` | `UInt64` | Max number of bytes the data source has, not including quarantine. |
| `rows` | `UInt64` | Max number of rows the data source has, not including quarantine. |
| `bytes_quarantine` | `UInt64` | Max number of bytes the data source has in quarantine. |
| `rows_quarantine` | `UInt64` | Max number of rows the data source has in quarantine. |

### tinybird.releases_log (deprecated) [¶](https://www.tinybird.co/docs/forward/monitoring/service-datasources#tinybird-releases-log-deprecated)

Contains operations performed to your releases. Tinybird tracks the following operations:

| Event | Description |
| --- | --- |
| `init` | First Release is created on Git sync. |
| `override` | Release commit is overridden. `tb init --override-commit {{commit}}`  . |
| `deploy` | Resources from a commit are deployed to a Release. |
| `preview` | Release status is changed to preview. |
| `promote` | Release status is changed to live. |
| `post` | Resources from a commit are deployed to the live Release. |
| `rollback` | Rollback is done a previous Release is now live. |
| `delete` | Release is deleted. |

Tinybird logs all operations with additional information in this data source.

| Field | Type | Description |
| `timestamp` | `DateTime64` | Date and time when the operation took place. |
| `event_type` | `String` | Name of your data source. |
| `semver` | `String` | Semantic version identifies a release. |
| `commit` | `String` | Git sha commit related to the operation. |
| `token` | `String` | API call token identifier used. |
| `token_name` | `String` | API call token name used. |
| `result` | `String` | `ok`   | `error` |
| `error` | `String` | Detailed error message. |

### tinybird.sinks_ops_log [¶](https://www.tinybird.co/docs/forward/monitoring/service-datasources#tinybird-sinks-ops-log)

Contains all operations performed to your Sink pipes.

| Field | Type | Description |
| `timestamp` | `DateTime64` | Date and time when the operation took place. |
| `service` | `LowCardinality(String)` | Type of Sink (GCS, S3, and so on). |
| `pipe_id` | `String` | The ID of the Sink pipe. |
| `pipe_name` | `String` | the name of the Sink pipe. |
| `token_name` | `String` | Token name used. |
| `result` | `LowCardinality(String)` | `ok`   | `error` |
| `error` | `Nullable(String)` | Detailed error message. |
| `elapsed_time` | `Float64` | The duration of the operation in seconds. |
| `job_id` | `Nullable(String)` | ID of the job that performed the operation, if any. |
| `read_rows` | `UInt64` | Read rows in the Sink operation. |
| `written_rows` | `UInt64` | Written rows in the Sink operation. |
| `read_bytes` | `UInt64` | Read bytes in the operation. |
| `written_bytes` | `UInt64` | Written bytes in the operation. |
| `output` | `Array(String)` | The outputs of the operation. In the case of writing to a bucket, the name of the written files. |
| `parameters` | `Map(String, String)` | The parameters used. Useful to debug the parameter query values. |
| `options` | `Map(String, String)` | Extra information. You can access the values with `options['key']`   where key is one of: file_template, file_format, file_compression, bucket_path, execution_type. |
| `cpu_time` | `Float64` | The CPU time used by the sinks, in seconds. |

### tinybird.data_transfer [¶](https://www.tinybird.co/docs/forward/monitoring/service-datasources#tinybird-data-transfer)

Stats of data transferred per hour by a workspace.

| Field | Type | Description |
| `timestamp` | `DateTime` | Date and time data transferred is tracked. By hour. |
| `event` | `LowCardinality(String)` | Type of operation generated the data (ie. `sink`   ). |
| `origin_provider` | `LowCardinality(String)` | Provider data was transferred from. |
| `origin_region` | `LowCardinality(String)` | Region data was transferred from. |
| `destination_provider` | `LowCardinality(String)` | Provider data was transferred to. |
| `destination_region` | `LowCardinality(String)` | Region data was transferred to. |
| `kind` | `LowCardinality(String)` | `intra`   | `inter`   depending if the data moves within or outside the region. |

### tinybird.llm_usage [¶](https://www.tinybird.co/docs/forward/monitoring/service-datasources#tinybird-llm-usage)

Contains LLM usage metrics from Tinybird AI features including token consumption, costs, and model usage for each request in the workspace.

| Field | Type | Description |
| --- | --- | --- |
| `start_time` | `DateTime` | Request start timestamp. |
| `end_time` | `DateTime` | Request end timestamp. |
| `organization_id` | `String` | Organization identifier. |
| `organization_name` | `String` | Organization name. |
| `workspace_id` | `String` | Workspace identifier. |
| `workspace_name` | `String` | Workspace name. |
| `user_email` | `String` | Email of the user who made the request. |
| `request_id` | `String` | Unique identifier for the LLM request. |
| `prompt_tokens` | `UInt32` | Number of tokens in the prompt. |
| `completion_tokens` | `UInt32` | Number of tokens in the completion. |
| `total_tokens` | `UInt32` | Total number of tokens used. |
| `duration` | `Float32` | Request duration in seconds. |
| `cost` | `Float32` | Cost of the LLM request. |
| `origin` | `String` | Origin of the request. |
| `feature` | `String` | Tinybird AI feature used. |

### tinybird.jobs_log [¶](https://www.tinybird.co/docs/forward/monitoring/service-datasources#tinybird-jobs-log)

Contains all job executions performed in your workspace. Tinybird logs all jobs with extra information in this data source:

| Field | Type | Description |
| --- | --- | --- |
| `job_id` | `String` | Unique identifier for the job. |
| `job_type` | `LowCardinality(String)` | Type of job execution: `copy`  , `delete`  , `deployment`  , `dynamodb_sync`  , `gcs_sync`  , `import`  , `populate`  , `s3_sync`  , `sink`  . |
| `workspace_id` | `String` | Unique identifier for the workspace. |
| `pipe_id` | `String` | Unique identifier for the pipe. |
| `pipe_name` | `String` | Name of the pipe. |
| `created_at` | `DateTime` | Timestamp when the job was created. |
| `updated_at` | `DateTime` | Timestamp when the job was last updated. |
| `started_at` | `DateTime` | Timestamp when the job execution started. |
| `status` | `LowCardinality(String)` | Current status of the job. `waiting`  , `working`  , `done`  , `error`  , `cancelled`  . |
| `error` | `Nullable(String)` | Detailed error message if the result was error. |
| `job_metadata` | `JSON String` | Additional metadata related to the job execution. |

Learn more about how to track background jobs execution in the [Jobs monitoring guide](./jobs).

### tinybird.query_validator_log [¶](https://www.tinybird.co/docs/forward/monitoring/service-datasources#tinybird-query-validator-log)

Contains information about query validation executions for Pipes and Materialized Views in your Workspace.

| Field | Type | Description |
| --- | --- | --- |
| `host` | `LowCardinality(String)` | Name of the host the query was extracted from — might differ from the host the query was executed on. |
| `version` | `LowCardinality(String)` | The version in which the query was tested. |
| `stable_version` | `LowCardinality(String)` | The version that was running in the cluster when the query was validated. |
| `query_hash` | `UInt64` | The normalized query hash. |
| `query_last_execution` | `DateTime` | The last execution time of this query at the time when it was extracted. |
| `region` | `String` | The region in which the cluster is deployed. |
| `workspace` | `String` | The name of the Workspace, for Pipe queries. |
| `database` | `String` | The internal database name for this Workspace. |
| `pipe_name` | `String` | The name of the Pipe, for Pipe queries. |
| `error_code` | `Int16` | Error code returned by ClickHouse. |
| `error` | `String` | Detailed error message if the query failed. |
| `query_id` | `String` | ClickHouse query ID. For Pipe queries, this can be matched with `system.query_log`   . For Materialized Views, it contains the qualified internal table name. |
| `query` | `String` | The full query. |
| `fix_suggestion` | `String` | Optionally, a suggestion on how to fix the query. |
| `run_validation` | `DateTime` | The time when this query was validated. |

Learn more about how to monitor your queries for validation in the [guide](./query-validation).

## Query _state columns [¶](https://www.tinybird.co/docs/forward/monitoring/service-datasources#query-state-columns)

Several of the Service data sources include columns suffixed with `_state` . This suffix identifies columns with values that are in an intermediate aggregated state. When reading these columns, merge the intermediate states to get the final value.

To merge intermediate states, wrap the column in the original aggregation function and apply the `-Merge` combinator.

For example, to finalize the value of the `avg_duration_state` column, you use the `avgMerge` function:

##### finalize the value for the avg_duration_state column

SELECT
  date,
  avgMerge(avg_duration_state) avg_time,
  quantilesTimingMerge(0.9, 0.95, 0.99)(quantile_timing_state) quantiles_timing_in_ms_array
FROM tinybird.pipe_stats
where pipe_id = 'PIPE_ID'
group by date See [Combinators](/docs/sql-reference/functions/aggregate-functions#aggregate-function-combinators) to learn more about the `-Merge` combinator.

## Organization service data sources [¶](https://www.tinybird.co/docs/forward/monitoring/service-datasources#organization-service-data-sources)

The following is a complete list of available Organization Service data sources:

| Field | Description |
| --- | --- |
| `organization.workspaces` | Lists all Organization workspaces and related information, including name, IDs, databases, plan, when it was created, and whether it has been soft-deleted. |
| `organization.processed_data` | Information related to all processed data per day per workspace. |
| `organization.datasources_storage` | Equivalent to tinybird.datasources_storage but with data for all Organization workspaces. |
| `organization.pipe_stats` | Equivalent to tinybird.pipe_stats but with data for all Organization workspaces. |
| `organization.pipe_stats_rt` | Equivalent to tinybird.pipe_stats_rt but with data for all Organization workspaces. |
| `organization.datasources_ops_log` | Equivalent to tinybird.datasources_ops_log but with data for all Organization workspaces. |
| `organization.data_transfer` | Equivalent to tinybird.data_transfer but with data for all Organization workspaces. |
| `organization.jobs_log` | Equivalent to tinybird.jobs_log but with data for all Organization workspaces. |
| `organization.sinks_ops_log` | Equivalent to tinybird.sinks_ops_log but with data for all Organization workspaces. |
| `organization.bi_stats` | Equivalent to tinybird.bi_stats but with data for all Organization workspaces. |
| `organization.bi_stats_rt` | Equivalent to tinybird.bi_stats_rt but with data for all Organization workspaces. |
| `organization.endpoint_errors` | Equivalent to tinybird.endpoint_errors but with data for all Organization workspaces. |
| `organization.shared_infra_active_minutes` | Contains information about vCPU active minutes consumption aggregated by minute for all Organization workspaces. Only available for Developer and Enterprise plans in shared infrastructure. |
| `organization.shared_infra_qps_overages` | Contains information about QPS consumption and overages aggregated by second for all Organization workspaces. Only available for Developer and Enterprise plans in shared infrastructure. |
| `organization.llm_usage` | Equivalent to tinybird.llm_usage but with data for all Organization workspaces. |

To query Organization Service data sources, go to any workspace that belongs to the Organization and use the previous as regular Service data source from the Playground or within pipes. Use the admin `Token of an Organization Admin` . You can also copy your admin Token and make queries using your preferred method, like `tb sql`.

### metrics_logs service data source [¶](https://www.tinybird.co/docs/forward/monitoring/service-datasources#metrics-logs-service-data-source)

The `metrics_logs` Service data source is available in all the organization's workspaces. As with the rest of Organization Service data sources, it's only available to Organization administrators. New records for each of the metrics monitored are added every minute with the following schema:

| Field | Type | Description |
| --- | --- | --- |
| timestamp | DateTime | Timestamp of the metric |
| cluster | LowCardinality(String) | Name of the cluster |
| host | LowCardinality(String) | Name of the host |
| metric | LowCardinality(String) | Name of the metric |
| value | String | Value of the metric |
| description | LowCardinality(String) | Description of the metric |
| organization_id | String | ID of your organization |

The available metrics are the following:

| Metric | Description |
| --- | --- |
| MemoryTracking | Total amount of memory, in bytes, allocated by the server. |
| OSMemoryTotal | The total amount of memory on the host system, in bytes. |
| InstanceType | Instance type of the host. |
| Query | Number of executing queries. |
| NumberCPU | Number of CPUs. |
| LoadAverage1 | The whole system load, averaged with exponential smoothing over 1 minute. The load represents   the number of threads across all the processes (the scheduling entities of the OS kernel), that   are currently running by CPU or waiting for IO, or ready to run but not being scheduled at this   point of time. This number includes all the processes, not only the server. The number   can be greater than the number of CPU cores, if the system is overloaded, and many processes are   ready to run but waiting for CPU or IO. |
| LoadAverage15 | The whole system load, averaged with exponential smoothing over 15 minutes. The load represents   the number of threads across all the processes (the scheduling entities of the OS kernel), that   are currently running by CPU or waiting for IO, or ready to run but not being scheduled at this   point of time. This number includes all the processes, not only the server. The number   can be greater than the number of CPU cores, if the system is overloaded, and many processes are   ready to run but waiting for CPU or IO. |
| CPUUsage | The ratio of time the CPU core was running OS kernel (system) code or userspace code. This is a   system-wide metric, it includes all the processes on the host machine, not just the   server. This includes also the time when the CPU was under-utilized due to the   reasons internal to the CPU (memory loads, pipeline stalls, branch mispredictions, running   another SMT core). |
| ReplicasSumQueueSize | Sum queue size (in the number of operations like get, merge) across Replicated tables. |
| ReplicasSumMergesInQueue | Sum of merge operations in the queue (still to be applied) across Replicated tables. |
| ReplicasSumInsertsInQueue | Sum of insert operations in the queue (still to be applied) across Replicated tables. |


