--- title: "Create a new pipeline" method: POST path: "/api/v2/obs-pipelines/pipelines" tags: ["Observability Pipelines"] --- # Create a new pipeline `POST /api/v2/obs-pipelines/pipelines` Create a new pipeline. ## Request body - ObservabilityPipelineSpec — Input schema representing an observability pipeline configuration. Used in create and validate requests. - `data` ObservabilityPipelineSpecData, required — Contains the the pipeline configuration. - `attributes` ObservabilityPipelineDataAttributes, required — Defines the pipeline’s name and its components (sources, processors, and destinations). - `config` ObservabilityPipelineConfig, required — Specifies the pipeline's configuration, including its sources, processors, and destinations. - `destinations` ObservabilityPipelineConfigDestinationItem[], required — A list of destination components where processed logs are sent. - union — A destination for the pipeline. - ObservabilityPipelineElasticsearchDestination — The `elasticsearch` destination writes logs or metrics to an Elasticsearch cluster. **Supported pipeline types:** logs, metrics - … - ObservabilityPipelineHttpClientDestination — The `http_client` destination sends data to an HTTP endpoint. **Supported pipeline types:** logs, metrics - … - ObservabilityPipelineAmazonOpenSearchDestination — The `amazon_opensearch` destination writes logs to Amazon OpenSearch. **Supported pipeline types:** logs - … - ObservabilityPipelineAmazonS3Destination — The `amazon_s3` destination sends your logs in Datadog-rehydratable format to an Amazon S3 bucket for archiving. **Supported pipeline types:** logs - … - ObservabilityPipelineAmazonS3GenericDestination — The `amazon_s3_generic` destination sends your logs to an Amazon S3 bucket. **Supported pipeline types:** logs - … - ObservabilityPipelineAmazonSecurityLakeDestination — The `amazon_security_lake` destination sends your logs to Amazon Security Lake. **Supported pipeline types:** logs - … - AzureStorageDestination — The `azure_storage` destination forwards logs to an Azure Blob Storage container. **Supported pipeline types:** logs - … - ObservabilityPipelineClickhouseDestination — The `clickhouse` destination sends log events to a ClickHouse database table over HTTP. **Supported pipeline types:** logs. - … - ObservabilityPipelineCloudPremDestination — The `cloud_prem` destination sends logs to Datadog CloudPrem. **Supported pipeline types:** logs - … - ObservabilityPipelineCrowdStrikeNextGenSiemDestination — The `crowdstrike_next_gen_siem` destination forwards logs to CrowdStrike Next Gen SIEM. **Supported pipeline types:** logs - … - ObservabilityPipelineDatadogLogsDestination — The `datadog_logs` destination forwards logs to Datadog Log Management. **Supported pipeline types:** logs - … - ObservabilityPipelineGoogleChronicleDestination — The `google_chronicle` destination sends logs to Google Chronicle. **Supported pipeline types:** logs - … - ObservabilityPipelineGoogleCloudStorageDestination — The `google_cloud_storage` destination stores logs in a Google Cloud Storage (GCS) bucket. It requires a bucket name, Google Cloud authentication, and metadata fields. **Supported pipeline types:** logs - … - ObservabilityPipelineGooglePubSubDestination — The `google_pubsub` destination publishes logs to a Google Cloud Pub/Sub topic. **Supported pipeline types:** logs - … - ObservabilityPipelineKafkaDestination — The `kafka` destination sends logs to Apache Kafka topics. **Supported pipeline types:** logs - … - MicrosoftSentinelDestination — The `microsoft_sentinel` destination forwards logs to Microsoft Sentinel. **Supported pipeline types:** logs - … - ObservabilityPipelineNewRelicDestination — The `new_relic` destination sends logs to the New Relic platform. **Supported pipeline types:** logs - … - ObservabilityPipelineOpenSearchDestination — The `opensearch` destination writes logs to an OpenSearch cluster. **Supported pipeline types:** logs - … - ObservabilityPipelineRsyslogDestination — The `rsyslog` destination forwards logs to an external `rsyslog` server over TCP or UDP using the syslog protocol. **Supported pipeline types:** logs - … - ObservabilityPipelineSentinelOneDestination — The `sentinel_one` destination sends logs to SentinelOne. **Supported pipeline types:** logs - … - ObservabilityPipelineSocketDestination — The `socket` destination sends logs over TCP or UDP to a remote server. **Supported pipeline types:** logs - … - ObservabilityPipelineSplunkHecDestination — The `splunk_hec` destination forwards logs to Splunk using the HTTP Event Collector (HEC). **Supported pipeline types:** logs - … - ObservabilityPipelineSumoLogicDestination — The `sumo_logic` destination forwards logs to Sumo Logic. **Supported pipeline types:** logs - … - ObservabilityPipelineSyslogNgDestination — The `syslog_ng` destination forwards logs to an external `syslog-ng` server over TCP or UDP using the syslog protocol. **Supported pipeline types:** logs - … - ObservabilityPipelineDatabricksZerobusDestination — The `databricks_zerobus` destination sends logs to Databricks using the Zerobus ingestion API, streaming data directly into your Databricks Lakehouse. **Supported pipeline types:** Logs, rehydration - … - ObservabilityPipelineDatadogMetricsDestination — The `datadog_metrics` destination forwards metrics to Datadog. **Supported pipeline types:** metrics - … - ObservabilityPipelineSplunkHecMetricsDestination — The `splunk_hec_metrics` destination forwards metrics to Splunk using the HTTP Event Collector (HEC). **Supported pipeline types:** metrics - … - `pipeline_type` 'logs' | 'metrics' — The type of data being ingested. Defaults to `logs` if not specified. - `processor_groups` ObservabilityPipelineConfigProcessorGroup[] — A list of processor groups that transform or enrich log data. - `display_name` string — The display name for a component. - `enabled` boolean, required — Whether this processor group is enabled. - `id` string, required — The unique identifier for the processor group. - `include` string, required — Conditional expression for when this processor group should execute. - `inputs` string[], required — A list of IDs for components whose output is used as the input for this processor group. - `processors` ObservabilityPipelineConfigProcessorItem[], required — Processors applied sequentially within this group. Events flow through each processor in order. - union — A processor for the pipeline. - … - `processors` ObservabilityPipelineConfigProcessorGroup[] — A list of processor groups that transform or enrich log data. **Deprecated:** This field is deprecated, you should now use the processor_groups field. - `display_name` string — The display name for a component. - `enabled` boolean, required — Whether this processor group is enabled. - `id` string, required — The unique identifier for the processor group. - `include` string, required — Conditional expression for when this processor group should execute. - `inputs` string[], required — A list of IDs for components whose output is used as the input for this processor group. - `processors` ObservabilityPipelineConfigProcessorItem[], required — Processors applied sequentially within this group. Events flow through each processor in order. - union — A processor for the pipeline. - … - `sources` ObservabilityPipelineConfigSourceItem[], required — A list of configured data sources for the pipeline. - union — A data source for the pipeline. - ObservabilityPipelineDatadogAgentSource — The `datadog_agent` source collects logs/metrics from the Datadog Agent. **Supported pipeline types:** logs, metrics - … - ObservabilityPipelineAmazonDataFirehoseSource — The `amazon_data_firehose` source ingests logs from AWS Data Firehose. **Supported pipeline types:** logs - … - ObservabilityPipelineAmazonS3Source — The `amazon_s3` source ingests logs from an Amazon S3 bucket. It supports AWS authentication, TLS encryption, and configurable compression. **Supported pipeline types:** logs - … - ObservabilityPipelineFluentBitSource — The `fluent_bit` source ingests logs from Fluent Bit. **Supported pipeline types:** logs - … - ObservabilityPipelineFluentdSource — The `fluentd` source ingests logs from a Fluentd-compatible service. **Supported pipeline types:** logs - … - ObservabilityPipelineGooglePubSubSource — The `google_pubsub` source ingests logs from a Google Cloud Pub/Sub subscription. **Supported pipeline types:** logs - … - ObservabilityPipelineHttpClientSource — The `http_client` source scrapes logs from HTTP endpoints at regular intervals. **Supported pipeline types:** logs - … - ObservabilityPipelineHttpServerSource — The `http_server` source collects logs over HTTP POST from external services. **Supported pipeline types:** logs - … - ObservabilityPipelineKafkaSource — The `kafka` source ingests data from Apache Kafka topics. **Supported pipeline types:** logs - … - ObservabilityPipelineLogstashSource — The `logstash` source ingests logs from a Logstash forwarder. **Supported pipeline types:** logs - … - ObservabilityPipelineRsyslogSource — The `rsyslog` source listens for logs over TCP or UDP from an `rsyslog` server using the syslog protocol. **Supported pipeline types:** logs - … - ObservabilityPipelineSocketSource — The `socket` source ingests logs over TCP or UDP. **Supported pipeline types:** logs - … - ObservabilityPipelineSplunkHecSource — The `splunk_hec` source implements the Splunk HTTP Event Collector (HEC) API. **Supported pipeline types:** logs - … - ObservabilityPipelineSplunkTcpSource — The `splunk_tcp` source receives logs from a Splunk Universal Forwarder over TCP. TLS is supported for secure transmission. **Supported pipeline types:** logs - … - ObservabilityPipelineSumoLogicSource — The `sumo_logic` source receives logs from Sumo Logic collectors. **Supported pipeline types:** logs - … - ObservabilityPipelineSyslogNgSource — The `syslog_ng` source listens for logs over TCP or UDP from a `syslog-ng` server using the syslog protocol. **Supported pipeline types:** logs - … - ObservabilityPipelineWebsocketSource — The `websocket` source ingests logs from a WebSocket server using the `ws://` or `wss://` protocol. **Supported pipeline types:** logs. - … - ObservabilityPipelineOpentelemetrySource — The `opentelemetry` source receives telemetry data using the OpenTelemetry Protocol (OTLP) over gRPC and HTTP. **Supported pipeline types:** logs, metrics - … - `use_legacy_search_syntax` boolean — Set to `true` to continue using the legacy search syntax while migrating filter queries. After migrating all queries to the new syntax, set to `false`. The legacy syntax is deprecated and will eventually be removed. Requires Observability Pipelines Worker 2.11 or later. Only applies to `logs` pipelines. This field is ignored for `metrics` pipelines. See [Upgrade Your Filter Queries to the New Search Syntax](https://docs.datadoghq.com/observability_pipelines/guide/upgrade_your_filter_queries_to_the_new_search_syntax/) for more information. - `name` string, required — Name of the pipeline. - `type` string, required — The resource type identifier. For pipeline resources, this should always be set to `pipelines`. ## Response `201` OK - ObservabilityPipeline — Top-level schema representing a pipeline. - `data` ObservabilityPipelineData, required — Contains the pipeline’s ID, type, and configuration attributes. - `attributes` ObservabilityPipelineDataAttributes, required — Defines the pipeline’s name and its components (sources, processors, and destinations). - `config` ObservabilityPipelineConfig, required — Specifies the pipeline's configuration, including its sources, processors, and destinations. - `destinations` ObservabilityPipelineConfigDestinationItem[], required — A list of destination components where processed logs are sent. - union — A destination for the pipeline. - ObservabilityPipelineElasticsearchDestination — The `elasticsearch` destination writes logs or metrics to an Elasticsearch cluster. **Supported pipeline types:** logs, metrics - … - ObservabilityPipelineHttpClientDestination — The `http_client` destination sends data to an HTTP endpoint. **Supported pipeline types:** logs, metrics - … - ObservabilityPipelineAmazonOpenSearchDestination — The `amazon_opensearch` destination writes logs to Amazon OpenSearch. **Supported pipeline types:** logs - … - ObservabilityPipelineAmazonS3Destination — The `amazon_s3` destination sends your logs in Datadog-rehydratable format to an Amazon S3 bucket for archiving. **Supported pipeline types:** logs - … - ObservabilityPipelineAmazonS3GenericDestination — The `amazon_s3_generic` destination sends your logs to an Amazon S3 bucket. **Supported pipeline types:** logs - … - ObservabilityPipelineAmazonSecurityLakeDestination — The `amazon_security_lake` destination sends your logs to Amazon Security Lake. **Supported pipeline types:** logs - … - AzureStorageDestination — The `azure_storage` destination forwards logs to an Azure Blob Storage container. **Supported pipeline types:** logs - … - ObservabilityPipelineClickhouseDestination — The `clickhouse` destination sends log events to a ClickHouse database table over HTTP. **Supported pipeline types:** logs. - … - ObservabilityPipelineCloudPremDestination — The `cloud_prem` destination sends logs to Datadog CloudPrem. **Supported pipeline types:** logs - … - ObservabilityPipelineCrowdStrikeNextGenSiemDestination — The `crowdstrike_next_gen_siem` destination forwards logs to CrowdStrike Next Gen SIEM. **Supported pipeline types:** logs - … - ObservabilityPipelineDatadogLogsDestination — The `datadog_logs` destination forwards logs to Datadog Log Management. **Supported pipeline types:** logs - … - ObservabilityPipelineGoogleChronicleDestination — The `google_chronicle` destination sends logs to Google Chronicle. **Supported pipeline types:** logs - … - ObservabilityPipelineGoogleCloudStorageDestination — The `google_cloud_storage` destination stores logs in a Google Cloud Storage (GCS) bucket. It requires a bucket name, Google Cloud authentication, and metadata fields. **Supported pipeline types:** logs - … - ObservabilityPipelineGooglePubSubDestination — The `google_pubsub` destination publishes logs to a Google Cloud Pub/Sub topic. **Supported pipeline types:** logs - … - ObservabilityPipelineKafkaDestination — The `kafka` destination sends logs to Apache Kafka topics. **Supported pipeline types:** logs - … - MicrosoftSentinelDestination — The `microsoft_sentinel` destination forwards logs to Microsoft Sentinel. **Supported pipeline types:** logs - … - ObservabilityPipelineNewRelicDestination — The `new_relic` destination sends logs to the New Relic platform. **Supported pipeline types:** logs - … - ObservabilityPipelineOpenSearchDestination — The `opensearch` destination writes logs to an OpenSearch cluster. **Supported pipeline types:** logs - … - ObservabilityPipelineRsyslogDestination — The `rsyslog` destination forwards logs to an external `rsyslog` server over TCP or UDP using the syslog protocol. **Supported pipeline types:** logs - … - ObservabilityPipelineSentinelOneDestination — The `sentinel_one` destination sends logs to SentinelOne. **Supported pipeline types:** logs - … - ObservabilityPipelineSocketDestination — The `socket` destination sends logs over TCP or UDP to a remote server. **Supported pipeline types:** logs - … - ObservabilityPipelineSplunkHecDestination — The `splunk_hec` destination forwards logs to Splunk using the HTTP Event Collector (HEC). **Supported pipeline types:** logs - … - ObservabilityPipelineSumoLogicDestination — The `sumo_logic` destination forwards logs to Sumo Logic. **Supported pipeline types:** logs - … - ObservabilityPipelineSyslogNgDestination — The `syslog_ng` destination forwards logs to an external `syslog-ng` server over TCP or UDP using the syslog protocol. **Supported pipeline types:** logs - … - ObservabilityPipelineDatabricksZerobusDestination — The `databricks_zerobus` destination sends logs to Databricks using the Zerobus ingestion API, streaming data directly into your Databricks Lakehouse. **Supported pipeline types:** Logs, rehydration - … - ObservabilityPipelineDatadogMetricsDestination — The `datadog_metrics` destination forwards metrics to Datadog. **Supported pipeline types:** metrics - … - ObservabilityPipelineSplunkHecMetricsDestination — The `splunk_hec_metrics` destination forwards metrics to Splunk using the HTTP Event Collector (HEC). **Supported pipeline types:** metrics - … - `pipeline_type` 'logs' | 'metrics' — The type of data being ingested. Defaults to `logs` if not specified. - `processor_groups` ObservabilityPipelineConfigProcessorGroup[] — A list of processor groups that transform or enrich log data. - `display_name` string — The display name for a component. - `enabled` boolean, required — Whether this processor group is enabled. - `id` string, required — The unique identifier for the processor group. - `include` string, required — Conditional expression for when this processor group should execute. - `inputs` string[], required — A list of IDs for components whose output is used as the input for this processor group. - `processors` ObservabilityPipelineConfigProcessorItem[], required — Processors applied sequentially within this group. Events flow through each processor in order. - union — A processor for the pipeline. - … - `processors` ObservabilityPipelineConfigProcessorGroup[] — A list of processor groups that transform or enrich log data. **Deprecated:** This field is deprecated, you should now use the processor_groups field. - `display_name` string — The display name for a component. - `enabled` boolean, required — Whether this processor group is enabled. - `id` string, required — The unique identifier for the processor group. - `include` string, required — Conditional expression for when this processor group should execute. - `inputs` string[], required — A list of IDs for components whose output is used as the input for this processor group. - `processors` ObservabilityPipelineConfigProcessorItem[], required — Processors applied sequentially within this group. Events flow through each processor in order. - union — A processor for the pipeline. - … - `sources` ObservabilityPipelineConfigSourceItem[], required — A list of configured data sources for the pipeline. - union — A data source for the pipeline. - ObservabilityPipelineDatadogAgentSource — The `datadog_agent` source collects logs/metrics from the Datadog Agent. **Supported pipeline types:** logs, metrics - … - ObservabilityPipelineAmazonDataFirehoseSource — The `amazon_data_firehose` source ingests logs from AWS Data Firehose. **Supported pipeline types:** logs - … - ObservabilityPipelineAmazonS3Source — The `amazon_s3` source ingests logs from an Amazon S3 bucket. It supports AWS authentication, TLS encryption, and configurable compression. **Supported pipeline types:** logs - … - ObservabilityPipelineFluentBitSource — The `fluent_bit` source ingests logs from Fluent Bit. **Supported pipeline types:** logs - … - ObservabilityPipelineFluentdSource — The `fluentd` source ingests logs from a Fluentd-compatible service. **Supported pipeline types:** logs - … - ObservabilityPipelineGooglePubSubSource — The `google_pubsub` source ingests logs from a Google Cloud Pub/Sub subscription. **Supported pipeline types:** logs - … - ObservabilityPipelineHttpClientSource — The `http_client` source scrapes logs from HTTP endpoints at regular intervals. **Supported pipeline types:** logs - … - ObservabilityPipelineHttpServerSource — The `http_server` source collects logs over HTTP POST from external services. **Supported pipeline types:** logs - … - ObservabilityPipelineKafkaSource — The `kafka` source ingests data from Apache Kafka topics. **Supported pipeline types:** logs - … - ObservabilityPipelineLogstashSource — The `logstash` source ingests logs from a Logstash forwarder. **Supported pipeline types:** logs - … - ObservabilityPipelineRsyslogSource — The `rsyslog` source listens for logs over TCP or UDP from an `rsyslog` server using the syslog protocol. **Supported pipeline types:** logs - … - ObservabilityPipelineSocketSource — The `socket` source ingests logs over TCP or UDP. **Supported pipeline types:** logs - … - ObservabilityPipelineSplunkHecSource — The `splunk_hec` source implements the Splunk HTTP Event Collector (HEC) API. **Supported pipeline types:** logs - … - ObservabilityPipelineSplunkTcpSource — The `splunk_tcp` source receives logs from a Splunk Universal Forwarder over TCP. TLS is supported for secure transmission. **Supported pipeline types:** logs - … - ObservabilityPipelineSumoLogicSource — The `sumo_logic` source receives logs from Sumo Logic collectors. **Supported pipeline types:** logs - … - ObservabilityPipelineSyslogNgSource — The `syslog_ng` source listens for logs over TCP or UDP from a `syslog-ng` server using the syslog protocol. **Supported pipeline types:** logs - … - ObservabilityPipelineWebsocketSource — The `websocket` source ingests logs from a WebSocket server using the `ws://` or `wss://` protocol. **Supported pipeline types:** logs. - … - ObservabilityPipelineOpentelemetrySource — The `opentelemetry` source receives telemetry data using the OpenTelemetry Protocol (OTLP) over gRPC and HTTP. **Supported pipeline types:** logs, metrics - … - `use_legacy_search_syntax` boolean — Set to `true` to continue using the legacy search syntax while migrating filter queries. After migrating all queries to the new syntax, set to `false`. The legacy syntax is deprecated and will eventually be removed. Requires Observability Pipelines Worker 2.11 or later. Only applies to `logs` pipelines. This field is ignored for `metrics` pipelines. See [Upgrade Your Filter Queries to the New Search Syntax](https://docs.datadoghq.com/observability_pipelines/guide/upgrade_your_filter_queries_to_the_new_search_syntax/) for more information. - `name` string, required — Name of the pipeline. - `id` string, required — Unique identifier for the pipeline. - `type` string, required — The resource type identifier. For pipeline resources, this should always be set to `pipelines`. ## Other responses - `400` — Bad Request - `403` — Not Authorized - `409` — Conflict - `429` — Too many requests --- [API](https://skmtc.net/datadog/apis/api-v2.md) · [All operations](https://skmtc.net/datadog/apis/api-v2/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/datadog/api-v2/revisions/da68bf029e4c/schema)