---
title: "Create or update the dataset definition of a DQ Job"
method: POST
path: "/v3/datasetDefs"
tags: ["DQ Dataset Definitions API"]
---

# Create or update the dataset definition of a DQ Job

`POST /v3/datasetDefs`

Creates or updates the dataset definition of a DQ Job. 
Pre-authorization requirements: 

- `hasRole('ROLE_ADMIN') or hasRole('ROLE_DATASET_MANAGER') or hasDatasetAccess(#datasetDef.dataset)`

## Request body

- DatasetDefDTO
  - `dataset` string
  - `runId` string
  - `runIdEnd` string
  - `runState` 'DRAFT' | 'DRY' | 'CERTIFIED'
  - `passFail` integer
  - `passFailLimit` integer
  - `jobDescription` string
  - `jobId` IdentifierDTO
    - `id` integer
    - `uuid` string, uuid
  - `coreMaxActiveConnections` integer
  - `linkId` string[]
  - `licenseKey` string
  - `logFile` string
  - `logLevel` string
  - `hootOnly` boolean
  - `prettyPrint` boolean
  - `useTemplate` boolean
  - `parallel` boolean
  - `plan` boolean
  - `dataPreviewOff` boolean
  - `datasetSafeOff` boolean
  - `obslimit` integer
  - `pgUser` string
  - `pgPassword` string
  - `host` string
  - `port` string
  - `user` string
  - `alertEmail` string
  - `scheduleTime` string
  - `schemaScore` number, double
  - `optionAppend` string
  - `keyDelimiter` string
  - `agentId` IdentifierDTO
    - `id` integer
    - `uuid` string, uuid
  - `load` LoadDefDTO
    - `readonly` boolean
    - `passwordManager` string
    - `alias` string
    - `query` string
    - `queryHistory` string
    - `key` string
    - `expression` string
    - `addDateColumn` boolean
    - `zeroFillNull` boolean
    - `replaceNulls` string
    - `stringMode` boolean
    - `operator` string
    - `dateColumn` string
    - `transform` string
    - `filter` string
    - `filterNot` string
    - `sample` number, double
    - `backRun` integer
    - `backRunBin` 'MINUTE' | 'HOUR' | 'DAY' | 'WEEK' | 'MONTH' | 'QUARTER' | 'YEAR'
    - `unionLookBack` boolean
    - `unionLookBackMinRow` integer
    - `cache` boolean
    - `dateFormat` string
    - `timeFormat` string
    - `timestamp` boolean
    - `filePath` string
    - `fileQuery` string
    - `fullFile` boolean
    - `fileHeader` string
    - `inferSchema` boolean
    - `fileType` string
    - `delimiter` string
    - `fileCharSet` string
    - `skipLines` integer
    - `avroSchema` string
    - `xmlRowTag` string
    - `flatten` boolean
    - `handleMaps` boolean
    - `handleMixedJson` boolean
    - `multiLine` boolean
    - `lib` string
    - `additionalLib` string
    - `driverName` string
    - `connectionName` string
    - `connectionUrl` string
    - `userName` string
    - `password` string
    - `connectionProperties` object
    - `hiveNative` boolean
    - `hiveNativeHWC` boolean
    - `useSql` boolean
    - `columnName` string
    - `lowerBound` string
    - `upperBound` string
    - `numPartitions` integer
    - `partitionNumber` integer
    - `escapeWithBackTick` boolean
    - `escapeWithSingleQuote` boolean
    - `escapeWithDoubleQuote` boolean
    - `escapeCharacter` string
    - `checkHeader` boolean
    - `archiveConnection` string
    - `coreFetchMode` boolean
    - `archiveBreaks` boolean
    - `ruleSerial` boolean
  - `pushdown` PushDownDefDTO
    - `dataset` string
    - `connectionName` string
    - `maxConnections` integer
    - `sourceQuery` string
    - `backRuns` integer
    - `backRunBin` string
    - `dateFormatType` string
    - `threads` integer
    - `manualSourceQuery` boolean
    - `key` string
    - `sourceOutputSchema` string
    - `sourceBreakRules` boolean
    - `sourceBreakOutliers` boolean
    - `sourceBreakDupes` boolean
    - `sourceBreakShapes` boolean
    - `sourceBreakResults` boolean
    - `sqlLoggingToggle` boolean
    - `unionLookback` boolean
    - `unionLookbackMin` integer
  - `outliers` OutlierDefDTO[]
    - `id` integer
    - `dataset` string
    - `only` boolean
    - `lookback` integer
    - `key` string[]
    - `include` string[]
    - `exclude` string[]
    - `dateColumn` string
    - `timeColumn` string
    - `timeBin` 'MIN' | 'HOUR' | 'DAY' | 'WEEK' | 'MONTH' | 'QUARTER' | 'YEAR'
    - `timeBinQuery` string
    - `categorical` boolean
    - `by` 'MIN' | 'HOUR' | 'DAY' | 'WEEK' | 'MONTH' | 'QUARTER' | 'YEAR'
    - `limit` integer
    - `minHistory` integer
    - `historyLimit` integer
    - `score` number, double
    - `aggFunc` string
    - `aggQuery` string
    - `query` string
    - `q1` number, double
    - `q3` number, double
    - `categoricalColumnConcatenation` boolean
    - `limitCategorical` integer
    - `measurementUnit` string
    - `multiplierUpper` number, double
    - `multiplierLower` number, double
    - `record` boolean
    - `filter` string
    - `combine` boolean
    - `categoricalConfidenceType` string
    - `categoricalTopN` integer
    - `categoricalBottomN` integer
    - `categoricalMaxConfidence` number, double
    - `categoricalMaxFrequencyPercentile` number, double
    - `categoricalMinFrequency` integer
    - `categoricalMinVariance` number, double
    - `categoricalMaxCategoryN` integer
    - `categoricalParallel` boolean
    - `categoricalAlgorithm` string
    - `categoricalAlgorithmParameters` object
    - `categoricalAlgorithmParametersString` string
  - `patterns` PatternDefDTO[]
    - `id` integer
    - `dataset` string
    - `only` boolean
    - `lookback` integer
    - `key` string[]
    - `dateColumn` string
    - `include` string[]
    - `exclude` string[]
    - `score` number, double
    - `minSupport` number, double
    - `confidence` number, double
    - `limit` integer
    - `query` string
    - `filter` string
    - `timeBin` 'MINUTE' | 'HOUR' | 'DAY' | 'WEEK' | 'MONTH' | 'QUARTER' | 'YEAR'
  - `dupe` DupeDefDTO
    - `on` boolean
    - `only` boolean
    - `include` string[]
    - `exclude` string[]
    - `depth` integer
    - `lowerBound` integer
    - `upperBound` integer
    - `approximate` integer
    - `limitPerDupe` integer
    - `checkHeader` boolean
    - `filter` string
    - `ignoreCase` boolean
    - `score` number, double
    - `limit` integer
  - `profile` ProfileDefDTO
    - `on` boolean
    - `only` boolean
    - `include` string[]
    - `exclude` string[]
    - `correlation` boolean
    - `histogram` boolean
    - `semantic` boolean
    - `dataConceptId` integer
    - `limit` integer
    - `histogramLimit` integer
    - `score` number, double
    - `shape` boolean
    - `shapeTotalScore` integer
    - `shapeSensitivity` number, double
    - `shapeMaxPerCol` integer
    - `shapeMaxColSize` integer
    - `shapeGranular` boolean
    - `behavioralDimension` string
    - `behavioralDimensionGroup` string
    - `behavioralValueColumn` string
    - `behaviorScoreOff` boolean
    - `behaviorLookback` integer
    - `behaviorMinSupport` integer
    - `profilePushDown` string[]
    - `behaviorRowCheck` boolean
    - `behaviorTimeCheck` boolean
    - `behaviorMinValueCheck` boolean
    - `behaviorMaxValueCheck` boolean
    - `behaviorMeanValueCheck` boolean
    - `behaviorNullCheck` boolean
    - `behaviorEmptyCheck` boolean
    - `behaviorUniqueCheck` boolean
    - `profileStringLength` boolean
    - `detectStringNumerics` boolean
    - `detectTopnBotn` boolean
    - `detectScalePrecision` boolean
    - `adaptiveTier` string
    - `advancedTier` boolean
    - `filter` string
  - `source` SourceDefDTO
    - `on` boolean
    - `only` boolean
    - `validateValues` boolean
    - `key` string[]
    - `include` string[]
    - `exclude` string[]
    - `includeSrc` string[]
    - `excludeSrc` string[]
    - `map` object
    - `score` number, double
    - `limit` integer
    - `sourcePushDownCount` boolean
    - `checkType` boolean
    - `checkCase` boolean
    - `validateSchemaOrder` boolean
    - `matches` boolean
    - `validateValuesThreshold` number, double
    - `validateValuesThresholdStrictDownScore` boolean
    - `validateValuesShowAll` boolean
    - `validateValuesIgnoreNull` boolean
    - `validateValuesIgnoreEmpty` boolean
    - `validateValuesIgnorePrecision` boolean
    - `validateValuesTrim` boolean
    - `validateValuesShowMissingKeys` boolean
    - `validateSrcJoinOnly` boolean
    - `validateValuesFilter` string
    - `dataset` string
    - `driverName` string
    - `user` string
    - `password` string
    - `passwordManager` string
    - `connectionName` string
    - `connectionUrl` string
    - `query` string
    - `lib` string
    - `connectionProperties` object
    - `filePath` string
    - `fileQuery` string
    - `fullFile` boolean
    - `header` string
    - `skipLines` integer
    - `inferSchema` boolean
    - `fileType` string
    - `delimiter` string
    - `fileCharSet` string
    - `avroSchema` string
    - `xmlRowTag` string
    - `flatten` boolean
    - `handleMaps` boolean
    - `handleMixedJson` boolean
    - `multiLine` boolean
    - `filterCols` string[]
  - `rule` RuleDefDTO
    - `on` boolean
    - `only` boolean
    - `lib` string
    - `name` string
    - `absoluteScoring` boolean
    - `ruleBreakPreviewLimit` integer
  - `colMatch` ColMatchDefDTO
    - `colMatchParallelProcesses` integer
    - `colMatchDurationMins` integer
    - `colMatchBatchSize` integer
    - `level` string
    - `fuzzyDistance` integer
    - `connectionList` Connection[]
      - `dataset` string
      - `url` string
      - `user` string
      - `password` string
      - `query` string
      - `filePath` string
      - `driver` string
      - `driverProps` object
  - `spark` SparkDefDTO
    - `numExecutors` integer
    - `driverMemory` string
    - `executorMemory` string
    - `executorCores` integer
    - `conf` string
    - `queue` string
    - `master` string
    - `principal` string
    - `keyTab` string
    - `deployMode` string
    - `jars` string[]
    - `packages` string[]
    - `files` string[]
  - `env` EnvDefDTO
    - `dataset` string
    - `jdbcPrincipal` string
    - `jdbcKeyTab` string
  - `record` RecordDefDTO
    - `on` boolean
    - `in` string
    - `notIn` string
    - `include` string[]
    - `percDeltaLimit` number, double
    - `score` number, double
    - `dataset` string
    - `dateColumn` string
    - `keyColumn` string
    - `timeBin` 'MIN' | 'HOUR' | 'DAY' | 'WEEK' | 'MONTH' | 'QUARTER' | 'YEAR'
    - `by` 'MIN' | 'HOUR' | 'DAY' | 'WEEK' | 'MONTH' | 'QUARTER' | 'YEAR'
  - `stream` StreamDefDTO
    - `type` 'Kafka'
    - `kafkaDef` KafkaDefDTO
      - `kafkaGroup` string
      - `kafkaBroker` string
      - `kafkaPort` string
      - `kafkaTopic` string
      - `kafkaKeySerde` string
      - `kafkaValSerde` string
      - `kafkaSaslEnabled` boolean
      - `kafkaSaslService` string
      - `kafkaSslEnabled` boolean
      - `streamSchemaRegistryUrl` string
      - `streamSchemaRegistryUser` string
      - `streamSchemaRegistryPassword` string
      - `streamInterval` string
      - `streamSchema` string
      - `streamFormat` string
      - `streamMaxLull` string
  - `pipeline` string[]
  - `runDate` string, date-time
  - `runDateEnd` string, date-time
  - `metaTags` string[]
  - `outlierConfiguration` union — Outlier configurations
    - object
      - `columnSensitivitySettings` OutlierDefColumnSensitivitySettingsDTO[] — List of column sensitivity settings
        - `name` string
        - `weight` number, double
        - `unit` 'DEFAULT' | 'UNIT_0_001' | 'UNIT_0_01' | 'UNIT_0_1' | 'UNIT_1_0' | 'UNIT_10_0' | 'UNIT_100_0' | 'UNIT_1000_0' | 'UNIT_10000_0' | 'UNIT_100000_0' | 'UNIT_1000000_0' | 'UNIT_10000000_0'
      - `configurations` object[]
        - `id` integer
        - `key` string[]
        - `include` string[]
        - `exclude` string[]
        - `timeColumn` string
        - `timeBinQuery` string
        - `limit` integer
        - `minHistory` integer
        - `historyLimit` integer
        - `score` number, double
        - `aggFunc` string
        - `aggQuery` string
        - `query` string
        - `categoricalColumnConcatenation` boolean
        - `limitCategorical` integer
        - `measurementUnit` string
        - `multiplierUpper` number, double
        - `multiplierLower` number, double
        - `record` boolean
        - `filter` string
        - `combine` boolean
        - `categoricalConfidenceType` string
        - `categoricalTopN` integer
        - `categoricalBottomN` integer
        - `categoricalMaxConfidence` number, double
        - `categoricalMaxFrequencyPercentile` number, double
        - `categoricalMinFrequency` integer
        - `categoricalMinVariance` number, double
        - `categoricalMaxCategoryN` integer
        - `categoricalParallel` boolean
        - `categoricalAlgorithm` string
        - `categoricalAlgorithmParameters` object
        - `categoricalAlgorithmParametersString` string
      - `only` boolean
      - `lookback` integer
      - `dateColumn` string
      - `timeBin` 'MIN' | 'HOUR' | 'DAY' | 'WEEK' | 'MONTH' | 'QUARTER' | 'YEAR'
      - `by` 'MIN' | 'HOUR' | 'DAY' | 'WEEK' | 'MONTH' | 'QUARTER' | 'YEAR'
      - `q1` number, double
      - `q3` number, double
      - `categorical` boolean
    - object
      - `columnSensitivitySettings` OutlierDefColumnSensitivitySettingsDTO[] — List of column sensitivity settings
        - `name` string
        - `weight` number, double
        - `unit` 'DEFAULT' | 'UNIT_0_001' | 'UNIT_0_01' | 'UNIT_0_1' | 'UNIT_1_0' | 'UNIT_10_0' | 'UNIT_100_0' | 'UNIT_1000_0' | 'UNIT_10000_0' | 'UNIT_100000_0' | 'UNIT_1000000_0' | 'UNIT_10000000_0'
      - `configurations` object[]
        - `id` integer
        - `dataset` string
        - `only` boolean
        - `lookback` integer
        - `key` string[]
        - `include` string[]
        - `exclude` string[]
        - `dateColumn` string
        - `timeColumn` string
        - `timeBin` 'MIN' | 'HOUR' | 'DAY' | 'WEEK' | 'MONTH' | 'QUARTER' | 'YEAR'
        - `timeBinQuery` string
        - `categorical` boolean
        - `by` 'MIN' | 'HOUR' | 'DAY' | 'WEEK' | 'MONTH' | 'QUARTER' | 'YEAR'
        - `limit` integer
        - `minHistory` integer
        - `historyLimit` integer
        - `score` number, double
        - `aggFunc` string
        - `aggQuery` string
        - `query` string
        - `q1` number, double
        - `q3` number, double
        - `categoricalColumnConcatenation` boolean
        - `limitCategorical` integer
        - `measurementUnit` string
        - `multiplierUpper` number, double
        - `multiplierLower` number, double
        - `record` boolean
        - `filter` string
        - `combine` boolean
        - `categoricalConfidenceType` string
        - `categoricalTopN` integer
        - `categoricalBottomN` integer
        - `categoricalMaxConfidence` number, double
        - `categoricalMaxFrequencyPercentile` number, double
        - `categoricalMinFrequency` integer
        - `categoricalMinVariance` number, double
        - `categoricalMaxCategoryN` integer
        - `categoricalParallel` boolean
        - `categoricalAlgorithm` string
        - `categoricalAlgorithmParameters` object
        - `categoricalAlgorithmParametersString` string
  - `shape` ShapeDefDTO
    - `enabled` boolean
    - `totalScore` integer
    - `sensitivity` number, double
    - `maxPerCol` integer
    - `maxColSize` integer
    - `granular` boolean
    - `columnSettings` ShapeColumnSettingsDTO[]
      - `enabled` boolean
      - `name` string
      - `type` string
  - `jobSchedule` AbstractJobScheduleConfigurationDTO
    - `enabled` boolean
    - `dataset` string, required
    - `agentId` Identifier, required
      - `id` integer
      - `uuid` string, uuid
    - `runDateFormat` string, required
    - `scheduleFrequency` 'DAILY' | 'MONTHLY' | 'QUARTERLY'
    - `updatedAt` string, date-time
    - `scheduleTime` string
  - `connectionType` 'PULLUP' | 'PUSHDOWN'

## Response `200`

Returns the created or updated dataset definition

- DatasetDefDTO
  - `dataset` string
  - `runId` string
  - `runIdEnd` string
  - `runState` 'DRAFT' | 'DRY' | 'CERTIFIED'
  - `passFail` integer
  - `passFailLimit` integer
  - `jobDescription` string
  - `jobId` IdentifierDTO
    - `id` integer
    - `uuid` string, uuid
  - `coreMaxActiveConnections` integer
  - `linkId` string[]
  - `licenseKey` string
  - `logFile` string
  - `logLevel` string
  - `hootOnly` boolean
  - `prettyPrint` boolean
  - `useTemplate` boolean
  - `parallel` boolean
  - `plan` boolean
  - `dataPreviewOff` boolean
  - `datasetSafeOff` boolean
  - `obslimit` integer
  - `pgUser` string
  - `pgPassword` string
  - `host` string
  - `port` string
  - `user` string
  - `alertEmail` string
  - `scheduleTime` string
  - `schemaScore` number, double
  - `optionAppend` string
  - `keyDelimiter` string
  - `agentId` IdentifierDTO
    - `id` integer
    - `uuid` string, uuid
  - `load` LoadDefDTO
    - `readonly` boolean
    - `passwordManager` string
    - `alias` string
    - `query` string
    - `queryHistory` string
    - `key` string
    - `expression` string
    - `addDateColumn` boolean
    - `zeroFillNull` boolean
    - `replaceNulls` string
    - `stringMode` boolean
    - `operator` string
    - `dateColumn` string
    - `transform` string
    - `filter` string
    - `filterNot` string
    - `sample` number, double
    - `backRun` integer
    - `backRunBin` 'MINUTE' | 'HOUR' | 'DAY' | 'WEEK' | 'MONTH' | 'QUARTER' | 'YEAR'
    - `unionLookBack` boolean
    - `unionLookBackMinRow` integer
    - `cache` boolean
    - `dateFormat` string
    - `timeFormat` string
    - `timestamp` boolean
    - `filePath` string
    - `fileQuery` string
    - `fullFile` boolean
    - `fileHeader` string
    - `inferSchema` boolean
    - `fileType` string
    - `delimiter` string
    - `fileCharSet` string
    - `skipLines` integer
    - `avroSchema` string
    - `xmlRowTag` string
    - `flatten` boolean
    - `handleMaps` boolean
    - `handleMixedJson` boolean
    - `multiLine` boolean
    - `lib` string
    - `additionalLib` string
    - `driverName` string
    - `connectionName` string
    - `connectionUrl` string
    - `userName` string
    - `password` string
    - `connectionProperties` object
    - `hiveNative` boolean
    - `hiveNativeHWC` boolean
    - `useSql` boolean
    - `columnName` string
    - `lowerBound` string
    - `upperBound` string
    - `numPartitions` integer
    - `partitionNumber` integer
    - `escapeWithBackTick` boolean
    - `escapeWithSingleQuote` boolean
    - `escapeWithDoubleQuote` boolean
    - `escapeCharacter` string
    - `checkHeader` boolean
    - `archiveConnection` string
    - `coreFetchMode` boolean
    - `archiveBreaks` boolean
    - `ruleSerial` boolean
  - `pushdown` PushDownDefDTO
    - `dataset` string
    - `connectionName` string
    - `maxConnections` integer
    - `sourceQuery` string
    - `backRuns` integer
    - `backRunBin` string
    - `dateFormatType` string
    - `threads` integer
    - `manualSourceQuery` boolean
    - `key` string
    - `sourceOutputSchema` string
    - `sourceBreakRules` boolean
    - `sourceBreakOutliers` boolean
    - `sourceBreakDupes` boolean
    - `sourceBreakShapes` boolean
    - `sourceBreakResults` boolean
    - `sqlLoggingToggle` boolean
    - `unionLookback` boolean
    - `unionLookbackMin` integer
  - `outliers` OutlierDefDTO[]
    - `id` integer
    - `dataset` string
    - `only` boolean
    - `lookback` integer
    - `key` string[]
    - `include` string[]
    - `exclude` string[]
    - `dateColumn` string
    - `timeColumn` string
    - `timeBin` 'MIN' | 'HOUR' | 'DAY' | 'WEEK' | 'MONTH' | 'QUARTER' | 'YEAR'
    - `timeBinQuery` string
    - `categorical` boolean
    - `by` 'MIN' | 'HOUR' | 'DAY' | 'WEEK' | 'MONTH' | 'QUARTER' | 'YEAR'
    - `limit` integer
    - `minHistory` integer
    - `historyLimit` integer
    - `score` number, double
    - `aggFunc` string
    - `aggQuery` string
    - `query` string
    - `q1` number, double
    - `q3` number, double
    - `categoricalColumnConcatenation` boolean
    - `limitCategorical` integer
    - `measurementUnit` string
    - `multiplierUpper` number, double
    - `multiplierLower` number, double
    - `record` boolean
    - `filter` string
    - `combine` boolean
    - `categoricalConfidenceType` string
    - `categoricalTopN` integer
    - `categoricalBottomN` integer
    - `categoricalMaxConfidence` number, double
    - `categoricalMaxFrequencyPercentile` number, double
    - `categoricalMinFrequency` integer
    - `categoricalMinVariance` number, double
    - `categoricalMaxCategoryN` integer
    - `categoricalParallel` boolean
    - `categoricalAlgorithm` string
    - `categoricalAlgorithmParameters` object
    - `categoricalAlgorithmParametersString` string
  - `patterns` PatternDefDTO[]
    - `id` integer
    - `dataset` string
    - `only` boolean
    - `lookback` integer
    - `key` string[]
    - `dateColumn` string
    - `include` string[]
    - `exclude` string[]
    - `score` number, double
    - `minSupport` number, double
    - `confidence` number, double
    - `limit` integer
    - `query` string
    - `filter` string
    - `timeBin` 'MINUTE' | 'HOUR' | 'DAY' | 'WEEK' | 'MONTH' | 'QUARTER' | 'YEAR'
  - `dupe` DupeDefDTO
    - `on` boolean
    - `only` boolean
    - `include` string[]
    - `exclude` string[]
    - `depth` integer
    - `lowerBound` integer
    - `upperBound` integer
    - `approximate` integer
    - `limitPerDupe` integer
    - `checkHeader` boolean
    - `filter` string
    - `ignoreCase` boolean
    - `score` number, double
    - `limit` integer
  - `profile` ProfileDefDTO
    - `on` boolean
    - `only` boolean
    - `include` string[]
    - `exclude` string[]
    - `correlation` boolean
    - `histogram` boolean
    - `semantic` boolean
    - `dataConceptId` integer
    - `limit` integer
    - `histogramLimit` integer
    - `score` number, double
    - `shape` boolean
    - `shapeTotalScore` integer
    - `shapeSensitivity` number, double
    - `shapeMaxPerCol` integer
    - `shapeMaxColSize` integer
    - `shapeGranular` boolean
    - `behavioralDimension` string
    - `behavioralDimensionGroup` string
    - `behavioralValueColumn` string
    - `behaviorScoreOff` boolean
    - `behaviorLookback` integer
    - `behaviorMinSupport` integer
    - `profilePushDown` string[]
    - `behaviorRowCheck` boolean
    - `behaviorTimeCheck` boolean
    - `behaviorMinValueCheck` boolean
    - `behaviorMaxValueCheck` boolean
    - `behaviorMeanValueCheck` boolean
    - `behaviorNullCheck` boolean
    - `behaviorEmptyCheck` boolean
    - `behaviorUniqueCheck` boolean
    - `profileStringLength` boolean
    - `detectStringNumerics` boolean
    - `detectTopnBotn` boolean
    - `detectScalePrecision` boolean
    - `adaptiveTier` string
    - `advancedTier` boolean
    - `filter` string
  - `source` SourceDefDTO
    - `on` boolean
    - `only` boolean
    - `validateValues` boolean
    - `key` string[]
    - `include` string[]
    - `exclude` string[]
    - `includeSrc` string[]
    - `excludeSrc` string[]
    - `map` object
    - `score` number, double
    - `limit` integer
    - `sourcePushDownCount` boolean
    - `checkType` boolean
    - `checkCase` boolean
    - `validateSchemaOrder` boolean
    - `matches` boolean
    - `validateValuesThreshold` number, double
    - `validateValuesThresholdStrictDownScore` boolean
    - `validateValuesShowAll` boolean
    - `validateValuesIgnoreNull` boolean
    - `validateValuesIgnoreEmpty` boolean
    - `validateValuesIgnorePrecision` boolean
    - `validateValuesTrim` boolean
    - `validateValuesShowMissingKeys` boolean
    - `validateSrcJoinOnly` boolean
    - `validateValuesFilter` string
    - `dataset` string
    - `driverName` string
    - `user` string
    - `password` string
    - `passwordManager` string
    - `connectionName` string
    - `connectionUrl` string
    - `query` string
    - `lib` string
    - `connectionProperties` object
    - `filePath` string
    - `fileQuery` string
    - `fullFile` boolean
    - `header` string
    - `skipLines` integer
    - `inferSchema` boolean
    - `fileType` string
    - `delimiter` string
    - `fileCharSet` string
    - `avroSchema` string
    - `xmlRowTag` string
    - `flatten` boolean
    - `handleMaps` boolean
    - `handleMixedJson` boolean
    - `multiLine` boolean
    - `filterCols` string[]
  - `rule` RuleDefDTO
    - `on` boolean
    - `only` boolean
    - `lib` string
    - `name` string
    - `absoluteScoring` boolean
    - `ruleBreakPreviewLimit` integer
  - `colMatch` ColMatchDefDTO
    - `colMatchParallelProcesses` integer
    - `colMatchDurationMins` integer
    - `colMatchBatchSize` integer
    - `level` string
    - `fuzzyDistance` integer
    - `connectionList` Connection[]
      - `dataset` string
      - `url` string
      - `user` string
      - `password` string
      - `query` string
      - `filePath` string
      - `driver` string
      - `driverProps` object
  - `spark` SparkDefDTO
    - `numExecutors` integer
    - `driverMemory` string
    - `executorMemory` string
    - `executorCores` integer
    - `conf` string
    - `queue` string
    - `master` string
    - `principal` string
    - `keyTab` string
    - `deployMode` string
    - `jars` string[]
    - `packages` string[]
    - `files` string[]
  - `env` EnvDefDTO
    - `dataset` string
    - `jdbcPrincipal` string
    - `jdbcKeyTab` string
  - `record` RecordDefDTO
    - `on` boolean
    - `in` string
    - `notIn` string
    - `include` string[]
    - `percDeltaLimit` number, double
    - `score` number, double
    - `dataset` string
    - `dateColumn` string
    - `keyColumn` string
    - `timeBin` 'MIN' | 'HOUR' | 'DAY' | 'WEEK' | 'MONTH' | 'QUARTER' | 'YEAR'
    - `by` 'MIN' | 'HOUR' | 'DAY' | 'WEEK' | 'MONTH' | 'QUARTER' | 'YEAR'
  - `stream` StreamDefDTO
    - `type` 'Kafka'
    - `kafkaDef` KafkaDefDTO
      - `kafkaGroup` string
      - `kafkaBroker` string
      - `kafkaPort` string
      - `kafkaTopic` string
      - `kafkaKeySerde` string
      - `kafkaValSerde` string
      - `kafkaSaslEnabled` boolean
      - `kafkaSaslService` string
      - `kafkaSslEnabled` boolean
      - `streamSchemaRegistryUrl` string
      - `streamSchemaRegistryUser` string
      - `streamSchemaRegistryPassword` string
      - `streamInterval` string
      - `streamSchema` string
      - `streamFormat` string
      - `streamMaxLull` string
  - `pipeline` string[]
  - `runDate` string, date-time
  - `runDateEnd` string, date-time
  - `metaTags` string[]
  - `outlierConfiguration` union — Outlier configurations
    - object
      - `columnSensitivitySettings` OutlierDefColumnSensitivitySettingsDTO[] — List of column sensitivity settings
        - `name` string
        - `weight` number, double
        - `unit` 'DEFAULT' | 'UNIT_0_001' | 'UNIT_0_01' | 'UNIT_0_1' | 'UNIT_1_0' | 'UNIT_10_0' | 'UNIT_100_0' | 'UNIT_1000_0' | 'UNIT_10000_0' | 'UNIT_100000_0' | 'UNIT_1000000_0' | 'UNIT_10000000_0'
      - `configurations` object[]
        - `id` integer
        - `key` string[]
        - `include` string[]
        - `exclude` string[]
        - `timeColumn` string
        - `timeBinQuery` string
        - `limit` integer
        - `minHistory` integer
        - `historyLimit` integer
        - `score` number, double
        - `aggFunc` string
        - `aggQuery` string
        - `query` string
        - `categoricalColumnConcatenation` boolean
        - `limitCategorical` integer
        - `measurementUnit` string
        - `multiplierUpper` number, double
        - `multiplierLower` number, double
        - `record` boolean
        - `filter` string
        - `combine` boolean
        - `categoricalConfidenceType` string
        - `categoricalTopN` integer
        - `categoricalBottomN` integer
        - `categoricalMaxConfidence` number, double
        - `categoricalMaxFrequencyPercentile` number, double
        - `categoricalMinFrequency` integer
        - `categoricalMinVariance` number, double
        - `categoricalMaxCategoryN` integer
        - `categoricalParallel` boolean
        - `categoricalAlgorithm` string
        - `categoricalAlgorithmParameters` object
        - `categoricalAlgorithmParametersString` string
      - `only` boolean
      - `lookback` integer
      - `dateColumn` string
      - `timeBin` 'MIN' | 'HOUR' | 'DAY' | 'WEEK' | 'MONTH' | 'QUARTER' | 'YEAR'
      - `by` 'MIN' | 'HOUR' | 'DAY' | 'WEEK' | 'MONTH' | 'QUARTER' | 'YEAR'
      - `q1` number, double
      - `q3` number, double
      - `categorical` boolean
    - object
      - `columnSensitivitySettings` OutlierDefColumnSensitivitySettingsDTO[] — List of column sensitivity settings
        - `name` string
        - `weight` number, double
        - `unit` 'DEFAULT' | 'UNIT_0_001' | 'UNIT_0_01' | 'UNIT_0_1' | 'UNIT_1_0' | 'UNIT_10_0' | 'UNIT_100_0' | 'UNIT_1000_0' | 'UNIT_10000_0' | 'UNIT_100000_0' | 'UNIT_1000000_0' | 'UNIT_10000000_0'
      - `configurations` object[]
        - `id` integer
        - `dataset` string
        - `only` boolean
        - `lookback` integer
        - `key` string[]
        - `include` string[]
        - `exclude` string[]
        - `dateColumn` string
        - `timeColumn` string
        - `timeBin` 'MIN' | 'HOUR' | 'DAY' | 'WEEK' | 'MONTH' | 'QUARTER' | 'YEAR'
        - `timeBinQuery` string
        - `categorical` boolean
        - `by` 'MIN' | 'HOUR' | 'DAY' | 'WEEK' | 'MONTH' | 'QUARTER' | 'YEAR'
        - `limit` integer
        - `minHistory` integer
        - `historyLimit` integer
        - `score` number, double
        - `aggFunc` string
        - `aggQuery` string
        - `query` string
        - `q1` number, double
        - `q3` number, double
        - `categoricalColumnConcatenation` boolean
        - `limitCategorical` integer
        - `measurementUnit` string
        - `multiplierUpper` number, double
        - `multiplierLower` number, double
        - `record` boolean
        - `filter` string
        - `combine` boolean
        - `categoricalConfidenceType` string
        - `categoricalTopN` integer
        - `categoricalBottomN` integer
        - `categoricalMaxConfidence` number, double
        - `categoricalMaxFrequencyPercentile` number, double
        - `categoricalMinFrequency` integer
        - `categoricalMinVariance` number, double
        - `categoricalMaxCategoryN` integer
        - `categoricalParallel` boolean
        - `categoricalAlgorithm` string
        - `categoricalAlgorithmParameters` object
        - `categoricalAlgorithmParametersString` string
  - `shape` ShapeDefDTO
    - `enabled` boolean
    - `totalScore` integer
    - `sensitivity` number, double
    - `maxPerCol` integer
    - `maxColSize` integer
    - `granular` boolean
    - `columnSettings` ShapeColumnSettingsDTO[]
      - `enabled` boolean
      - `name` string
      - `type` string
  - `jobSchedule` AbstractJobScheduleConfigurationDTO
    - `enabled` boolean
    - `dataset` string, required
    - `agentId` Identifier, required
      - `id` integer
      - `uuid` string, uuid
    - `runDateFormat` string, required
    - `scheduleFrequency` 'DAILY' | 'MONTHLY' | 'QUARTERLY'
    - `updatedAt` string, date-time
    - `scheduleTime` string
  - `connectionType` 'PULLUP' | 'PUSHDOWN'

---

[API](https://skmtc.net/collibra/apis/collibra-data-governance-center-core-api.md) · [All operations](https://skmtc.net/collibra/apis/collibra-data-governance-center-core-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/collibra/collibra-data-governance-center-core-api/revisions/6d091bc86a33/schema)