<b>Allowed user roles:</b><br/><ul><li>sdzAdmin</li></ul><br/><br/>Create a data pipeline in the secure data zone.

For all the details, see the Dev Docs!

An data pipeline is a series of steps (at least 1) that GraphChain will process inbound data through. These steps are of 3 types: Field Mapping, JavaScript Transform, and Rules. The input to the first step is the full record of the inbound data. The output of that step is then the input into the next step and so forth.

The structure of an data pipeline is name, stepIds (optional), and description (optional). This is the basic structure:

<pre> { "name": "testPipeline", "description": "a test pipeline", "stepIds": [ ... ] } </pre>

The stepIds are an array of stepIds these stepIds refer to objects of these three types.

• FieldMappingStep

• JSTransformStep

• RuleSetStep

This typically is the order of simplest to most complex of steps. Each subsequent step in the list is capable of performing the functions of the previous steps. It is up to the admin (or developer) to decide the best tool for the job.

<b>FieldMappingStep</b>

This step will rename inbound field names into the field (attribute) names expected by the dictionary. Can rename nested attributes too, using a dot path notation. Not all fields must be in the map; if the inbound field name is already correct it can be omitted from the map. If a field is in the map but the value is NULL then it will be removed from the inbound data.

The step has a required map field and type field (FIELD_MAPPING). The name and description fields are optional, the exist to help the user with documentation.

<b>JSTransformStep</b>

This step will take the entire inbound data record as the only argument to a function. That function needs to return a JS dictionary (object) as the return value. The type (JS_TRANSFORM) and jsBase64 fields are required. The JavaScript can contain more than 1 function if that helps the developer, but there must be one function that is the entry point of the transform. If the developer names that function "xform" then the funcName field is not required (since it defaults to xform). In this case the JS has a function called "convert" that should be called. The name and description fields are optional for this step just like Field Mapping (omitted here for example).

<b>RuleSetStep</b>

The final step available is RuleSet, a Ruleset contains an array of ruleIds, each ruleId refers to a Drools Rule that is to be executed. This rule set can be used for numerous use-cases: field mapping, transforming data, applying data quality rules, rejecting or tagging data as issues. T