Configure data flow (CIP execution)

Prev Next

Set execution on each Business Entity so enabled capabilities compile to vendor calls with identity, RBAC/ABAC, and audit applied at runtime.

CIP is the declarative pipeline that obtains and normalizes data for a capability key, then emits records or persists binary documents and returns references.
CIP is not the Role Assistant / Decision Engine loop — Runtime continues work after CIP returns its governed result.

Prerequisites

Where it lives

Layer Location
Engine + CIP <datasourceKey>.json → execution
Operations execution.cip.operations.<operationKey>
Step bindings Same file — fetch.openapiRef ↔ openapi.operations keys
UI Business Entity Data Flow tab (?tab=cip)

Manifest key: execution.

Builder MCP

Manifest key execution
UI tab Data Flow (?tab=cip)
Help topics cipOverview, section (sectionKey: execution), dependencyGraph, schemaCatalog
Repair hooks After OpenAPI or capability changes: repair with expose/rbac as needed, then validate

Safe agent loop: help cipOverview → section → patch → repair → validate → test. Full registry: Build reference.

How to set

  1. Validate upstream bands before editing CIP:
aifabrix repair <systemKey> --expose --dry-run
aifabrix validate <systemKey>
  1. Choose execution.engine
Engine Use when
cip Default for external Business Entities — declarative steps (this page)
python Custom handler entrypoint when CIP steps are insufficient (advanced)
datasource Chained CIP that calls other Business Entity operations (advanced)

Most integrations use engine: cip.

  1. Prefer generated CIP from OpenAPI import. Extend only for pagination, mapping, filters, or multi-source fetches.

  2. Author the CIP envelope — required: version (1.0) and operations. Optional logical types are fixed: inputType: void, outputType: records.

{
  "execution": {
    "engine": "cip",
    "cip": {
      "version": "1.0",
      "inputType": "void",
      "outputType": "records",
      "operations": {
        "list": {
          "enabled": true,
          "description": "List records scoped by ABAC",
          "steps": [
            {
              "fetch": {
                "source": "openapi",
                "openapiRef": "list"
              }
            },
            {
              "map": {
                "useFieldMappings": true,
                "inputPath": "$.items[*]"
              }
            },
            {
              "output": {
                "mode": "records"
              }
            }
          ]
        },
        "get": {
          "enabled": true,
          "description": "Retrieve one record by primary key",
          "steps": [
            {
              "fetch": {
                "source": "openapi",
                "openapiRef": "get"
              }
            },
            {
              "map": {
                "useFieldMappings": true,
                "inputPath": "$"
              }
            },
            {
              "output": {
                "mode": "records"
              }
            }
          ]
        }
      }
    }
  }
}
  1. Binding law (fail closed if broken):
Band Must align
capabilities[]. key Same string as execution.cip.operations key
fetch.openapiRef (when source: openapi) Key under openapi.operations
Operation naming Pattern ^[a-z][a-zA-Z0-9]*$ (standard: list, get, create, update, upsert, delete)
  1. Multi-step list pattern (common): fetch → paginate → map → filter → output. Step property details: CIP step catalog.

  2. Prove execution with testPayload and the CLI ladder on CIP execution.

Defaults and examples

Element Default / rule Notes
engine cip for external entities python / datasource are advanced
cip.version 1.0 Only allowed enum today
Operation enabled true Disable instead of deleting during bring-up
map.useFieldMappings Prefer true Applies fieldMappings.attributes
output.mode records or documents Use documents for binary document persistence

Fetch sources (fetch.source):

Source Required fields
openapi openapiRef and/or operationId
http method + path
datasource target datasource + operation (default list)
viewpoint SQL string query; entity storageType: view
record none beyond source (local table I/O; persist only for a declared write shape)

Required-field detail: CIP step catalog — fetch.

Minimal create skeleton (OpenAPI-bound):

{
  "create": {
    "enabled": true,
    "description": "Create one record",
    "shape": "create",
    "steps": [
      {
        "fetch": {
          "source": "openapi",
          "openapiRef": "create"
        }
      },
      {
        "map": {
          "useFieldMappings": true,
          "inputPath": "$"
        }
      },
      {
        "output": {
          "mode": "records"
        }
      }
    ]
  }
}

Use operation shape: create | update | upsert | delete when inference from fetch is not enough for published contract risk.

Retrieve a file by business key

A document datasource can expose get without a list operation. Declare
entityType: documentStorage, enable documentStorage, and pair an OpenAPI or
HTTP fetch with responseType: binary and an output step with mode: documents.
Map a stable document key from the operation parameters or response headers.
The pipeline stores the file through document processing and returns references,
not bytes. The CIP envelope's outputType: records remains unchanged; it is
not the per-step storage mode.

This path does not require a synthetic list or scheduled sync. Validate the
keyed operation, persisted metadata and entitled/unassigned access separately.
See Build a document storage entity.

Compose a declarative upsert

Declare shape: "upsert" when the caller supplies one record and CIP must
choose create or update from that record's business identity. The operation
must contain exactly one datasource fetch for the standard create operation
and exactly one for the standard update operation on the same datasource.
Upsert is never inferred from the operation key or fetch layout.

Identity comes from validation.repeatingValues entries whose strategy is
reject. Every such business identity field must be present in the request.
CIP resolves existence once, selects one half, and checks the caller's access
for that selected half. Missing or ambiguous identity, an unresolved record,
or denied access stops the operation without trying the other half.

{
  "upsert": {
    "enabled": true,
    "description": "Create or update one record by business identity",
    "shape": "upsert",
    "steps": [
      {
        "fetch": {
          "source": "datasource",
          "datasource": "example-customers",
          "operation": "create"
        }
      },
      {
        "fetch": {
          "source": "datasource",
          "datasource": "example-customers",
          "operation": "update"
        }
      },
      {
        "output": {
          "mode": "records"
        }
      }
    ]
  }
}

The fragment assumes the datasource also declares its standard create and
update operations plus the matching reject-on-duplicate validation fields.
See the CIP step catalog
for the composition rules.

Advanced engines

engine: python — provide execution.python.entrypoint (module callable). Keep CIP out of scope for that entity unless platform docs for hybrid layouts say otherwise.

engine: datasource — use when this entity orchestrates other Business Entity CIP operations (fetch.source: datasource). Validate child keys and RBAC carefully.

Author CIP via Builder API (agents)

  1. Help topic cipOverview
  2. Help topic section with sectionKey: execution
  3. Patch the section fragment (do not invent step keys outside the schema oneOf)
  4. aifabrix repair <systemKey> as dependency graph requires
  5. aifabrix validate <systemKey> then test / test-integration / datasource test-e2e

Validate

aifabrix validate <systemKey>
aifabrix test-integration <systemKey>
aifabrix datasource test-e2e <datasourceKey> --app <systemKey>

Compile errors that name operation keys usually mean OpenAPI ref mismatch or a missing capability — fix bands before live E2E. Certification details: CIP execution.

Common mistakes

Mistake Fix
Legacy pipeline.steps shape Use execution.cip.operations
Invented step keys (transform, …) Only fetch, paginate, map, merge, filter, output, pythonInline — see step catalog
openapiRef ≠ openapi.operations key Rename one side; CIP does not auto-rename
Enabled CIP without capability Add matching capabilities[] key + description
CIP without map/output on list/get Prefer map + output so sync and agents receive normalized records

Limits

This page teaches engine choice, binding law, and schema-valid skeletons. Full per-step property indexes live in the CIP step catalog. Python and datasource engines are advanced — prefer cip unless validate/repair guidance says otherwise. Cross-entity joins belong in separate entities linked by foreignKeys[], not one giant CIP. Runtime work continuation (Ask/Approval, Decision Engine) is outside CIP — see Role Assistants docs.

Declarative upsert is validated for the datasource execution path. Production
proof for every platform connector, worker, REST, MCP, and Role Assistant path
remains separately gated; validate the transport used by your integration.