Test ladder and protection

Run the right test command at the right phase — unit validation, integration, E2E, operations certification, and governance verification prove different things.

Prerequisites

Datasource JSON passes aifabrix validate <systemKey>
Credentials resolved (aifabrix auth status)
Upload completed for integration/E2E against live dataplane

Where it runs

Layer	Typical command	Proves
Local schema	`aifabrix validate`, `aifabrix datasource validate`	JSON matches schemas
Unit (builder apps)	`aifabrix test <app>`	Container unit tests
Integration	`aifabrix test-integration <systemKey>`	Pipeline + credentials + sync against dataplane
E2E	`aifabrix test-e2e <systemKey>`	All datasources end-to-end
Per-datasource E2E	`aifabrix datasource test-e2e <datasourceKey> --app <systemKey>`	One entity path deep
Operations cert	`aifabrix verify-operations <systemKey>`	Validate → test → integration → E2E ladder
Agent metadata	`aifabrix verify-trust <systemKey>`	Business metadata trust for agents
Governance	`aifabrix verify-governance <systemKey>`	ABAC/protection visibility for subjects

How to set

After every JSON edit — local validate only:

aifabrix validate <systemKey>
aifabrix datasource validate <datasourceKey>

After upload — integration before E2E:

aifabrix upload <systemKey> --probe
aifabrix test-integration <systemKey>

When integration is green — full or scoped E2E:

aifabrix test-e2e <systemKey>
aifabrix datasource test-e2e <datasourceKey> --app <systemKey> --verbose

After dimensions + protection uploaded — governance verification (not a substitute for E2E):

aifabrix protection validate
aifabrix protection upload
aifabrix verify-governance <systemKey> --subject-email user@example.com

See Configure business policies and Protection manifests and upload. For a focused governance test walkthrough, see Test protection and governance.

Before production AI — certification pillars:

aifabrix verify-operations <systemKey>
aifabrix verify-trust <systemKey>
aifabrix verify-governance <systemKey> --subject-email user@example.com
aifabrix lifecycle <systemKey>

Defaults and examples

Recommended order for a new external system:

validate → upload → test-integration → test-e2e
  → (govern: dimensions, protection upload)
  → verify-governance → verify-operations → verify-trust → lifecycle

If this fails…	Check…
validate	Schema required fields, resourceType catalog, externalId
test-integration	Credentials, upload drift, sync config
test-e2e	Vendor sandbox data, primaryKey, exposed operations
verify-governance	Protection manifests, dimension bindings, subject membership

Example validate output you should see before upload (sanitized):

{
  "success": true,
  "systemKey": "example-crm",
  "checks": ["schema", "resourceType", "externalId"]
}

Governance verification uses real subjects — configure dimension membership before expecting scoped row counts.

Validate

Do not skip integration and jump to E2E — integration catches upload and credential issues faster.

Do not treat wizard success as governance pass — run verify-governance after protection is in place. Keep a written record of which datasource key and systemKey each test command targeted — mixed keys are a frequent source of false “green” E2E on the wrong entity.

Common mistakes

Mistake	Fix
E2E only once at end	Run integration after each upload
verify-governance before protection upload	Upload protection manifests first
Same subject for all scenarios	Use scoped governance packs
Confusing test-e2e with verify-operations	Operations cert wraps the full ladder
Wrong datasource `key` in E2E	Pass `--app <systemKey>` and correct datasourceKey

Limits

Governance scenarios require realistic subjects and dimension data in your tenant — empty ABAC dimensions produce passes that do not reflect production. Manual scenario packs may need trainer-supervised setup. The ladder assumes systemKey, datasource key, and root identity fields are already valid — rerun validate after any JSON edit even when you only changed protection YAML. displayName and entityType on each datasource must match the scenario you test — mixed record and document paths need separate E2E runs. Log files under integration/<systemKey>/logs/ supplement CLI output when --verbose is enabled on E2E.

Documentation Index