Test Role Assistants

Run lifecycle checks and optional YAML cases for a Role Assistant after you promote the package — with a portal deep link for human review. This page is the operator how-to for proving a Role Assistant still works in an environment; it does not replace Connected System unit tests or integration package scoring.

Why this matters: A Role Assistant is not a ready-made workflow or business-process engine. It runs on a governed AI enterprise Runtime — so behavior stays determinate even when there is no coded process graph. Ordinary agents are hard to test; AI Fabrix lets you prove the assistant still works correctly in your process.

Prerequisites

Logged in: aifabrix auth status (command fails if not authenticated — it never skips)
Role Assistant package available locally or already running on the dataplane
Know the target environment: dev, tst, or pro
When settings or Evidence changed, promote first

Promote path: Manage Role Assistant packages.
Evidence readiness: Author Role Assistant Evidence.

Concept background: How Role Assistants are created.

Where it lives

Item	Location
Package	`role-assistant/` workspace for the Role Assistant (bare role key in CLI)
Cases	Numbered suite folders of YAML case files discovered by the CLI
Portal	Deep link printed by the CLI for step-by-step status

Runnable suites are for CLI evaluation. Separate investigation captures for process-execution honesty are not the primary role-assistant test path — keep those folders out of day-to-day suite runs.

Integration package folder parity (package-score) measures Connected System package directories against a reference. It does not evaluate Role Assistant YAML cases.

How to set

Auth check — never proceed when status is not authenticated:

aifabrix auth status

Lifecycle smoke (portal deep link for operators):

aifabrix role-assistant test <roleKey> --env dev

List cases, then run one case or a suite:

aifabrix role-assistant test <roleKey> --list-cases
aifabrix role-assistant test <roleKey> --case smoke
aifabrix role-assistant test <roleKey> --suite 001 --fail-fast --json

Publish policy: default uses the public/running Role Assistant. Refresh the package with aifabrix upload <roleKey> --env … or pass --publish only when you intentionally republish.
Case shape: each executable case needs a message or content input; optional answers and approvals match wait payloads (questionKey, requestKey, waitReason, or capability). Prefer soft expectations for exploratory benches.

Example case skeleton:

caseId: smoke-ok
input:
  message: "Create a customer named Acme"
answers: []
approvals: []
expected:
  soft:
    - description: "Task reaches a governed wait or completion without crash"

Optional OpenAPI attachments for start:

caseId: admit-openapi
input:
  message: "Build a Connected System from the attached contracts"
  openapiDir: openapi

auth status
  → role-assistant test (lifecycle)
  → --list-cases / --suite / --case
  → portal review + JSON for automation

Defaults and examples

Topic	Default
`--env`	`dev`
`--publish`	Off (public/running worker)
Hard assertion failure	Exit `2`
Auth failure / missing prereq	Exit `1` (never skip)
Suite stop	Continue unless `--fail-fast`

aifabrix role-assistant test sales-representative --env dev
aifabrix role-assistant test sales-representative --suite 001 --json

Alias for digital-worker packages: aifabrix test ra-<roleKey> follows the same lifecycle path as role-assistant test with the bare role key.

Machine-readable automation should prefer --json and treat portal URLs as human review, not as the only pass signal.

Validate

Check	Signal
Auth	`aifabrix auth status` shows Authenticated
Lifecycle	CLI verdict + portal URL
Suite	`--json` summary; exit `0` pass / `2` assertion failure
Evidence promote	Upload Evidence counts before relying on Evidence-backed asks
Knowledge sync	Fewer redundant asks after Knowledge packs are available — see Author Role Assistant Knowledge

After a failed suite, open the portal link, fix package Evidence or Knowledge if the ask was wrong, re-upload when needed, then re-run the same --suite or --case.

Common mistakes

Mistake	Fix
Expecting skip when not logged in	Log in; command must fail closed
Using package-score for Role Assistant cases	package-score is Connected System folder parity only
Treating investigation captures as suite YAML	Keep runnable suites separate from PE capture folders
Publishing on every test	Leave `--publish` off unless refreshing the package
Hard assertions for exploratory honesty	Prefer soft expectations until the path is stable

Limits

Exact flags follow your Builder version (aifabrix role-assistant test --help).
Process-execution honesty benches are contributor tooling — not a second product CLI on this page.
Full Evidence and Knowledge authoring are separate how-tos linked below.
Case roots may live beside the package or in a repo cases directory depending on your workspace layout — use --list-cases and --cases-root when discovery is empty.

Documentation Index