Connect Fabric Experiments

Fabric Experiments is Airlift's executable validation provider. Airlift decides what must be proven for a migration object and whether admitted evidence satisfies policy. Experiments runs the checks and owns the raw run, results, and evidence manifest.

This division prevents two unsafe shortcuts: converter success cannot certify itself, and Airlift does not grow a second test engine.

Composition contract

Airlift stores foreign references and digests, not a copy of Experiments run state. It may block on an Experiments result; it never re-owns the result.

Worker configuration

Create an ExperimentsValidationAdapter and an evidence publisher, then inject both into the worker:

const worker = await createAirliftWorker({
  mode: 'temporal',
  runtime,
  effector,
  validationAdapter: new ExperimentsValidationAdapter(experimentsRunner),
  validationEvidencePublisher: evidenceStore,
});

The runner receives stable idempotency keys for request creation, every object suite, and manifest finalization. Provider retries must return the same logical run. The publisher must durably persist evidence before returning; otherwise Airlift refuses to admit the validation record.

Use Experiments BDD scenarios for business behavior and source-specific semantics, A/B tests for modernization releases against a certified baseline, and live Databricks testkit runs for workspace behavior. Local or synthetic fixtures can exercise adapters and policy, but cannot close a client-specific readiness track.

Failure behavior

• A failed required check becomes a discrepancy and a failed readiness observation.
• Provider outage or cancellation leaves the execution uncertain until reconciled.
• Duplicate workflow starts collapse to one execution.
• Evidence with an unregistered producer, mutable reference, or wrong digest fails closed.
• Changing an artifact digest, source watermark, profile, or policy can stale an existing readiness decision and invalidate certificate eligibility.

See Build and run validation suites for commands and the discrepancy lifecycle.

For generated pipeline candidates, release qualification also requires every requested object to have a passing Experiments run bound to the exact artifact-set digest and the same Databricks workspace used by Runway.

Modernization comparisons

A modernization comparison has two named treatments: the certified baseline and modernization treatment. The active migration certificate is the baseline and the separately deployed native artifact is the candidate. The Experiments evaluation reference must include functional guardrails plus at least one declared outcome metric. Airlift refuses readiness when any guardrail or metric fails.

Use fa modernization evidence to admit the Experiments manifest and evaluation digest; do not paste raw test rows into an Airlift action. Use paired_execution benchmark observations when Experiments also measures engineering effort or runtime under comparable conditions.