# Architecture # Architecture [#architecture] Airlift owns the migration lifecycle: estates, objects, waves, readiness, certificates, and cutover decisions. It composes execution, conversion, validation, persistence, and observability through published contracts rather than rebuilding those capabilities. ## Request and evidence flow [#request-and-evidence-flow] Specialist tools perform the work. Airlift admits their outputs by immutable reference, checks provenance and policy, and makes the resulting decision traceable. Converter success alone never proves production readiness. ## Repository map [#repository-map] | Path | Developer responsibility | | ----------------------------- | ------------------------------------------------------------------------------------------------------------------- | | `packages/airlift` | domain module, governed actions, policies, projections, source profiles, certificate contracts | | `packages/cli` | authenticated engagement onboarding plus local source planning, diagnostics, catalogs, and certificate verification | | `packages/adapter-lakebridge` | Databricks job submission and immutable Lakebridge result references | | `packages/store` | durable event and projection persistence | | `apps/airlift-worker` | assessment and conversion activities, schedules, cutover workflow, effector seam | | `apps/console` | Databricks Apps identity boundary and operator workbench | Source integrations extend narrow adapters. They do not add a parallel database or mutation API. Start with `createSourceMigrationPlan()` to discover the commands, actions, outputs, and evidence expected by a source profile. ## Governed domain module [#governed-domain-module] Every externally meaningful state change is an `airlift.*` Platform action. Handlers return pending domain events; the host appends them only after schema, authorization, policy, and state-machine checks pass. This produces one mutation and audit path for the console, workers, agents, and application integrations. The event ledger folds into organization-scoped views for inventory, the migration funnel, readiness, waves, conversion attempts, certificates, and audit. Illegal transitions—such as certifying before conversion or recording a cutover before it is authorized—are rejected structurally. Use the CLI to inspect the installed action surface: ```bash fa actions fa actions --json > .airlift/action-contract.json ``` ## Worker and durable cutover [#worker-and-durable-cutover] The application owns its deterministic Temporal domain workflow. Harness supplies the worker and connection plumbing. Workflow code performs no I/O; activities call adapters and invoke governed actions with stable idempotency keys. Cutover follows a strict sequence: 1. re-read frozen scope, fresh readiness, certificates, and approvals; 2. create a durable external checkpoint; 3. apply the non-idempotent endpoint change once; 4. verify external state independently; 5. compensate once when verification proves a failed effect was applied; 6. record success/rollback, or leave an uncertain outcome open for reconciliation. The authenticated `airlift.wave_approve` action is the only approval authority. Temporal signals can wake a readiness check but cannot carry actor identity, approval, denial, or waiver authority. ## Extension seams [#extension-seams] | Seam | Development implementation | Production implementation | | ---------- | ---------------------------------------- | ------------------------------------------------------------------------------- | | store | in-memory event store | durable Postgres-compatible store through `@fabricorg/airlift/store` | | converter | `StubLakebridgeAdapter` | version-pinned Lakebridge workspace jobs | | repair | disabled unless injected | bounded Harness agent producing one reviewable candidate | | transfer | typed contract driver | source-specific snapshot/incremental driver with checkpoints and reconciliation | | validation | fixtures or test runner | admitted provider with immutable evidence and snapshot identities | | cutover | `StubCutoverEffector` in local mock mode | certified checkpoint/apply-once/verify/compensate implementation | ## Ownership boundaries [#ownership-boundaries] | Airlift does not implement | Compose instead | | --------------------------------------- | ------------------------------------------- | | SQL transpilation | Databricks Lakebridge | | Temporal connection and worker plumbing | Fabric Harness Temporal package | | row comparison and test execution | Lakebridge Reconcile or Experiments testkit | | mutation pipeline and audit storage | Fabric Platform and Platform Host | | Databricks authentication and clients | Fabric Harness Databricks package | | general-purpose agent runtime | Fabric Harness | These boundaries keep source adapters replaceable and prevent a migration project from creating competing definitions of deployment, monitoring, or task state. # Deploy Airlift # Deploy Airlift [#deploy-airlift] An Airlift installation has four runtime responsibilities: | Component | Responsibility | Recommended placement | | -------------- | -------------------------------------------------------------- | ----------------------------------- | | console | authenticated migration workbench and governed command ingress | Databricks App | | durable store | Platform invocation, event, idempotency, and projection state | Lakebase or compatible PostgreSQL | | migration jobs | Lakebridge assessment and conversion adapters | Databricks Jobs | | worker | schedules, activities, and the durable cutover workflow | separately operated Temporal worker | The console is not the worker. Keep interactive request handling separate from durable orchestration so a console deployment cannot interrupt an active migration workflow. ## Validate the Asset Bundle [#validate-the-asset-bundle] The repository contains a declarative Databricks Asset Bundle: ```bash pnpm validate:databricks databricks bundle validate -t databricks bundle deploy -t ``` Bundle variables supply environment-specific resource names and references. Do not commit workspace URLs, access tokens, client secrets, warehouse IDs, database passwords, or signing keys. ## Configure durable storage [#configure-durable-storage] Production installations use `AIRLIFT_STORE=postgres`. Bind either a Databricks Lakebase resource or a compatible PostgreSQL connection, then run schema creation as an explicit controlled-startup step: ```ts const store = await createAirliftStoreFromEnv(process.env); await store.ensureSchema(); ``` Schema creation must not happen as an import side effect. Run it once during controlled startup for both the console and worker composition roots. ## Configure identity and authorization [#configure-identity-and-authorization] The Databricks App authenticates the workspace user before resolving display labels from forwarded headers. The application derives actor and organization from authenticated server context; neither value may come from form data, action parameters, workflow signals, or CLI flags. Production startup requires: * a tenant-scoped authorization directory; * admitted worker and validation-provider principals; * Databricks App identity verification; * an immutable evidence registry; * an Ed25519 signing key stored through a deployment secret reference. Use [security configuration](/docs/reference/security) and [`fa doctor`](/docs/cli/doctor) to validate the configuration shape. ## Configure Lakebridge jobs [#configure-lakebridge-jobs] Provision assessment and conversion jobs in the target workspace. Airlift records job run IDs, tool versions, output references, and digests; reports and converted artifacts remain in workspace-controlled artifact storage. Required adapter settings include the job IDs, an artifact-volume root, the accepted Lakebridge version, bounded polling, and stable idempotency tokens. Version drift or malformed output fails the action instead of producing evidence. ## Configure the Temporal worker [#configure-the-temporal-worker] Create the worker with Temporal mode and an explicitly injected cutover effector: ```ts await createAirliftWorker({ mode: 'temporal', runtime, effector: cutoverEffector, }); ``` The effector identifies its certified profile, implementation version, and certification digest, then implements `createCheckpoint`, `applyCutover`, `verifyCutover`, and `compensateCutover`. Apply and compensation are each attempted once. Unknown outcomes remain unresolved for reconciliation; they are never converted into success by a retry. ## Pre-production verification [#pre-production-verification] Before admitting migration data, prove: 1. authenticated identity and cross-tenant denial; 2. durable write, projection read-back, and idempotent replay; 3. Lakebridge job connectivity, version enforcement, and artifact digest verification; 4. evidence-signing and offline certificate verification; 5. Temporal bundle determinism, query, cancellation, restart, and replay; 6. frozen-scope staleness, timed rehearsal, certified-effector binding, checkpoint, apply-once, verification, uncertainty, compensation, and rollback behavior; 7. parallel-run, cutover-verification, hypercare, incident, and source-disposition evidence paths. See [production readiness](/docs/status) for the reusable acceptance contract. # Fabric Airlift developer documentation # Move your estate to Databricks, with proof [#move-your-estate-to-databricks-with-proof] Fabric Airlift runs your migration as one governed job instead of a spreadsheet of hand-offs. A migration team connects a source estate, inventories it, scopes the work, builds the Databricks target, moves the data, proves parity, and goes live — and Airlift records every decision, artifact, and piece of evidence on one policy-enforced ledger so the speed never costs you the audit. ```text Connect → Inventory → Scope → Build → Move → Prove → Go-live ↘ Fixes (residue / discrepancies) ↗ ``` Speed comes from doing each stage once against a shared, typed inventory. Safety comes from the proof behind it: nothing reaches Go-live without independent validation evidence and a migration certificate that is **system-minted and Ed25519-signed**, derived from recorded evidence — never authored by the caller. What the certificate does and does not claim is spelled out in the [trust appendix](/docs/parity-certificates). ## Who uses which surface [#who-uses-which-surface] | You are | Your home | What you do there | | ---------------------------- | ------------------------------------- | --------------------------------------------------------------------------------------------------------------- | | a migrator | the Airlift Databricks App | run each stage, work the Fixes queue, watch progress and blockers | | a migration lead or approver | the Airlift Databricks App | freeze scope, approve waves, accept for the business, decide go-live | | an auditor or sponsor | the App trust view | read certificates, evidence, and the audit export — read-mostly | | automation or CI | the `fa` CLI and versioned remote API | record diagnostics and evidence under the admitted automation role — never approve, waive, certify, or cut over | Every surface invokes the same governed actions; the App, CLI, and API never take different mutation paths. ## Start in the App [#start-in-the-app] Follow the [App walkthrough](/docs/getting-started/ui-walkthroughs) — annotated, public synthetic screenshots of one migration from engagement setup through cutover, each paired with the exact next developer action. ## What Airlift implements under each stage [#what-airlift-implements-under-each-stage] | Stage | What your project calls or implements | What Airlift records/enforces | | ---------------------- | ----------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | | Connect / Inventory | source metadata/export adapter or Lakebridge Profiler/Analyzer | run identity, inventory/dependency digests, tool version | | Build | Lakebridge, entity mapper, pipeline implementation, or stream bootstrap | method, immutable artifacts, residue disposition | | Move | source-specific snapshot/incremental driver | watermarks, restart checkpoints, lag, counts, reconciliation | | Prove | Lakebridge Reconcile, Experiments testkit, or admitted external runner | provider run, immutable evidence, readiness observation | | Go-live | client-certified checkpoint/apply-once/verify/compensate effector | approvals, separation of duties, freeze digest, rehearsal, operational evidence, durable outcome, rollback, hypercare | | the certificate itself | no caller implementation | profile-derived, system-minted and Ed25519-signed envelope — see [parity certificates](/docs/parity-certificates) | ## Supported source profiles [#supported-source-profiles] Airlift ships profiles for warehouses and databases, SAP and SaaS applications, ETL platforms, mainframe data, federated query engines, and event streams. Every profile publishes evidence-derived support alongside separate non-evidentiary implementation routing. An empty tenant registry is always `cataloged`, regardless of shipped implementation paths. Run `fa source inspect ` to see the exact boundary, then open [source systems](/docs/sources) for developer routes. ## The governed programming model [#the-governed-programming-model] Every meaningful state change is an `airlift.*` Platform action. Console handlers, workers, and agents invoke the same authenticated runtime. Event handlers return pending domain events; projections build the inventory, funnel, readiness matrix, wave board, certificate ledger, and audit trail. Agents may propose or record conversion work but cannot approve, waive, certify, or cut over. Read [architecture](/docs/architecture) for package boundaries and [action catalog](/docs/reference/action-catalog) for the exact mutation contract. ## Automation and CLI [#automation-and-cli] Install the CLI when you need scripted source plans, diagnostics, or evidence admission: ```bash npm install --global @fabricorg/airlift-cli fa sources fa source inspect dynamics_365 fa source plan dynamics_365 \ --variant dynamics_365_finance_operations \ --json > dynamics-finance-plan.json ``` The generated plan covers inventory, code conversion, data movement, orchestration, security, validation, performance, consumers, cutover, and post-parity modernization. It names the applicable specialist adapter boundary, the Airlift actions your integration must invoke, project outputs, and exit criteria. Use the same registry in application code: ```ts import { AIRLIFT_ACTION_IDS, createSourceMigrationPlan, resolveSourceSystemProfile, } from '@fabricorg/airlift'; const source = resolveSourceSystemProfile('d365'); const plan = createSourceMigrationPlan(source.id, { variant: 'dynamics_365_finance_operations', }); console.log(source.workloadSurfaces); console.log(plan.steps.map((step) => step.airliftActions)); console.log(AIRLIFT_ACTION_IDS.assessmentRecord); ``` Continue with the [source developer workflow](/docs/sources/developer-workflow) for the complete assessment-to-cutover integration. ## Documentation map [#documentation-map] * [App walkthrough](/docs/getting-started/ui-walkthroughs) — the primary path through one migration. * [Quickstart](/docs/getting-started/quickstart) — build, test, run, and inspect a plan. * [CLI command reference](/docs/cli/command-reference) — every implemented command. * [Cutover commands](/docs/cli/cutover) — frozen scope through hypercare and source disposition. * [Source systems](/docs/sources) — executable source-specific routes and known gaps. * [Migration lifecycle](/docs/migration) — how adapters and actions compose by phase. * [Parity certificates](/docs/parity-certificates) — the certificate honesty model. * [Operations](/docs/operations) — identity, evidence, recovery, and scale. * [Integrations](/docs/integrations) — Platform, Harness, Experiments, Runway, Radar, Tower. * [Reference](/docs/reference) — actions, events, profiles, configuration, and errors. Use [production readiness](/docs/status) to verify the installation, source profile, evidence providers, and cutover effector before operating on production data. # How Airlift works # How Airlift works [#how-airlift-works] Airlift does not replace Lakebridge, a data-transfer engine, or a validation runner. It connects them through a governed migration model so developers can answer four questions for every object: 1. What is in scope and what depends on it? 2. Which artifact was produced, by which tool generation? 3. Which independent evidence shows that it is ready? 4. Which approved, reversible wave may move it into production? ## The lifecycle [#the-lifecycle] | Stage | Tool or integration | What Airlift records and enforces | | --------- | --------------------------------------------------------------- | ----------------------------------------------------------------------------------- | | assess | Lakebridge Profiler and Analyzer | source identity, inventory, dependencies, exclusions, complexity, immutable reports | | plan | target-design and delivery decisions | object ownership, dependency-aware waves, scope freeze, separation of duties | | convert | Lakebridge plus optional bounded repair | attempts, tool versions, artifact digests, warnings, and residue disposition | | transfer | project-specific snapshot and incremental driver | watermarks, manifests, lag, restart checkpoints, counts, and reconciliation | | validate | Lakebridge Reconcile, Experiments, or another admitted provider | provider run, evidence digest, snapshots, verdicts, and readiness observations | | certify | Airlift policy evaluation | a system-minted signed envelope derived from the active evidence profile | | cut over | Temporal workflow plus a target-specific effector | approvals, checkpoint, apply-once result, independent verification, and rollback | | modernize | Databricks-native release work | a separate backlog and validation profile that preserves baseline parity history | ## Assess the complete estate [#assess-the-complete-estate] The assessment adapter executes Lakebridge against exported source assets and records the run and immutable outputs. Inventory SQL together with orchestration, external data, security mappings, and downstream consumers; otherwise a “converted” warehouse can still fail when its pipelines or reports move. Use a source profile to generate the applicable commands and outputs: ```bash fa source inspect synapse fa source plan synapse ``` ## Plan dependency-aware waves [#plan-dependency-aware-waves] `airlift.wave_plan` and `airlift.wave_assign` group objects into rehearsable units. The planner, owner, object assignments, validation expectations, and rollback responsibility remain explicit. Once approved, a wave cannot silently gain new objects. ## Convert without hiding residue [#convert-without-hiding-residue] Run deterministic conversion first. If policy admits bounded repair, an agent may return one typed candidate; it cannot approve, certify, waive, or cut over anything. Route every unsupported construct into one visible lane: * deterministic conversion; * bounded repair candidate; * human engineering; * governed exclusion. Both automated and human conversion use `airlift.conversion_start` and `airlift.conversion_record`, which keeps provenance and funnel state consistent. ## Validate independently [#validate-independently] Conversion produces a candidate, not proof. Validation providers compare source and target behavior and write detailed results to immutable storage. Airlift admits the run reference, digest, snapshots, tool version, and verdict, then links that observation to one exact object-profile requirement. A table, stored procedure, pipeline, and report can require different evidence. A passed table comparison does not certify a failed orchestration path or downstream consumer. ## Mint a certificate from current evidence [#mint-a-certificate-from-current-evidence] `airlift.migration_certificate_mint` accepts an object identity and expected digests, not a caller-authored certificate. The handler derives the signed envelope from the current governed projection and fails on missing, stale, mismatched, or waived evidence that policy does not permit. The useful progress metric is therefore the percentage of in-scope objects deployed and certified against their assigned profile, with provenance and a rollback path—not the percentage of files that produced output. ## Cut over with durable verification [#cut-over-with-durable-verification] Before cutover, Airlift rechecks frozen wave scope, fresh certificates, readiness, and authenticated approvals. The worker then: 1. creates a durable checkpoint; 2. applies the non-idempotent external effect once; 3. verifies the resulting state independently; 4. records success only after verification; 5. leaves uncertainty unresolved for reconciliation rather than guessing. Rollback is a governed action with its own reason, evidence, and result. ## Preserve an attestable trail [#preserve-an-attestable-trail] `airlift.evidence_export` assembles the assessment, conversion, validation, certification, approval, cutover, and rollback history for an object or wave. The export is content-digested. Signed certificate envelopes can be verified independently with the Airlift CLI. This is Airlift's role: migration tools perform specialist work; Airlift makes their outputs traceable, policy-bound, independently provable, and safe to use in a production decision. # Parity and migration certificates A migration's credibility is decided by one question: *how do you know the converted object is safe for its intended release?* Airlift answers with two deliberately distinct artifacts. Keeping them separate prevents “the rows matched” from becoming “the object is ready to cut over.” ## Parity facet [#parity-facet] `airlift.parity_certify` records what a particular comparison actually proved. Required reproducibility fields include the validator generation, validation run, and source and target snapshots: ```ts interface ParityCertificate { evidenceDepth: 'row_count' | 'aggregates' | 'sampled_rows' | 'full_checksum'; rowCountSource: number; rowCountTarget: number; aggregateChecksums?: Record; sampledRowCount?: number; checksumMatch?: boolean; toolVersion: string; validationRunRef: string; sourceSnapshotAt: string; targetSnapshotAt: string; } ``` `airlift.parity_evidence.v1` enforces: 1. **Minimum depth.** An organization can require evidence stronger than row counts. 2. **No overclaiming.** `sampled_rows` requires a sample count; `full_checksum` requires its verdict; aggregate depths require their checksums. 3. **Freshness.** Source and target snapshots must remain inside policy's evidence window. This facet is inspectable evidence. It does not by itself advance the object to the migration-certified state. ## Readiness profiles [#readiness-profiles] Every object receives a versioned profile over nine tracks: * inventory; * target design; * code; * data movement; * deployment; * functional parity; * non-functional behavior; * business acceptance; * cutover readiness. Each requirement is explicitly `required` or `not_applicable`. Evidence can be pending, passed, failed, stale, or governed by a time-bounded waiver when policy permits it. Identity, tenant isolation, separation of duties, functional parity, and uncertain non-idempotent cutover outcomes are not waivable. Validation runs enter readiness only when: * the producer principal is admitted for the organization/provider; * the provider run completed successfully; * its immutable evidence reference and SHA-256 digest match the admitted registry; * the artifact digest, source watermark, target snapshot, tool version, and completion timestamp are present; * the functional-parity observation resolves to the active parity run. ## System-minted signed envelope [#system-minted-signed-envelope] `airlift.migration_certificate_mint` accepts no caller-authored certificate payload. A system principal supplies only the object ID and the expected profile/readiness digests; the handler derives the envelope from current governed projections and fails closed on drift. The envelope binds: * organization, estate, object, and optional wave; * profile ID/version/digest and readiness digest; * artifact digest, source watermark, and target snapshot; * parity certificate ID and evidence-manifest digest; * policy revision and tool generations; * issuer, issue time, signer key ID, signed digest, and Ed25519 signature. Private keys come from secret-backed deployment configuration. Offline verification uses the corresponding key-ID/public-key directory. A hash without a valid signature is never described as signed evidence. ## Stale and revoked state [#stale-and-revoked-state] Artifact, watermark, target snapshot, profile, readiness, dependency, policy, or release drift invalidates the certificate's claim. Airlift preserves the old envelope and marks it stale or revoked; it never deletes history. Sending an object to rework also removes its active certification from the wave gate. ## Why this is the moat [#why-this-is-the-moat] Lakebridge and other engines will keep improving conversion. Airlift's independent value is the versioned answer to what was validated, by whom, against which identities, under which profile and policy, and with which signature. Conversion produces a candidate; independent evidence and governed acceptance produce certification. # Production readiness # Production readiness [#production-readiness] Airlift separates installed capability from project readiness. A successful build or configured connector does not prove that a source estate is safe to migrate. ## Installation checks [#installation-checks] The installation must prove: * authenticated actor and organization derivation; * fail-closed authorization and cross-tenant denial; * durable event, projection, idempotency, and recovery behavior; * secret-backed evidence signing and public-key verification; * Lakebridge job and artifact-store connectivity; * Temporal workflow replay, cancellation, timeout, and restart behavior. Run the structural diagnostic first: ```bash fa doctor --profile production ``` `doctor` validates configuration shape. Connectivity and behavior require integration and deployment tests. ## Source-profile checks [#source-profile-checks] For the selected source, retain evidence for: * accepted inventory, dependencies, exclusions, and source version; * representative deterministic conversion and residue classification; * snapshot and incremental transfer, watermark, restart, lag, and reconciliation; * object-specific functional, data, security, and performance scenarios; * business-owner acceptance of the selected validation profile. Generate the source-specific checklist with: ```bash fa source plan ``` ## Automated cutover checks [#automated-cutover-checks] Do not enable automated cutover until the target-specific effector proves: 1. a durable pre-effect checkpoint; 2. one non-idempotent apply attempt; 3. independent observation of the resulting external state; 4. explicit handling of `verified`, `not_applied`, `failed_applied`, and `uncertain` outcomes; 5. one compensation attempt after a definitively failed applied effect; 6. a frozen scope, timed runbook, passing rehearsal, and unexpired matching effector certification; 7. passing parallel-run evidence and no open incidents. Projects that do not provide and certify this effector can use an approved manual procedure while Airlift records scope, approvals, evidence, outcome, and rollback state. Automated production cutover remains disabled for that project. ## Claim language [#claim-language] Use precise terms in project documentation: * **configured** means required settings are present; * **connected** means the external boundary completed an authenticated smoke test; * **validated** means the named scenarios produced immutable admitted evidence; * **certified** means Airlift minted a signed certificate from the active profile; * **cut over** means the external effect was independently verified and recorded. Do not use one term as a substitute for another. ## Modernization and value checks [#modernization-and-value-checks] Before promoting a modernization release, prove that its baseline migration certificate is still active, the separate Runway deployment reconciled to the intended artifact, the Experiments comparison passed every functional guardrail and declared outcome metric, and the promoter is separate from the recommender and decision-maker. Before publishing measured value to a client, retain at least three scoped observations and achieve medium or high computed confidence. A new observation makes an earlier summary stale. Engagement results are not universal product claims. # Assessment to frozen plan # Assessment to frozen plan [#assessment-to-frozen-plan] This tutorial exercises Airlift's authenticated F1 workflow. It starts with a registered estate and active engagement, admits a normalized assessment, records dependency edges, accepts the scope, generates a Databricks target scenario, and freezes the selected plan. The CLI is a remote client. Set the URL of your deployed Airlift Databricks App and a Databricks OAuth token first: ```bash export AIRLIFT_API_URL="https://your-airlift-app.example" export DATABRICKS_TOKEN="" npx --yes --package @fabricorg/airlift-cli@0.18.4 fa version ``` The API derives the actor and organization from authenticated Databricks identity. None of the following requests contains an actor or tenant field. ## 1. Register the estate [#1-register-the-estate] Create `estate.json`: ```json { "name": "Synthetic Synapse warehouse", "sourceSystem": "synapse", "owner": "migration-team@example.test", "environment": "prod", "priority": "priority", "connectionRef": "databricks-connection://migration/synapse-metadata" } ``` ```bash fa estate register --file estate.json --idempotency-key tutorial-estate-1 --json fa estate list --json ``` Copy the returned `estateId`. Add it to the `estateIds` array in an engagement create file, then create and activate the engagement. See [Engagements and connections](/docs/getting-started/engagements) for that request. ## 2. Run and record the assessment [#2-run-and-record-the-assessment] The Airlift worker composes the source adapter. `assessment_start` records the governed request; `assessment_record` admits the immutable result produced by that adapter. ```json title="assessment-start.json" { "estateId": "est_01ARZ3NDEKTSV4RRFFQ69G5FAV", "toolVersion": "lakebridge@0.14.x", "requestRef": "synthetic-assessment-001" } ``` ```bash fa assessment start \ --file assessment-start.json \ --idempotency-key tutorial-assessment-start-1 \ --json ``` The configured worker normally records the result. Adapter developers can test the same contract with a synthetic `assessment-record.json`: ```json { "assessmentId": "asm_01ARZ3NDEKTSV4RRFFQ69G5FAV", "estateId": "est_01ARZ3NDEKTSV4RRFFQ69G5FAV", "jobRunRef": "jobs/run/synthetic-001", "reportRef": "volume://airlift/tutorial/assessment.json", "reportDigest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "inventoryDigest": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", "dependencyDigest": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc", "toolVersion": "lakebridge@0.14.x", "objectCounts": { "tables": 1, "views": 1, "storedProcedures": 0, "etlJobs": 1, "other": 0 } } ``` ```bash fa assessment record \ --file assessment-record.json \ --idempotency-key tutorial-assessment-record-1 ``` On later runs, Airlift compares the newly admitted inventory and dependency digests with the preceding assessment for the same estate. **Discover → Assessment studio** displays per-category object-count deltas and makes digest drift an explicit review state. Missing digests are treated as drift, not as proof that nothing changed. Register normalized objects through `airlift.object_register`. Each object records its type, source identity, complexity, and assessment reference. Supported inventory types include tables, views, routines, ETL jobs, notebooks, reports, semantic models, ML assets, security objects, and external dependencies. ```bash fa inventory register --file object-table.json --idempotency-key tutorial-object-table fa inventory register --file object-job.json --idempotency-key tutorial-object-job fa inventory list --estate-id est_01ARZ3NDEKTSV4RRFFQ69G5FAV \ --assessment-id asm_01ARZ3NDEKTSV4RRFFQ69G5FAV --json ``` ## 3. Build and accept the dependency graph [#3-build-and-accept-the-dependency-graph] A dependency graph is created with an expected edge count. Adapters then submit batches of at most 500 secret-free edges. Airlift validates every object reference, rejects self-references and duplicates, and refuses acceptance until the declared count is complete. The governed action sequence is `dependency_graph_start`, `dependency_batch_record`, then `dependency_graph_accept`. The CLI commands below submit those exact actions through the authenticated API. ```json title="graph-start.json" { "engagementId": "eng_01ARZ3NDEKTSV4RRFFQ69G5FAV", "assessmentId": "asm_01ARZ3NDEKTSV4RRFFQ69G5FAV", "expectedEdgeCount": 1, "graphRef": { "system": "lakebridge", "type": "dependency_graph", "id": "synthetic-graph-001" }, "toolVersion": "lakebridge@0.14.x" } ``` ```json title="graph-batch.json" { "dependencyGraphId": "dpg_01ARZ3NDEKTSV4RRFFQ69G5FAV", "edges": [ { "fromObjectId": "obj_01ARZ3NDEKTSV4RRFFQ69G5FAW", "toObjectId": "obj_01ARZ3NDEKTSV4RRFFQ69G5FAV", "kind": "data", "confidence": 1, "critical": true } ] } ``` ```bash fa inventory graph start --file graph-start.json --idempotency-key tutorial-graph-start fa inventory graph record --file graph-batch.json --idempotency-key tutorial-graph-batch-1 fa inventory graph show dpg_01ARZ3NDEKTSV4RRFFQ69G5FAV --json ``` Graph acceptance is a human-authority operation: ```json title="graph-accept.json" { "dependencyGraphId": "dpg_01ARZ3NDEKTSV4RRFFQ69G5FAV", "expectedEdgeCount": 1 } ``` ```bash fa inventory graph accept \ --file graph-accept.json \ --idempotency-key tutorial-graph-accept ``` A cycle does not disappear. Airlift records it as an explicit planning blocker that must be dispositioned before plan freeze. ## 4. Accept the normalized scope [#4-accept-the-normalized-scope] Assessment acceptance recomputes the normalized inventory digest from projected objects and binds it to the accepted dependency digest, exclusions, assumptions, engagement, and authenticated reviewer. The `assessment_accept` action is a natural-person decision; the CLI submits it as `fa assessment accept`. ```json title="assessment-accept.json" { "assessmentId": "asm_01ARZ3NDEKTSV4RRFFQ69G5FAV", "engagementId": "eng_01ARZ3NDEKTSV4RRFFQ69G5FAV", "dependencyGraphId": "dpg_01ARZ3NDEKTSV4RRFFQ69G5FAV", "expectedObjectCount": 3, "exclusions": [], "assumptions": [ "A representative production validation window is available." ] } ``` ```bash fa assessment accept \ --file assessment-accept.json \ --idempotency-key tutorial-assessment-accept ``` Service accounts and agents may record assessment and graph proposals, but they cannot accept the graph or scope. ## 5. Generate and compare delivery scenarios [#5-generate-and-compare-delivery-scenarios] `plan_generate` maps every in-scope object type to a Databricks target pattern, performs a topological wave grouping, exposes cycles, and calculates effort and value from the provided assumptions. The calculations remain visible in the returned plan; they are not a hidden percentage claim. The governed sequence is `plan_generate`, `plan_select`, then `plan_freeze`. Selection and freeze require human authority; generation may be performed by admitted automation. ```json title="plan-generate.json" { "engagementId": "eng_01ARZ3NDEKTSV4RRFFQ69G5FAV", "assessmentId": "asm_01ARZ3NDEKTSV4RRFFQ69G5FAV", "name": "Parity-first delivery plan", "scenario": "parity_first", "targetWorkspaceRef": { "system": "databricks", "type": "workspace", "id": "synthetic-target" }, "maxObjectsPerWave": 100, "economics": { "manualHoursPerComplexityPoint": 8, "assistedHoursPerComplexityPoint": 3, "blendedHourlyCostUsd": 180, "sourceMonthlyRunCostUsd": 25000, "targetMonthlyRunCostUsd": 18000, "dualRunMonths": 2 }, "assumptions": [ "Unity Catalog prerequisites are complete before Wave 1." ] } ``` ```bash fa plan generate --file plan-generate.json --idempotency-key tutorial-plan-parity fa plan compare --engagement-id eng_01ARZ3NDEKTSV4RRFFQ69G5FAV --json fa plan select pln_01ARZ3NDEKTSV4RRFFQ69G5FAV --idempotency-key tutorial-plan-select ``` Freeze the engagement scope before freezing its selected plan: ```bash fa engagement freeze eng_01ARZ3NDEKTSV4RRFFQ69G5FAV \ --idempotency-key tutorial-engagement-freeze fa plan freeze pln_01ARZ3NDEKTSV4RRFFQ69G5FAV \ --idempotency-key tutorial-plan-freeze ``` The frozen digest binds engagement scope, accepted inventory, dependencies, target workspace reference, target mappings, wave candidates, economics, assumptions, and risks. ## 6. Export the assessment pack [#6-export-the-assessment-pack] ```json title="assessment-export.json" { "estateId": "est_01ARZ3NDEKTSV4RRFFQ69G5FAV", "reason": "Record the accepted discovery and selected delivery scenario." } ``` ```bash fa assessment export \ --file assessment-export.json \ --idempotency-key tutorial-assessment-export \ --json ``` The content-digested pack includes the estate, accepted assessment, normalized inventory, dependency graph and edges, applicable plan scenarios, generation time, and governance revision. ## Databricks App views [#databricks-app-views] The same state is available in two authenticated pages: * **Discover → Assessment studio** shows runs, inventory, graph edges, blockers, acceptance, and immutable export. * **Plan → Target blueprint and wave planner** shows target services, mappings, wave prerequisites, economics, scenario selection, and freeze evidence. Both pages invoke the same Platform actions used by the CLI. A UI action cannot bypass schema validation, authorization, tenant isolation, policy, idempotency, or audit. ## Failure and recovery behavior [#failure-and-recovery-behavior] | Condition | Result | Recovery | | -------------------------------------------- | --------------------------------- | --------------------------------------------------------------------------- | | unknown or cross-estate object in an edge | blocked | correct the normalized object mapping and submit a new batch | | duplicate edge | blocked | remove the duplicate; do not change the idempotency key to force it through | | incomplete edge count | blocked | record the remaining batches or start a corrected graph | | inventory count or digest changed | conflict | reload the current assessment projection and review drift | | dependency cycle | plan generated with blocker | disposition or redesign the cycle, then generate a new scenario | | agent or service account attempts acceptance | forbidden | an admitted natural-person operator performs the decision | | plan selected before scope freeze | selection allowed, freeze blocked | freeze the engagement, then retry the same logical freeze safely | | adapter or API unavailable | exit `6` | retry with the same stable idempotency key after the dependency recovers | Scope acceptance also requires the operator to choose the engagement explicitly. Airlift never infers acceptance authority from whichever engagement happens to appear first. This workflow produces planning evidence. It does not certify conversion, data parity, deployment, or production cutover; those use their own evidence profiles and gates. # Choose a developer path # Choose your path [#choose-your-path] Airlift moves an estate to Databricks as one governed job: ```text Connect → Inventory → Scope → Build → Move → Prove → Go-live ↘ Fixes (residue / discrepancies) ↗ ``` Start from where you sit in that job rather than reading the documentation front to back. ## Run or follow a migration [#run-or-follow-a-migration] Most readers should start in the App and only drop to developer surfaces when a stage needs an adapter or integration: | You are | Start here | What you will do | | -------------------------------- | -------------------------------------------------------------------------- | -------------------------------------------------------------------------- | | a migrator running the job | [App walkthroughs](/docs/getting-started/ui-walkthroughs) | follow one migration from engagement setup through cutover, stage by stage | | setting up your first engagement | [Engagements](/docs/getting-started/engagements) | create the engagement, bind a connection, run guided setup | | a migration lead or approver | [Guided migration journey](/docs/getting-started/guided-migration-journey) | see what each gate requires, who acts next, and what evidence clears it | | an auditor or sponsor | [Parity certificates](/docs/parity-certificates) | verify what a signed certificate claims and export evidence | ## Implement an integration surface [#implement-an-integration-surface] When a stage needs code from your project, start with the component you are implementing: | You are building | Start here | What you will implement | | ------------------------------ | ------------------------------------------------------------- | ------------------------------------------------------------------------------ | | a source migration integration | [Source developer workflow](/docs/sources/developer-workflow) | source plan, Lakebridge route, transfer, validation, and governed action calls | | an assessment adapter | [Run the assessment](/docs/migration/assessment) | workspace-job submission and immutable output admission | | a conversion factory | [Conversion factory](/docs/migration/conversion) | deterministic conversion, residue classification, and artifact recording | | a transfer driver | [Data transfer](/docs/migration/transfer) | checkpointed snapshot/incremental movement and reconciliation | | a validation provider | [Validation and readiness](/docs/migration/validation) | admitted evidence records and profile-track observations | | an Airlift deployment | [Deployment](/docs/deployment) | App, durable storage, jobs, Temporal, identity, secrets, and production checks | | a cutover effector | [Cutover and rollback](/docs/migration/cutover) | checkpoint, apply-once, independent verification, and recovery | | CI or operator automation | [Airlift CLI](/docs/cli) | source plans, diagnostics, contract snapshots, certificate verification | ## Recommended implementation order [#recommended-implementation-order] Build one complete, evidence-producing path before adding optional integrations: 1. Generate and review the source profile and migration plan. 2. Wire a live Lakebridge assessment and import the normalized inventory. 3. Convert representative hard objects and retain immutable artifacts and residue. 4. Implement one restartable transfer profile and independent validation runner. 5. Resolve profile tracks, mint certificates, and rehearse a reversible wave. Runway, Radar, and Tower add release, operations, and work-management evidence. They are optional integrations rather than prerequisites for assessment, conversion, validation, or certification. See [family composition](/docs/integrations). ## Trust & certificates [#trust--certificates] Whichever path you take, two reference pages define what the resulting evidence is worth: * [Parity and migration certificates](/docs/parity-certificates) — what a signed certificate claims, the evidence depth behind it, and how to verify it offline. * [Security model](/docs/reference/security) — the identity, tenancy, agent, and approval boundaries the whole migration job runs inside. # Guided client onboarding # Guided client onboarding [#guided-client-onboarding] Use the guided setup flow for a new Azure Synapse-to-Databricks or SQL Server-to-Lakebase engagement. It turns seven governed requirements into one ordered journey: 1. create the migration workspace; 2. register the source estate; 3. bind a secret-safe connection reference; 4. verify source access with retained evidence; 5. activate the engagement; 6. request an assessment; and 7. review and accept the discovered scope. The Databricks App and `fa` call the same Fabric Platform actions. The App does not keep a second checklist: a step becomes complete only when its domain event is present in the organization-scoped projection. ## Use the guided Databricks App flow [#use-the-guided-databricks-app-flow] Open **Engagements → New engagement**. Enter a program name, accountable owner, and the delivery services being requested. **Create draft** creates only the governed container; it does not connect to a source, run code, or claim readiness. After creation, Airlift opens **Guided setup** at: ```text //engagements//setup ``` The page shows all seven requirements but enables only the first unmet action. Each action panel explains what to provide, why it is required, what evidence it creates, and where to read the source-specific guide. Completed steps have a green check. Later steps remain locked so operators cannot accidentally perform onboarding out of order. ### Register the source estate [#register-the-source-estate] Choose one of the currently supported Azure onboarding routes: | Client route | Source and variant | Typical target | | -------------------------- | ----------------------------------------------- | -------------------------------- | | Synapse dedicated pool | **Azure Synapse · Synapse dedicated SQL pool** | Databricks Lakehouse | | Synapse serverless pool | **Azure Synapse · Synapse serverless SQL pool** | Databricks Lakehouse | | Mixed Synapse workspace | **Azure Synapse · Mixed Synapse estate** | Databricks Lakehouse | | SQL Server | **SQL Server · SQL Server** | Databricks Lakehouse or Lakebase | | Azure SQL Database | **SQL Server · Azure SQL Database** | Lakebase or Databricks Lakehouse | | Azure SQL Managed Instance | **SQL Server · Azure SQL Managed Instance** | Lakebase or Databricks Lakehouse | Name the actual client environment, identify its owner, and select `development`, `staging`, or `production`. Only estates attached to this engagement appear in its sidebar and source workspaces. ### Register a secret-safe connection reference [#register-a-secret-safe-connection-reference] Create the credential in the client-approved secret system first. Then enter only an opaque reference such as: ```text databricks-connection://migration/client-synapse-metadata databricks-secret://migration/sql-server-reader secret://client-vault/source-metadata ``` A connection reference is never a password. Airlift rejects passwords, tokens, JDBC URLs containing credentials, query strings, and fragments. The binding declares the source estate and allowed capabilities such as `inventory_read` and `metadata_read`; the secret value never enters the Airlift ledger. ### Verify source access [#verify-source-access] Run the admitted source preflight with the same credential reference. The probe records a **connectivity diagnostic** covering every declared capability and proves, at minimum: * authentication using the referenced credential; * source identity and version; * metadata visibility for the intended schemas/databases; * denied access outside the admitted boundary; and * the adapter/tool generation that produced the result. The probe is recorded through `airlift.connection_diagnostic_record` by an admitted system principal; the handler derives the diagnostic digest — a caller can never supply it. Then choose **Verify binding** in the guided setup panel. The App attaches the recorded diagnostic automatically; it never asks an operator to paste a digest. **Verification evidence digest** is the 64-character SHA-256 digest of the recorded connectivity diagnostic. Airlift records the digest and authenticated verifier; it does not copy the source credentials or report body. Unattended jobs perform the same step with `fa connection verify`; its digest recipe lives in [Authenticated automation](/docs/cli/remote-automation). The binding changes from `pending` to `verified` only when the named diagnostic is fresh, matches the current binding revision, and every required probe passed. A login screenshot, a caller-authored “passed” flag, or a caller-chosen digest is not verification evidence. ### Activate the engagement [#activate-the-engagement] Review the source estate and verified connection, then choose **Activate engagement**. Activation accepts the starting configuration and allows governed discovery work. It does not freeze final scope, deploy artifacts, certify parity, or authorize cutover. ### Run the first assessment [#run-the-first-assessment] Choose the estate, retain the pinned adapter generation, and supply a unique request reference. **Start assessment** records an idempotent request for the Airlift worker. The worker uses the configured source adapter and later records the immutable report, normalized inventory, dependency data, source version, and exact tool generation. ```json title="assessment-start.json" { "estateId": "est_01ARZ3NDEKTSV4RRFFQ69G5FAV", "toolVersion": "lakebridge@", "requestRef": "client-discovery-001" } ``` ```bash fa assessment start \ --file assessment-start.json \ --idempotency-key client-discovery-001 fa assessment status --json ``` The setup flow displays **Assessment is running** until a configured worker records the result. It never invents inventory to make the step green. ### Accept the discovered scope [#accept-the-discovered-scope] When assessment recording finishes, choose **Open assessment review**. Review: * normalized source objects and object types; * source paths, complexity, and exclusions; * dependency edges and critical paths; * tool and report references; * the expected object and edge counts; and * assumptions that must remain true for planning. Accept the dependency graph first, then accept the assessment scope. Airlift computes the inventory, dependency, and scope digests. Planning consumes these accepted digests rather than a live source query or mutable spreadsheet. ```bash fa inventory list --estate-id --assessment-id --json fa inventory graph show --json fa assessment accept \ --file assessment-accept.json \ --idempotency-key client-scope-accept-v1 ``` After acceptance, **Guided setup** is complete and its primary action changes to **Build migration plan**. ## Perform the same setup with `fa` [#perform-the-same-setup-with-fa] Install the private CLI package from the registry configured by your organization: ```bash npm install --global @fabricorg/airlift-cli fa --version fa doctor --profile production ``` The CLI derives organization and actor identity from the authenticated Databricks context. Do not place an organization ID or user identity in action payloads. Create the estate, engagement, binding, and assessment with stable idempotency keys. Verification sits between diagnosis and activation: interactive operators choose **Verify binding** in the App, while unattended jobs use `fa connection verify` with the recorded diagnostic digest recipe in [Authenticated automation](/docs/cli/remote-automation) — a caller-chosen digest is rejected either way. ```bash fa estate register --file estate.json --idempotency-key client-estate-v1 fa engagement create --file engagement.json --idempotency-key client-engagement-v1 fa connection register --file source-binding.json --idempotency-key client-binding-v1 fa connection diagnose --file probes.json --idempotency-key client-diagnose-v1 # Verify the binding in the App, or with the digest recipe in Authenticated automation. fa engagement activate --idempotency-key client-activate-v1 fa assessment start --file assessment-start.json --idempotency-key client-assessment-v1 fa engagement status --json ``` Example estate payloads differ only where the source contract differs: ```json title="estate.json — Synapse" { "name": "Client production Synapse", "sourceSystem": "synapse", "sourceVariant": "synapse_dedicated_sql", "owner": "client-data-platform", "environment": "prod", "priority": "priority" } ``` ```json title="estate.json — SQL Server" { "name": "Client operational SQL Server", "sourceSystem": "sql_server", "sourceVariant": "sql_server", "owner": "client-application-team", "environment": "prod", "priority": "priority" } ``` The App immediately renders CLI-created records because both surfaces read the same governed projections. ## What the client can inspect [#what-the-client-can-inspect] After onboarding, each engagement exposes five shareable views: | View | What the client sees | | -------------------- | ------------------------------------------------------------------------------------------- | | **Guided setup** | Completed prerequisites, the current requirement, and exact next action | | **Overview** | Current phase, route, blockers, source scope, and recommended action | | **Migration status** | Eight evidence gates with terminal counts and blocker reasons | | **Artifacts** | Immutable report/code/release references, SHA-256 digests, lineage, and producer generation | | **Run ledger** | Assessment, conversion, movement, deployment, and validation executions | ## Production onboarding boundary [#production-onboarding-boundary] Before connecting production data, confirm the Databricks App uses workspace identity, the organization registry admits the client and operators, the durable store and artifact store are configured, and the source connection follows the client's least-privilege policy. Automated cutover remains disabled until a source-and-target-specific effector has passed the production cutover and rollback capability suite. Do not represent local fixtures or a successful setup walkthrough as client migration proof. Workspace and client claims require immutable provider run references for the same source, target, artifact digest, and validation profile. # Core concepts # Core concepts [#core-concepts] ## Organization and estate [#organization-and-estate] An **organization** is the tenant boundary. Membership, events, projections, evidence, and policy are organization-scoped. An **estate** represents one source system and environment under migration, such as a production Synapse estate or a staging SQL Server estate. Credentials never live in an estate record; it carries only an opaque connection reference. ## Migration object [#migration-object] A migration object is a table, view, stored procedure, function, ETL job, or report. Its executive funnel is intentionally simple: ```text discovered → planned → converting → converted → certified → cutover ↘ rework ↘ rollback/reconciliation excluded is a governed terminal lane ``` The funnel is a projection, not a substitute for evidence. `converted` means an artifact was produced. `certified` means the assigned validation profile resolved from admitted, fresh evidence and Airlift minted a signed certificate. ## Readiness tracks [#readiness-tracks] Nine tracks explain why an object can or cannot advance: inventory, target design, code, data movement, deployment, functional parity, non-functional evidence, business acceptance, and cutover readiness. A profile marks every track `required` or `not_applicable`; a required track resolves to `pending`, `passed`, `failed`, `stale`, or `waived`. ## Wave [#wave] A wave freezes a set of objects into one rehearsable cutover unit. Airlift owns wave scope, approvals, gate evaluation, cutover decision, outcome, and rollback record. A wave does not absorb Runway deployment state, Radar observations, or Tower task completion; it references their stable IDs and digests when those products are present. ## Evidence and certificates [#evidence-and-certificates] The parity certificate states the depth of a functional or data comparison. The migration certificate is broader: Airlift derives and signs an envelope from the full readiness profile. A report digest is not a signature. See [parity and migration certificates](/docs/parity-certificates). # Configuration map # Configuration map [#configuration-map] Airlift defaults are safe for tests and demonstrations, not production. Use `fa doctor --profile production` to inspect configuration shape without printing secret values. | Concern | Selection | Production expectation | | ----------------------- | ----------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | State store | `AIRLIFT_STORE` | `postgres` with a Lakebase App binding or explicit Postgres binding | | Authorization directory | `AIRLIFT_AUTHORIZATION_JSON` | Tenant, principal type, role, trusted worker/agent/installer, and validation-principal entries | | App identity | `AIRLIFT_TRUST_DATABRICKS_APP_HEADERS`, `DATABRICKS_APP_NAME` | Both gates present; OBO remains the authenticated workspace boundary | | Development self-review | `AIRLIFT_DEPLOYMENT_ENV`, `AIRLIFT_ALLOW_DEVELOPMENT_SELF_REVIEW` | Optional only for a `dev` deployment whose App name ends in `-dev`; recorded as a non-certifying override | | Evidence admission | `AIRLIFT_EVIDENCE_REGISTRY_JSON` | Immutable provider/run/reference/digest entries from admitted principals; entries bind the run's `artifactDigest`, `verdict`, and `completedAt` but establish no provider provenance, so a governed hazard profile needs a provider-backed `airliftEvidenceVerifier` | | Certificate signing | `AIRLIFT_EVIDENCE_SIGNING_KEY_ID`, `AIRLIFT_EVIDENCE_SIGNING_PRIVATE_KEY_PEM` | Secret-backed Ed25519 private key and stable key ID | | Offline verification | `AIRLIFT_EVIDENCE_VERIFY_KEYS_JSON` | Public-key map; private key is never required for verification | | Converter | `AIRLIFT_CONVERTER` | `lakebridge` plus job IDs, volume root, Databricks App service principal, and a certified version pin | | Domain workflow | `AIRLIFT_TEMPORAL_MODE`, Temporal connection variables | Temporal for durable cutover; signals wake and never approve | | Cutover effect | composition-injected effector | Client-certified checkpoint, apply-once, verify, and compensation contract | Do not place credentials in action parameters, workflow inputs, event payloads, evidence exports, or CLI flags. See the complete [configuration reference](/docs/reference/configuration). # Conversion to remediation tutorial # Conversion to remediation tutorial [#conversion-to-remediation-tutorial] This tutorial starts after [assessment acceptance and planning](/docs/getting-started/assessment-to-plan). It uses synthetic identifiers and JSON files so you can exercise the remote API without placing source code, credentials, or client details in the command payload. ## Prerequisites [#prerequisites] You need an active engagement, an accepted assessment, and at least one object in `planned` or `rework`. Configure `AIRLIFT_API_URL` and a short-lived `DATABRICKS_TOKEN`, then confirm the resources visible to your authenticated organization: ```bash fa engagement list fa assessment list fa inventory list --assessment-id ``` ## Run the factory path [#run-the-factory-path] 1. Create a batch with `fa conversion batch create --file batch.json`. 2. Start it with `fa conversion batch start --file batch-start.json`. 3. For each object, call `fa conversion attempt start`, execute your adapter, then call `fa conversion attempt record`. 4. Register successful output using `fa artifact register`. 5. Create a residue for each failed object. 6. Reconcile with `fa conversion batch complete`. The batch completes only after Airlift can account for every object. If a converted attempt's output digest has no matching target artifact, or a failed attempt has no active residue, reconciliation is blocked with an actionable error. ## Complete the human path [#complete-the-human-path] ```bash fa residue assign --file assignment.json --idempotency-key example-assignment fa artifact register --file repaired-artifact.json --idempotency-key example-repair-artifact fa residue resolve --file resolution.json --idempotency-key example-resolution fa residue review --file review.json --idempotency-key example-review ``` Assignment works directly from `open`. A delivery lead may record an optional forecast first with `fa residue estimate`; it is planning metadata, not a developer prerequisite. Use a different admitted person for `review` than for `resolve`. Use an Experiments run reference in the resolution file. Review closes the engineering case; it does not mint a migration certificate. Continue with [validation](/docs/migration/validation). ## Inspect in the App [#inspect-in-the-app] Open **Conversion** to see batch and attempt lineage. Open **Remediation** to see the human/agent lanes, effort, skills, assignment, external work reference, artifact, and review status. Refreshing or replaying the event stream reconstructs the same projections. ## Expected denial checks [#expected-denial-checks] Verify these before connecting real scope: * duplicate object IDs are rejected at batch creation; * an object outside the accepted assessment is rejected; * starting an attempt with a different converter generation is rejected; * batch completion blocks on missing attempts or artifacts; * an agent cannot assign or review remediation; * a resolver cannot review the same residue; * a foreign-organization artifact, batch, or residue ID is not readable or mutable. These are production controls, not optional tutorial assertions. # Demonstrate an Azure migration # Demonstrate an Azure migration [#demonstrate-an-azure-migration] This page is a presentation script layered on the [guided migration journey](/docs/getting-started/guided-migration-journey/), which is the primary App-first walkthrough of the Azure Synapse to Databricks Lakehouse path. Walk the screens there; use this page for what to say, what to show, and which honesty boundaries to hold. Use this walkthrough when evaluating Airlift with an engineering team. The goal is not to show a checklist with every box green. The goal is to show how Airlift turns source-system work into inspectable artifacts, independent evidence, explicit human remediation, and one safe next action. Lead with the artifacts and the next action; open the run ledger only if the audience asks about audit. Start in **Engagements**. Each card is a governed migration workspace. It identifies the source and Databricks target, current evidence gate, blocker count, owner, completion derived from terminal evidence, and the next action that can advance the migration. ## Synapse to Databricks Lakehouse [#synapse-to-databricks-lakehouse] Present the Synapse journey as the complete factory flow, following the screen sequence in the [guided migration journey](/docs/getting-started/guided-migration-journey/): 1. Airlift admits the source assessment, inventory, dependencies, and accepted scope. 2. Lakebridge or another admitted converter produces target candidates; Airlift records the exact producer generation and artifact digests. 3. Unsupported transaction, temporary-table, or orchestration behavior becomes visible residue — normal specialist work, presented as such rather than as an unsafe success. 4. A repaired target artifact is registered and independently validated. 5. Data movement records watermarks, restart checkpoints, counts, and reconciliation. 6. Airlift mints only the certificates supported by the evidence. 7. Runway deployment and production cutover remain independent gates. Open **Artifacts** from the engagement proof snapshot. A useful technical review should show the executed source reference, generated or repaired Databricks target, Experiments validation evidence, SHA-256 digests, and producer versions. Open **Run ledger** only when an engineer asks what actually ran or how audit works. ## SQL Server to Databricks Lakebase [#sql-server-to-databricks-lakebase] Use the SQL Server journey to explain target fit and human remediation: 1. `fa lakebase plan` classifies schemas, tables, views, identities, constraints, procedures, and external operational workloads. 2. PostgreSQL-compatible objects receive Lakebase target mappings and immutable DDL artifacts. 3. A representative workspace certification can exercise a connected SQL Server engine and a managed Lakebase target with the same plan and dataset digests. `pnpm demo:admit` reports `workspaceEvidenceCorroborated: false` and preserves the warning until an admitted provider records a passing validation run that names the registered artifact ID and exact certification digest; artifact registration never self-attests that proof. 4. SQL Agent, SSIS, reporting, or application behavior that does not belong in Lakebase stays in a named redesign lane with skills, owner, output artifact, validation, and review requirements. That last item is a feature, not a failed demo. It shows that Airlift does not hide specialist work inside an automation percentage. Open the remediation case to show the diagnosis and the sequence that clears it: assign an owner, register the repaired artifact, attach validation, submit for review, and record an independent decision. ## Commands to show engineers [#commands-to-show-engineers] The App and CLI read the same organization-scoped projection: ```bash fa engagement list --json fa engagement status --json fa artifact list --engagement-id --json fa validation list --engagement-id --json fa deployment list --engagement-id --json ``` For SQL Server target planning: ```bash fa lakebase inspect --file sql-server-manifest.json fa lakebase plan --file sql-server-manifest.json --json \ > .airlift/sql-server-lakebase-plan.json fa lakebase qualification-check \ --file .airlift/sql-server-lakebase-workspace-qualification.json ``` For Synapse source planning: ```bash fa source inspect synapse fa source plan synapse --variant synapse_dedicated_sql --json \ > .airlift/synapse-source-plan.json fa engagement status --json ``` ## What to say about readiness [#what-to-say-about-readiness] Use the proof label shown by the App and keep its boundary intact: | Visible state | What it proves | What it does not prove | | ---------------------------- | ----------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------ | | Artifacts recorded | Immutable inputs or outputs and their lineage exist | Behavioral parity | | Development assurance | Source and target artifacts are linked to a passing governed validation run | Independently observed release or client acceptance | | Connected workspace evidence | Available only after a sanitized same-digest producer run is admitted; this candidate does not currently claim it | The client estate or production cutover | | Client proven | Accepted client scope passed the required profile | Production cutover unless its separate gates also pass | Do not promise a generic time reduction. Use Airlift's measured-value records after a client baseline exists, and report automation, human effort, elapsed time, defects, and validation outcomes with their observation windows and reviewers. # Create a migration engagement # Create a migration engagement [#create-a-migration-engagement] An Airlift engagement is the top-level delivery record for one migration or modernization program. It connects source estates, services, immutable external references, connection bindings, waves, artifacts, evidence, and operator decisions under one organization. Use an engagement when your application needs to answer: * which estates and services are in scope; * who owns the migration; * which source, target, and service connections have been verified; * which exact scope was frozen for delivery or cutover; and * which governed actions changed that scope. ## Use the Databricks App [#use-the-databricks-app] Start in **Migration inbox**. It ranks active engagements by blockers and failed evidence, then gives each row one primary action: resolve the next blocker or continue the earliest incomplete gate. Use **Status** when you need the complete governed projection rather than the operating queue. Open **Engagements** to search or filter the portfolio and see one card per migration workspace. Each card shows the source-to-target route, owner, current phase, completed gates, active blockers, capability preflight, and gate-completion percentage. * Select the card title or non-action area to open the engagement overview. * Choose **Continue migration** to bypass the overview and open the current actionable workspace. When conversion residue blocks the engagement, this opens the matching remediation case rather than another status screen. * Choose **View migration status** to inspect all eight delivery phases and their evidence. * Choose **New engagement** to open `/engagements/new`. After draft creation, Airlift opens `/engagements//setup` and guides source registration, a secret-safe connection reference, verification evidence, activation, assessment, and scope acceptance in order. It never redirects to an unrelated existing engagement. The sidebar starts in **Portfolio view** and deliberately hides source-specific and journey links until you choose a current engagement. Open **Current engagement**, then select a migration workspace. Airlift keeps that engagement visible in the desktop context bar and adds only its attached sources and phase workspaces to navigation. Returning to **Portfolio view** removes that source-specific navigation without changing migration state. Choose **Search migrations** or press ⌘K/Ctrl K to jump directly to an engagement overview, migration status, artifact explorer, run ledger, or source workspace. Search is navigation only; it never changes scope or runs a governed action. The sidebar presents the eight evidence gates in delivery order: discover, plan, convert, move data, prepare the release, validate, certify, and cut over. **Resolve conversion issues** is a contextual workbench used when the Convert gate produces residue; it is not a ninth evidence gate. Hover or focus a navigation item to see what that workspace owns. Compiler, modernization, portfolio, and administration workbenches remain available under **Advanced tools**; most migration operators do not need them for the everyday guided journey. The engagement workspace places **Your next step** before the journey and metrics. Its primary button opens the first incomplete gate or exact blocker, and the current phase card repeats that action for discoverability. Source-specific entry points, artifact and validation counts, and **Manage engagement** follow. Management opens automatically when a draft needs activation or when no source is configured; otherwise it remains collapsed. Every engagement has five stable, shareable views: | View | Use it for | | -------------------- | ------------------------------------------------------------------------------------------------------------------- | | **Overview** | Read the next action, journey, source scope, and governed lifecycle controls. | | **Guided setup** | Complete client prerequisites in order from source registration through accepted discovery scope. | | **Migration status** | Explain phase verdicts, blockers, evidence references, and the derived completion percentage. | | **Artifacts** | Inspect immutable assessment, converted-code, validation, and release references with digests and lineage. | | **Run ledger** | Trace assessment, conversion, transfer, deployment, and validation executions without copying provider-owned state. | The lifecycle controls mean: | Control | What it does | What it does not do | | ------------------------ | ---------------------------------------------------------------------------- | ------------------------------------------------------------ | | **Create draft** | Creates the governed container for ownership, services, and initial estates. | It does not start assessment or claim readiness. | | **Activate** | Makes the accepted configuration available for governed migration work. | It does not freeze scope or authorize cutover. | | **Capability preflight** | Compares requested services with current provider evidence. | It does not certify this customer migration. | | **Freeze scope** | Computes a stable digest for the accepted estates, services, and references. | It does not deploy, validate, certify, or cut over anything. | Inline help buttons expose these definitions with mouse, keyboard, or touch. Consequential prerequisites and blocked reasons remain visible without requiring a tooltip. ## Check the capability boundary before execution [#check-the-capability-boundary-before-execution] An engagement preflight translates the selected services into an evidence requirement, then checks every scoped estate and source variant against the tenant capability registry. Run it before activating a factory pilot, promising a client outcome, or scheduling a cutover: ```bash fa engagement preflight eng_ fa engagement preflight eng_ --json ``` Discovery requires hermetic assessment and dependency proof. A factory pilot requires workspace-proven execution, movement, validation, and target generation. A migration factory requires representative client proof. Cutover assurance additionally requires a production-certified cutover and rollback path. Modernization adds its own separately evidenced requirement. The command exits with status `1` when blocked and explains the next action for each missing, proposed, stale, unavailable, or under-proved capability. Human and mixed lanes are valid delivery paths when proven, but remain visible with their required skills so the engagement can staff and estimate them explicitly. ## Create and activate the engagement [#create-and-activate-the-engagement] Applications invoke the same Platform actions used by the Airlift workbench. Derive the organization and actor from your authenticated server context; never accept them as authoritative request-body fields. ```ts import { AIRLIFT_ACTION_IDS, contentDigest } from "@fabricorg/airlift"; const created = await runtime.invokeAction(AIRLIFT_ACTION_IDS.engagementCreate, { tenantId: session.organizationId, actorId: session.userId, actorType: "natural_person", idempotencyKey: commandId, params: { name: "Enterprise warehouse migration", owner: "data-platform", services: ["discovery", "migration_factory", "modernization"], estateIds: [estateId], externalRefs: [ { system: "crm", type: "opportunity", id: "OPP-1042" }, ], }, }); if (!created.ok) throw new Error(`${created.stage}: ${created.error}`); const engagementId = created.data.engagementId as string; await runtime.invokeAction(AIRLIFT_ACTION_IDS.engagementActivate, { tenantId: session.organizationId, actorId: session.userId, actorType: "natural_person", idempotencyKey: `${commandId}:activate`, params: { engagementId }, }); ``` Creation produces a `draft`. Activation marks the record ready for execution. Updates are accepted only while the engagement is `draft` or `active`. ## Register connection bindings [#register-connection-bindings] A connection binding describes what Airlift may ask an external connection to do. It stores an opaque credential reference and declared capabilities; it does not store a password, token, connection string, or secret value. ```ts const binding = await runtime.invokeAction( AIRLIFT_ACTION_IDS.connectionBindingRegister, { tenantId: session.organizationId, actorId: session.userId, actorType: "natural_person", idempotencyKey: `${commandId}:source-binding`, params: { engagementId, estateId, name: "Source metadata access", sourceSystem: "synapse", direction: "source", credentialRef: "databricks-connection://synapse-metadata", capabilities: ["inventory_read", "metadata_read"], }, }, ); ``` Accepted reference schemes are `connection://`, `databricks-connection://`, `databricks-secret://`, and `secret://`. User information, query strings, fragments, and unrecognized schemes are rejected by the action schema. After your connection verifier completes its source-specific checks, record the connectivity diagnostic as an admitted system principal, then verify the binding with the recorded diagnostic's derived digest: ```ts const diagnostic = await runtime.invokeAction( AIRLIFT_ACTION_IDS.connectionDiagnosticRecord, { tenantId: session.organizationId, actorId: session.servicePrincipalId, actorType: "system", idempotencyKey: `${commandId}:diagnose-source-binding`, params: { connectionBindingId: binding.data.connectionBindingId, bindingRevisionDigest: connectionBindingRevisionDigest(bindingRow), providerId: "acme-source-probe", providerVersion: "probe@1.0.0", evidenceRef: "volumes/evidence/diagnostics/binding.json", evidenceDigest: contentDigest(probeReport), runRef: "runs/diagnostic/binding", startedAt, completedAt, probes, // every probe required by the declared capabilities, all passing }, }, ); await runtime.invokeAction(AIRLIFT_ACTION_IDS.connectionBindingVerify, { tenantId: session.organizationId, actorId: session.userId, actorType: "natural_person", idempotencyKey: `${commandId}:verify-source-binding`, params: { connectionBindingId: binding.data.connectionBindingId, // verificationDigest must be the recorded diagnostic digest; a caller-chosen // digest is not evidence. verificationDigest: diagnostic.data.diagnosticDigest, }, }); ``` Registration does not imply connectivity. The binding remains `pending` until admitted verification evidence is recorded. A retired binding remains visible in the audit and projection history but cannot return to `verified`. ## Freeze the delivery scope [#freeze-the-delivery-scope] Freezing is a governed action, not a client-authored flag. Airlift computes the SHA-256 scope digest from the recorded engagement ID, services, estates, and external references. ```ts const frozen = await runtime.invokeAction(AIRLIFT_ACTION_IDS.engagementFreeze, { tenantId: session.organizationId, actorId: session.userId, actorType: "natural_person", idempotencyKey: `${commandId}:freeze`, params: { engagementId }, }); ``` A frozen engagement rejects later edits. Use the resulting `scopeDigest` as the stable input reference for delivery plans, evidence exports, deployment releases, and cutover prechecks. ## Read the projections [#read-the-projections] Queries use organization-scoped projections. They do not mutate state. ```ts const engagement = runtime.db.engagements.get(session.organizationId, engagementId); const bindings = runtime.db.connectionBindings.listByEngagement( session.organizationId, engagementId, ); ``` The Databricks App exposes the same records under **Engagements**. Workbench forms are thin authenticated transports over these actions; policy or schema denials are shown to the operator and remain in the governed invocation ledger. ## Action summary [#action-summary] | Task | Governed action | | ------------------------------------------ | -------------------------------------- | | create a draft | `airlift.engagement_create` | | change editable scope | `airlift.engagement_update` | | activate execution | `airlift.engagement_activate` | | compute and freeze scope digest | `airlift.engagement_freeze` | | register an opaque connection reference | `airlift.connection_binding_register` | | record an admitted connectivity diagnostic | `airlift.connection_diagnostic_record` | | verify with a recorded diagnostic digest | `airlift.connection_binding_verify` | | retire a binding | `airlift.connection_binding_retire` | Agent principals cannot invoke these actions. A separately admitted service principal may use the narrow `automation` role through the authenticated remote API. Migration agents may assess and convert within their admitted bounds; they cannot govern engagement scope or connection admission. # Guided migration journey (App walkthrough) # Follow the guided migration journey [#follow-the-guided-migration-journey] This is the primary App walkthrough. It follows one synthetic **Azure Synapse to Databricks Lakehouse** engagement through the App, reading the same governed projection the CLI and API read. [App walkthroughs](/docs/getting-started/ui-walkthroughs/) is the annotated screenshot catalog with matching `fa` commands; [Demonstrate an Azure migration](/docs/getting-started/demo-walkthrough/) adapts this journey into a presentation script for engineering audiences. Every screenshot is an automated capture from a governed, public synthetic workspace. The images contain no client data, credentials, real workspace identifiers, or production claims. If this is your first visit, start with [the in-app guidance](/docs/getting-started/in-app-guidance/) to learn the product tour, persistent Guide, and contextual help controls. ## Start with the artifact and the next action [#start-with-the-artifact-and-the-next-action] Airlift does not ask developers to interpret a dashboard and guess what to do. Every engagement phase page reads the governed migration projection and presents one **What Airlift needs next** panel before metrics and ledgers, alongside direct links to the artifacts that phase has already produced. The panel always answers: 1. **What this phase does.** The outcome this workspace governs and why it exists. 2. **What Airlift observed.** The current phase summary from governed records. 3. **Why progress stopped.** The first missing prerequisite or active blocker. 4. **What is the risk if bypassed?** The concrete migration failure the gate prevents. 5. **Who acts next?** The engineer, architect, provider, or independent approver who owns the action. 6. **Evidence that clears the gate.** The exact artifact, run, digest, approval, or provider observation Airlift must admit. Choose the primary **Do this next** action. It opens the relevant form or returns you to the earliest incomplete phase. It never changes readiness merely by navigating. ## The eight phase workspaces [#the-eight-phase-workspaces] | Phase | Developer work | Evidence Airlift expects | | --------- | --------------------------------------------------------------- | --------------------------------------------------------------------------------------- | | Discover | Run the source assessment and review inventory and dependencies | Admitted report plus accepted inventory and dependency digests | | Plan | Select a target blueprint and dependency-ordered waves | Selected and frozen plan bound to accepted source digests | | Convert | Run converter batches, inspect attempts, and repair residue | Immutable target artifacts plus governed residue closure | | Move data | Plan snapshot/incremental movement and operate checkpoints | Verified bindings, watermarks, restart checkpoints, and reconciliation digest | | Deploy | Declare the required release and use Runway to execute it | Matching release reference, artifact digest, environment, operation, and terminal state | | Validate | Request independent Experiments-backed checks | Passing object-specific runs and disposition of discrepancies | | Certify | Review profiles, readiness tracks, and acceptance | System-minted certificates backed by active admitted evidence | | Cut over | Freeze the wave, rehearse, approve, execute once, and verify | Certified scope, approvals, runbook, operational evidence, and rollback proof | Later phases remain visible but are locked until the earliest incomplete gate is cleared. The primary action on a locked page returns to that earlier phase and preserves the engagement identifier. ## Discover: admit the assessment [#discover-admit-the-assessment] The discovery workspace turns Lakebridge Profiler and Analyzer output into reviewed inventory and dependency evidence. The artifact that matters is the immutable assessment pack; the next action is to accept or dispute its scope. ## Plan: freeze scope against accepted digests [#plan-freeze-scope-against-accepted-digests] Planning binds the target blueprint, dependency order, and wave scope to the accepted discovery digests. Freezing is the artifact; conversion starts only against a frozen plan. ## Convert: candidates become immutable artifacts [#convert-candidates-become-immutable-artifacts] The conversion factory records deterministic attempts and registers every output as an immutable artifact with its producer generation and content digest. Converter success is an attempt outcome, not parity evidence. ## Repair residue: normal specialist work [#repair-residue-normal-specialist-work] Unsupported transaction, temporary-table, or orchestration behavior surfaces as a named residue case with an owner, a diagnosis, and the evidence needed to close it. Residue is normal specialist work on any real estate, not a defect in the walkthrough: Airlift makes it explicit, assignable, measurable, and independently reviewable instead of hiding it in an automation percentage. ## Move data: restartable, reconciled transfer [#move-data-restartable-reconciled-transfer] The transfer workspace tracks snapshot and incremental movement through watermarks, restart checkpoints, counts, rejects, and a reconciliation digest. A completed copy job is insufficient without that evidence. ## Deploy: a contract between Airlift and Runway [#deploy-a-contract-between-airlift-and-runway] The deployment page is a contract between Airlift and Runway: * **Airlift** records what must be deployed and decides whether observed evidence matches. * **Runway** deploys, promotes, rolls back, reconciles, and owns environment state. A page showing four zeros means **no release requirement has been declared**. It does not mean deployment succeeded, failed, or is complete. | Counter | Meaning | What to do | | -------------------- | ----------------------------------------------------------------------------- | --------------------------------------------------------------------------- | | Requirements | Desired release outcomes recorded for this engagement | If zero, select deployable artifacts and create the first requirement | | Awaiting observation | A requirement exists but no verified matching Runway result has been admitted | Deploy with Runway, then admit or reconcile its release reference | | Matched | Required and observed digest, environment, operation, and state match | Continue to independent validation | | Blocked or uncertain | Evidence failed, is unverifiable, or differs from the requirement | Inspect the requirement detail and correct or re-observe the Runway release | ### Create the first deployment requirement [#create-the-first-deployment-requirement] 1. Select the immutable converted code, configuration, and deployment manifests that form one release. 2. Choose **Deploy a new release** for the first release. Use Promote only for an existing release and Roll back only for an intentional recovery. 3. Choose the environment where the next validation run will execute. 4. Normally require **Release succeeded**. 5. Create the requirement. Airlift computes the desired digest and waits; it does not execute Runway. Runway then deploys the same artifact set. Airlift advances the deployment gate only when the admitted Runway observation matches every required field. ## Validate: independent evidence against exact digests [#validate-independent-evidence-against-exact-digests] The validation laboratory requests object-specific Experiments checks against exact artifact digests, separating validation evidence from the converter that produced the candidate. Failed required checks become discrepancies with a governed disposition path. ## Certify: readiness is evidence-derived [#certify-readiness-is-evidence-derived] The assurance center is a read-only projection of which object-specific evidence tracks have passed, failed, expired, or been waived. Certificates are system-minted from admitted evidence; nobody authors one. ## Cut over: every gate agrees before execution [#cut-over-every-gate-agrees-before-execution] The cutover control room stays blocked until frozen scope, evidence, rehearsal, distinct approvals, operational health, and rollback capability all agree. It reflects the durable workflow and governed gate; it is not a manually editable checklist. ## If someone asks about audit [#if-someone-asks-about-audit] The walkthrough leads with artifacts and next actions, not the ledger. When an audience asks how Airlift proves what ran, open the run ledger: it links provider-owned executions to the governed engagement without copying their internal state, and a provider success remains evidence to evaluate rather than an automatic readiness advance. ## Use contextual help [#use-contextual-help] Select a question-mark control to define a term without leaving the page. Tooltips render in the browser document layer, reposition on scrolling and resizing, and stay inside the viewport. They are available by pointer and keyboard; press Escape to close one and return focus to its help control. A help card is supplementary. Blockers, prerequisites, and recovery instructions always remain visible in the phase guidance panel. ## Use the CLI with the same truth [#use-the-cli-with-the-same-truth] The App and `fa` read the same organization-scoped projections: ```bash fa engagement status --json fa artifact list --engagement-id --json fa deployment list --engagement-id --json fa validation list --engagement-id --json ``` Use the returned `currentPhase`, `blockers`, and phase `nextAction`; do not compute a second percentage in automation. A UI click, task completion, or LLM response never advances the ledger without the owning governed Platform action and its required evidence. ## AI assistance boundary [#ai-assistance-boundary] An assistant may summarize the same phase guidance, link the relevant developer guide, or explain a field in more conversational language. Airlift does not let an LLM invent readiness: the assistant must treat the governed projection as ground truth. It cannot mint evidence, change the current phase, approve a waiver, certify an object, or authorize cutover. # Use the in-app guidance # Use the in-app guidance [#use-the-in-app-guidance] Airlift is designed to tell a developer what to do next, why the action is required, and which evidence will advance the migration. You do not need to infer readiness from a counter or learn the entire navigation before starting work. The console has three complementary guidance surfaces: | Surface | When to use it | What it provides | | -------------------------- | ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | | First-use welcome and tour | Your first visit, or whenever you replay it | A short orientation to engagement context, search, the migration journey, phase workspaces, and the persistent Guide | | Airlift Guide | From any console page | A plain-language explanation of the current page, the next action, current engagement context, and relevant developer articles | | Question-mark help | Beside an unfamiliar term or metric | A concise definition plus a link to the deeper developer article when one exists | ## Take the first-use tour [#take-the-first-use-tour] On your first visit, Airlift explains three product rules: 1. An **engagement** is the governed record for one client migration. 2. Airlift recommends the **earliest action** the ledger can safely accept. 3. **Evidence**, not navigation or ticket completion, advances the migration journey. Select **Take the 2-minute tour** to highlight the engagement switcher, migration search, journey navigation, phase workspace, and Guide. Use **Back**, **Next**, or **Skip tour** at any time. Press Escape to exit. The tour records only a browser-local display preference. It does not create a Platform action, change an engagement, or write migration evidence. ## Open the Guide from any page [#open-the-guide-from-any-page] Select **Guide** in the lower-right corner of the console. The panel answers: * What is this page for? * What should I do next? * Which engagement and phase am I working in? * How many evidence gates and blockers are recorded? * Where is the detailed developer article? Select **Replay the product tour** if you want the orientation again. The Guide is read-only. It cannot approve, certify, waive, execute cutover, or change any migration record. ## Use contextual help [#use-contextual-help] Select a question-mark button beside a field, counter, or workflow term. The help card stays within the browser viewport and has an opaque, high-contrast surface. It can be opened by pointer or keyboard. * Select **Read the developer article** for the complete workflow. * Press Escape to close the card and return focus to its button. * Select the close button or click outside the card to dismiss it. Contextual help defines a term. The phase guidance panel remains the authoritative place for the observed state, blocker diagnosis, owner, required evidence, and next action. ## Use specialist workbenches without guessing [#use-specialist-workbenches-without-guessing] The advanced compiler, qualification, enterprise-application, modernization, Databricks-native, and application-kit pages use the same guidance hierarchy: 1. **Purpose** explains when the workbench belongs in a migration and what it does not own. 2. **How this workspace works** names the prerequisite and the three-step developer path. 3. **Summary metrics** show governed records, with question-mark help for terms such as native generator, registered bundle, Runway matched, or workspace proof. 4. **Current state** distinguishes an empty prerequisite from a completed result. 5. **One primary action** opens the exact upstream workspace needed to continue. A zero does not automatically mean failure. For example, **Generated candidates: 0** means the pipeline compiler has not registered an executable artifact set, while **Workspace proven: 0** means no candidate has yet joined generated bytes, Runway, Experiments, and workspace identity on one digest. Open the question-mark help to read the definition, then follow the visible primary action to create the missing upstream record. These workbenches are intentionally under **Advanced tools**. Most migration operators should follow the engagement journey. Open a specialist workbench when the journey sends you there or when you are responsible for that compiler, release, validation, or modernization boundary. ## Understand why guidance does not use a free-form agent [#understand-why-guidance-does-not-use-a-free-form-agent] The console derives guidance from governed engagement projections and deterministic workflow rules. An LLM explanation must never invent readiness, approve evidence, or select a cutover decision. Harness-backed agents can assist bounded migration work, but the Platform action ledger and Airlift policies remain the source of truth. Continue with [the guided migration journey](/docs/getting-started/guided-migration-journey/) to learn what each of the eight phase workspaces expects. # Local lifecycle cookbook This walkthrough is a single runnable file shipped with the SDK — see the [full listing](/docs/getting-started/local-lifecycle-listing). It drives the whole governed lifecycle through `createAirliftRuntime()` composed with `createAirliftConfiguredAuthorization()` — the composition intended for unit tests, examples, and local demos — and it is executable documentation: the repository's test suite runs it on every change, so every snippet below reflects contracts that actually execute. The cookbook is versioned against **action manifest generation 11** and asserts the pin at startup: ```js import { AIRLIFT_ACTION_MANIFEST_GENERATION } from '@fabricorg/airlift'; if (AIRLIFT_ACTION_MANIFEST_GENERATION !== 11) { throw new Error('This walkthrough is versioned against action manifest generation 11.'); } ``` To run it, install `@fabricorg/airlift`, save the [full listing](/docs/getting-started/local-lifecycle-listing) as `local-lifecycle.mjs`, and run it with Node.js 22+: ```bash npm install @fabricorg/airlift node local-lifecycle.mjs ./cookbook-out ``` It also ships inside the npm package itself, at `node_modules/@fabricorg/airlift/examples/local-lifecycle.mjs`, in releases published in 0.16.3 and later. Contributors working from a repository checkout can run it directly against the built package surface instead: ```bash pnpm install pnpm --filter @fabricorg/airlift build node packages/airlift/examples/local-lifecycle.mjs ./cookbook-out ``` ## Configured authorization [#configured-authorization] The authorization directory is install-time configuration, not request input. Memberships bind natural persons to tenant roles; system and agent principals are admitted only through narrow trusted-id lanes. ```js const authorizationConfig = { memberships: [ { organizationId: ORG, principal: 'operator@acme.example', principalType: 'natural_person', role: 'admin' }, { organizationId: ORG, principal: 'approver@acme.example', principalType: 'natural_person', role: 'approver' }, ], // System principals admitted to the narrow worker transition set. trustedWorkerIds: ['svc-airlift-worker'], // Agent principals admitted only to a bounded action set: assessment, // dependency mapping, plan generation, inventory, conversion, artifacts, // residue, discrepancy triage, and modernization recommendations. Agents can // NEVER approve, certify, mint certificates, or execute cutover. trustedAgentIds: ['agent-convert-01'], // System principals admitted to the platform registry bootstrap boundary. platformInstallerIds: ['svc-airlift-installer'], // Tenant-bound principals permitted to submit independently produced evidence. admittedValidationPrincipals: [ { organizationId: ORG, principal: 'svc-airlift-worker', provider: 'experiments' }, ], }; ``` The runtime composes that directory with evidence services and an Ed25519 signer: ```js const runtime = createAirliftRuntime({ authorization: createAirliftConfiguredAuthorization(authorizationConfig), services: { // Admits a validation run only when producer, provider, run ref, evidence // ref, and digest all match an immutable registry entry. ...configuredAirliftEvidenceServices(authorizationConfig, [evidenceRegistryEntry]), airliftEvidenceSigner: createEd25519EvidenceSigner({ keyId, privateKeyPem }), }, }); ``` ## Platform-tenant bootstrap [#platform-tenant-bootstrap] Organizations are minted from the platform registry tenant, never from an org tenant. `airlift.org_provision` requires **all three** of: * `tenantId = AIRLIFT_PLATFORM_TENANT` (`'airlift-platform'`), * a `system` actor listed in `platformInstallerIds`, * `{ orgId, name }` params. ```js await runtime.invokeAction(AIRLIFT_ACTION_IDS.orgProvision, { tenantId: AIRLIFT_PLATFORM_TENANT, actorId: 'svc-airlift-installer', actorType: 'system', params: { orgId: 'acme-analytics', name: 'Acme Analytics' }, idempotencyKey: 'cookbook:org', }); ``` Invoking it with the org's **own** tenant id fails with `Module "airlift" is not enabled for tenant …`. That message reads like a defect but is the installer boundary working as designed: the installer grant is tenant-bound, so entitlement fails closed outside the platform tenant. ## Happy-path ordering [#happy-path-ordering] Each stage lists its acting principal; prerequisites are explicit. The runnable script contains the full parameter shapes for every call. 1. **Estate** — `estate_register` (operator). `environment: 'prod'` engages the full gate policies. 2. **Engagement** — `engagement_create` with `services: ['discovery', 'migration_factory']`, then `engagement_activate` (operator). Discovery and factory actions require an active engagement. 3. **Assessment** — `assessment_record` (operator) with tool version, report refs, digests, and object counts. 4. **Inventory + dependency graph accept** — `object_register` per object (operator); `dependency_graph_start` and `dependency_batch_record` (trusted agent); `dependency_graph_accept` with an explicit `expectedEdgeCount`, then `assessment_accept` with `expectedObjectCount` (operator). Acceptance freezes scope digests. 5. **Plan generate → select → freeze** — `plan_generate` (dependency-aware wave candidates plus economics), `plan_select`, then `engagement_freeze` **before** `plan_freeze` (operator) — freezing a plan requires the engagement scope frozen first. 6. **Wave plan/assign** — `wave_plan`, `wave_assign` per object (operator); objects reach `planned`. 7. **Conversion batch** — `conversion_batch_create` (operator), `conversion_batch_start` (worker), then per object `conversion_start` / `conversion_record` (worker). Every converted object must carry an immutable `target_code` artifact (`artifact_register`) whose digest matches the recorded `outputDigest`. 8. **Residue lane** — for each failed conversion: `residue_create → residue_estimate` (agent) `→ residue_assign` (operator). The batch then completes with `completed_with_residue` — completion requires every failed object to hold an *active* residue case. Remediation continues after completion: `artifact_register` + `residue_resolve` (operator), `residue_review` by a **distinct natural person** (approver), then a fresh governed `conversion_start` / `conversion_record` brings the repaired object from `rework` back to `converted`. 9. **Validation evidence admission** — `validation_profile_assign` (operator), then `validation_run_record` (admitted principal). The run is admitted only because producer, provider, run ref, evidence ref, and digest all match the immutable registry entry — a caller's digest is never trusted merely because it is well formed. 10. **Readiness tracks** — `readiness_record` per required track of the assigned profile, admitting the validation run as evidence. 11. **Parity certify** — `parity_certify` with honest depth fields (see below). 12. **Business acceptance** — `business_accept` is a human decision by the approver. 13. **Migration certificate mint** — `migration_certificate_mint` binds the certificate to the exact `expectedProfileDigest` and `expectedReadinessDigest` the caller observed; the Ed25519 signer signs the envelope digest. 14. **Wave approval** — `wave_approve` by the approver (a natural person distinct from the operator). 15. **Offline verification** — the script verifies the envelope in-process, exports the envelope and public key, and prints the CLI command for you to run, fully offline: ```bash fa certificate verify ./cookbook-out/migration-certificate.json --keys ./cookbook-out/verify-keys.json ``` ## Evidence export [#evidence-export] `airlift.evidence_export` assembles the replayable evidence pack for one wave (or one migration object) from the org's event log — assessment, conversions with tool versions, parity certificates with declared evidence depth, approvals, and the governance revision in force at export time. It returns the pack as `reportJson` alongside a sha256 `reportDigest` over that exact string, and records the export itself as an `AirliftEvidenceExported` event keyed by that digest — the audit stream proves what was attested without persisting the pack contents. The params contract requires a `waveId` or `objectId` plus a `reason` (1–500 characters): ```js const result = await runtime.invokeAction(AIRLIFT_ACTION_IDS.evidenceExport, { tenantId: ORG, actorId: 'operator@acme.example', actorType: 'natural_person', params: { waveId, reason: 'Assemble the replayable evidence pack for the approved wave.' }, idempotencyKey: 'cookbook:evidence-export', }); // result.data: { waveId, exportKey, reportDigest, reason, reportJson } ``` Authorization requires the `airlift:evidence:export` permission, held by the admin, operator, approver, and validator roles for natural-person principals (the cookbook's operator is configured as `admin`). Evidence export is deliberately **not** a trusted-worker action — unlike the conversion and certification steps above, a `system` principal is denied even though it drove most of the lifecycle. The cookbook demonstrates this: the conversion worker's attempt fails closed with the same generic entitlement error as the other module-boundary denials, before the operator invokes the action successfully. ## Expected denials are first-class outcomes [#expected-denials-are-first-class-outcomes] A suite of passing checks behaves identically with the boundary disabled, so the cookbook demonstrates each denial and fails if any of them unexpectedly succeeds: | Attempt | Outcome | | ----------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `org_provision` from the org's own tenant | Entitlement fails closed: `Module "airlift" is not enabled for tenant …` | | Agent actor invoking `wave_approve` | Entitlement fails closed with the same generic error — approvals are outside the bounded agent action set, so the agent is not entitled to the module for this action at all | | Parity certificate claiming `sampled_rows` depth without `aggregateChecksums` | Policy blocks the overclaim: a certificate must carry the evidence for every depth it claims | | Resolver reviewing their own residue | Policy blocks it: `The resolver cannot independently review the same residue.` | | System worker invoking `evidence_export` | Entitlement fails closed with the same generic error — evidence export is natural-person-only and deliberately not a trusted-worker action | ## Idempotency keys [#idempotency-keys] Every mutation passes an `idempotencyKey` following a stable `:[:]` convention (for example `cookbook:cnv-record:`). Within the same runtime, re-invoking with the same key collapses onto the original invocation instead of duplicating the logical command — the cookbook demonstrates this by registering the estate twice and asserting both calls return the same estate id. The in-memory store lives only as long as the process, so a rerun of the script starts from scratch; against the durable Postgres store the same keys give cross-process replay safety. ## The pre-cutover boundary [#the-pre-cutover-boundary] The local walkthrough stops at certified objects and an approved wave. It deliberately does **not** claim cutover execution or rehearsal, data transfer, workspace job runs, deployment evidence, or production operational evidence. Those stages require the durable runtime, real effectors, and independently produced evidence — a local in-memory demo proving them would be an overclaim, which is exactly what the governance model exists to prevent. # Local lifecycle cookbook — full listing Save this listing as `local-lifecycle.mjs` in a directory with `@fabricorg/airlift` installed (`npm install @fabricorg/airlift`), then run `node local-lifecycle.mjs ./cookbook-out` with Node.js 22+. {/* GENERATED:COOKBOOK-LISTING:START */} ```js // Local lifecycle cookbook for the Fabric Airlift SDK's in-memory runtime. // // Runnable, end-to-end walkthrough of the governed migration lifecycle using only // the published package surface: `createAirliftRuntime()` composed with // `createAirliftConfiguredAuthorization()` — the composition intended for unit // tests, examples, and local demos. Every mutation goes through the governed // action pipeline; nothing here writes to a real workspace. // // Usage: // node examples/local-lifecycle.mjs [output-directory] // // The script exits non-zero if any expected-success step fails or any // expected-denial unexpectedly succeeds. It writes the minted migration // certificate envelope and the Ed25519 public key to the output directory // (default: a temporary directory) and prints the offline verification command. import { generateKeyPairSync } from 'node:crypto'; import { mkdirSync, mkdtempSync, writeFileSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { AIRLIFT_ACTION_IDS, AIRLIFT_ACTION_MANIFEST_GENERATION, AIRLIFT_PLATFORM_TENANT, configuredAirliftEvidenceServices, createAirliftConfiguredAuthorization, createAirliftRuntime, createEd25519EvidenceSigner, createEd25519EvidenceVerifier, defaultValidationProfile, readinessSnapshot, verifyMigrationCertificateEnvelope, } from '@fabricorg/airlift'; // ── Generation pin ─────────────────────────────────────────────────────────── // This cookbook is written against action manifest generation 11. A generation // bump changes governed action contracts; re-verify every step before bumping. if (AIRLIFT_ACTION_MANIFEST_GENERATION !== 11) { console.error( `This cookbook is versioned against action manifest generation 11; the installed @fabricorg/airlift reports generation ${AIRLIFT_ACTION_MANIFEST_GENERATION}. Check the package changelog before trusting these steps.`, ); process.exit(1); } // ── Cast of principals ─────────────────────────────────────────────────────── // Roles are deliberately narrow. Natural persons hold tenant roles; system and // agent principals are admitted only through the trusted-id lanes below. const ORG = 'acme-analytics'; const OPERATOR = { actorId: 'operator@acme.example', actorType: 'natural_person' }; const APPROVER = { actorId: 'approver@acme.example', actorType: 'natural_person' }; // distinct person: separation of duties const WORKER = { actorId: 'svc-airlift-worker', actorType: 'system' }; const AGENT = { actorId: 'agent-convert-01', actorType: 'agent' }; const INSTALLER = { actorId: 'svc-airlift-installer', actorType: 'system' }; // ── Configured authorization ───────────────────────────────────────────────── // The directory is install-time configuration, not request input: // - memberships bind natural-person principals to tenant roles, // - trustedWorkerIds admits system principals to the narrow worker transition set, // - trustedAgentIds admits agents only to a bounded action set (assessment, // dependency mapping, plan generation, inventory, conversion, artifacts, // residue, discrepancy triage, modernization recommendations) — never // approvals, certification, or cutover, // - platformInstallerIds admits system principals to the org-registry bootstrap // boundary (org_provision / org_retire in the platform tenant only), // - admittedValidationPrincipals lists who may submit independently produced // validation evidence, per organization and provider. const VALIDATION_RUN_REF = 'jobs/run/2002'; const VALIDATION_EVIDENCE_REF = 'volumes/evidence/wave-1/fact_sales.json'; const VALIDATION_EVIDENCE_DIGEST = 'a'.repeat(64); const VALIDATION_ARTIFACT_DIGEST = 'b'.repeat(64); const authorizationConfig = { memberships: [ { // The delivery lead holds `admin` here so the separation-of-duties denial // below is decided by the governance policy (resolver ≠ reviewer), not // masked by a missing role permission. Production teams typically split // the narrower operator / approver / validator roles across people. organizationId: ORG, principal: OPERATOR.actorId, principalType: 'natural_person', role: 'admin', }, { organizationId: ORG, principal: APPROVER.actorId, principalType: 'natural_person', role: 'approver', }, ], trustedWorkerIds: [WORKER.actorId], trustedAgentIds: [AGENT.actorId], platformInstallerIds: [INSTALLER.actorId], admittedValidationPrincipals: [ { organizationId: ORG, principal: WORKER.actorId, provider: 'experiments' }, ], }; // ── Runtime services ───────────────────────────────────────────────────────── // The evidence registry stands in for a provider's authoritative run API: the // verifier admits a validation run only when every field (producer, provider, // run ref, evidence ref, digest) matches an immutable registry entry, so a // caller's digest is never trusted merely because it is well formed. // // Binding and attestation are different claims, and this manifest only does the // first. The entry below BINDS the run's artifact digest and verdict: a run // submitted with either one different fails verification, so neither can be // restated after the fact. It does not ATTEST them — a static manifest never sees // the provider's evidence body, so it cannot derive what the provider reported and // check the submission against it. // // That is sufficient here because this estate is Synapse, outside the governed // hazard profile set. A governed conversion additionally requires provider-ESTABLISHED // `verdict` and `completedAt`, and only a verifier that reads the provider's evidence // body can establish them — the worker's Volume-backed store derives both from the // digest-checked document. A static manifest cannot, no matter how complete, so a // governed conversion fails closed until such a verifier is injected. The artifact // digest is never attested by anyone: provider run documents do not name the artifact // they exercised, so the conversion-hazard gate corroborates it at certification. // // The Ed25519 signer signs migration-certificate envelopes so they verify offline. const { publicKey, privateKey } = generateKeyPairSync('ed25519'); const SIGNING_KEY_ID = 'cookbook-local-2026'; const publicKeyPem = publicKey.export({ type: 'spki', format: 'pem' }).toString(); const runtime = createAirliftRuntime({ authorization: createAirliftConfiguredAuthorization(authorizationConfig), services: { ...configuredAirliftEvidenceServices(authorizationConfig, [ { organizationId: ORG, producerPrincipal: WORKER.actorId, provider: 'experiments', providerRunRef: VALIDATION_RUN_REF, evidenceRef: VALIDATION_EVIDENCE_REF, evidenceDigest: VALIDATION_EVIDENCE_DIGEST, artifactDigest: VALIDATION_ARTIFACT_DIGEST, verdict: 'passed', }, ]), airliftEvidenceSigner: createEd25519EvidenceSigner({ keyId: SIGNING_KEY_ID, privateKeyPem: privateKey.export({ type: 'pkcs8', format: 'pem' }).toString(), }), }, }); // ── Invocation helpers ─────────────────────────────────────────────────────── // Idempotency keys make every mutation safely replayable: re-invoking with the // same key collapses onto the original invocation instead of duplicating it. // Convention: a stable `:[:]` string per logical mutation. let failures = 0; async function ok(actionId, tenantId, actor, params, idempotencyKey) { const result = await runtime.invokeAction(actionId, { tenantId, actorId: actor.actorId, actorType: actor.actorType, params, ...(idempotencyKey ? { idempotencyKey } : {}), }); if (!result.ok) { failures += 1; console.error(`✗ ${actionId} failed: [${result.stage}] ${result.error}`); process.exit(1); } console.log(`✓ ${actionId}`); return result; } async function denied(label, expected, actionId, tenantId, actor, params) { const result = await runtime.invokeAction(actionId, { tenantId, actorId: actor.actorId, actorType: actor.actorType, params, }); if (result.ok) { failures += 1; console.error(`✗ EXPECTED DENIAL but ${actionId} succeeded: ${label}`); process.exit(1); } // A denial only proves the named boundary if it fired for the expected reason. if (result.stage !== expected.stage || !result.error.includes(expected.errorIncludes)) { failures += 1; console.error( `✗ ${actionId} was denied for the WRONG reason: expected [${expected.stage}] …${expected.errorIncludes}…, got [${result.stage}] ${result.error}`, ); process.exit(1); } console.log(`✓ denied as expected — ${label}\n [${result.stage}] ${result.error}`); return result; } // ── 0. Platform-tenant bootstrap ───────────────────────────────────────────── // Organizations are minted from the platform registry tenant, never from an org // tenant. The installer boundary requires ALL of: tenantId = AIRLIFT_PLATFORM_TENANT // ('airlift-platform'), a system actor listed in platformInstallerIds, and // {orgId, name} params. Invoking org_provision with the org's own tenantId is the // classic trap: the installer grant is tenant-bound, so entitlement fails with // `Module "airlift" is not enabled for tenant …` — a boundary, not a defect. console.log('\n— Platform bootstrap —'); await denied( 'org_provision from the org tenant (the installer grant is platform-tenant-bound)', { stage: 'handler', errorIncludes: 'is not enabled for tenant' }, AIRLIFT_ACTION_IDS.orgProvision, ORG, INSTALLER, { orgId: ORG, name: 'Acme Analytics' }, ); await ok( AIRLIFT_ACTION_IDS.orgProvision, AIRLIFT_PLATFORM_TENANT, INSTALLER, { orgId: ORG, name: 'Acme Analytics' }, 'cookbook:org', ); // ── 1. Estate ──────────────────────────────────────────────────────────────── // The estate is the source system under migration. `environment: 'prod'` engages // the full gate policies (dev estates get the approval-skip edge). console.log('\n— Discovery and planning —'); const estate = await ok( AIRLIFT_ACTION_IDS.estateRegister, ORG, OPERATOR, { name: 'Legacy Synapse DW', sourceSystem: 'synapse', owner: 'data-platform', environment: 'prod', }, 'cookbook:estate', ); const estateId = estate.data.estateId; // Idempotent replay: within the same runtime, re-invoking with the same key // collapses onto the original invocation instead of double-applying. (The // in-memory store lives only as long as this process; the durable Postgres // store gives the same keys cross-process replay semantics.) const replay = await ok( AIRLIFT_ACTION_IDS.estateRegister, ORG, OPERATOR, { name: 'Legacy Synapse DW', sourceSystem: 'synapse', owner: 'data-platform', environment: 'prod', }, 'cookbook:estate', ); if (replay.data.estateId !== estateId) { console.error('✗ idempotent replay minted a second estate instead of collapsing'); process.exit(1); } console.log(`✓ replay with the same idempotency key returned the original estate ${estateId}`); // ── 2. Engagement ──────────────────────────────────────────────────────────── // Engagements scope commercial services over estates; discovery and factory // actions require an active engagement. const engagement = await ok( AIRLIFT_ACTION_IDS.engagementCreate, ORG, OPERATOR, { name: 'Synapse modernization wave 1', services: ['discovery', 'migration_factory'], owner: OPERATOR.actorId, estateIds: [estateId], }, 'cookbook:engagement', ); const engagementId = engagement.data.engagementId; await ok(AIRLIFT_ACTION_IDS.engagementActivate, ORG, OPERATOR, { engagementId }, 'cookbook:engage'); // ── 3. Assessment ──────────────────────────────────────────────────────────── // The assessment records the analyzer run (tool, report refs, digests, counts). const assessment = await ok( AIRLIFT_ACTION_IDS.assessmentRecord, ORG, OPERATOR, { estateId, jobRunRef: 'jobs/run/assessment-1', reportRef: 'volumes/discovery/assessment.json', reportDigest: 'b'.repeat(64), inventoryDigest: 'c'.repeat(64), dependencyDigest: 'd'.repeat(64), toolVersion: 'lakebridge@0.14.2', objectCounts: { tables: 2, views: 1, storedProcedures: 0, etlJobs: 0, other: 0 }, }, 'cookbook:assessment', ); const assessmentId = assessment.data.assessmentId; // ── 4. Inventory + dependency graph accept ─────────────────────────────────── // Objects are registered against the assessment; the dependency graph is loaded // by a trusted agent and accepted by a human with an explicit expected edge // count, then the assessment itself is accepted with the expected object count. // Acceptance freezes scope digests — later drift is detectable. const objectIds = []; for (const row of [ { name: 'dbo.fact_sales', objectType: 'table', complexity: 'medium' }, { name: 'dbo.dim_customer', objectType: 'table', complexity: 'low' }, { name: 'dbo.vw_sales_by_region', objectType: 'view', complexity: 'medium' }, ]) { const object = await ok( AIRLIFT_ACTION_IDS.objectRegister, ORG, OPERATOR, { estateId, name: row.name, objectType: row.objectType, sourcePath: `dw/${row.name}.sql`, complexity: row.complexity, assessmentRef: assessmentId, }, `cookbook:obj:${row.name}`, ); objectIds.push(object.data.objectId); } const graph = await ok( AIRLIFT_ACTION_IDS.dependencyGraphStart, ORG, AGENT, { engagementId, assessmentId, expectedEdgeCount: 2, graphRef: { system: 'lakebridge', type: 'dependency_graph', id: 'graph-wave-1' }, toolVersion: 'lakebridge@0.14.2', }, 'cookbook:graph', ); const dependencyGraphId = graph.data.dependencyGraphId; await ok( AIRLIFT_ACTION_IDS.dependencyBatchRecord, ORG, AGENT, { dependencyGraphId, edges: [ { fromObjectId: objectIds[2], toObjectId: objectIds[0], kind: 'consumer', confidence: 1, critical: true, }, { fromObjectId: objectIds[2], toObjectId: objectIds[1], kind: 'consumer', confidence: 1, critical: false, }, ], }, 'cookbook:edges', ); await ok( AIRLIFT_ACTION_IDS.dependencyGraphAccept, ORG, OPERATOR, { dependencyGraphId, expectedEdgeCount: 2 }, 'cookbook:graph-accept', ); await ok( AIRLIFT_ACTION_IDS.assessmentAccept, ORG, OPERATOR, { assessmentId, engagementId, dependencyGraphId, expectedObjectCount: objectIds.length, assumptions: ['A representative production window is available for validation.'], exclusions: [], }, 'cookbook:assessment-accept', ); // ── 5. Migration plan: generate → select → freeze ──────────────────────────── // Plan generation produces dependency-aware wave candidates plus economics; a // human selects the scenario. Freezing requires the engagement scope frozen // first — the plan digest then pins the delivery contract. const generated = await ok( AIRLIFT_ACTION_IDS.planGenerate, ORG, OPERATOR, { engagementId, assessmentId, name: 'Parity-first delivery plan', scenario: 'parity_first', targetWorkspaceRef: { system: 'databricks', type: 'workspace', id: 'acme-target' }, maxObjectsPerWave: 3, economics: { manualHoursPerComplexityPoint: 8, assistedHoursPerComplexityPoint: 3, blendedHourlyCostUsd: 180, sourceMonthlyRunCostUsd: 25_000, targetMonthlyRunCostUsd: 18_000, dualRunMonths: 2, }, assumptions: ['Unity Catalog prerequisites are completed before deployment.'], }, 'cookbook:plan', ); const migrationPlanId = generated.data.migrationPlanId; await ok(AIRLIFT_ACTION_IDS.planSelect, ORG, OPERATOR, { migrationPlanId }, 'cookbook:plan-select'); await ok( AIRLIFT_ACTION_IDS.engagementFreeze, ORG, OPERATOR, { engagementId }, 'cookbook:engagement-freeze', ); await ok(AIRLIFT_ACTION_IDS.planFreeze, ORG, OPERATOR, { migrationPlanId }, 'cookbook:plan-freeze'); // ── 6. Wave plan/assign — objects reach `planned` ──────────────────────────── console.log('\n— Wave and conversion factory —'); const wave = await ok( AIRLIFT_ACTION_IDS.wavePlan, ORG, OPERATOR, { estateId, name: 'Wave 1 — sales marts', targetWindow: '2026-09-15' }, 'cookbook:wave', ); const waveId = wave.data.waveId; for (const objectId of objectIds) { await ok( AIRLIFT_ACTION_IDS.waveAssign, ORG, OPERATOR, { waveId, objectId }, `cookbook:assign:${objectId}`, ); } // ── 7. Conversion batch + per-object conversions ───────────────────────────── // The batch groups conversions under one governed unit; the worker drives each // object through conversion_start / conversion_record. One object fails on // purpose to open the residue lane. const batch = await ok( AIRLIFT_ACTION_IDS.conversionBatchCreate, ORG, OPERATOR, { engagementId, assessmentId, name: 'Wave 1 stored-code batch', method: 'lakebridge', toolVersion: 'lakebridge@0.14.2', objectIds, }, 'cookbook:batch', ); const conversionBatchId = batch.data.conversionBatchId; await ok( AIRLIFT_ACTION_IDS.conversionBatchStart, ORG, WORKER, { conversionBatchId }, 'cookbook:batch-start', ); const conversionIds = []; for (const [index, objectId] of objectIds.entries()) { const start = await ok( AIRLIFT_ACTION_IDS.conversionStart, ORG, WORKER, { objectId, batchId: conversionBatchId, method: 'lakebridge', toolVersion: 'lakebridge@0.14.2', }, `cookbook:cnv-start:${objectId}`, ); conversionIds.push(start.data.conversionId); const isResidue = index === 2; // the view fails conversion and becomes residue await ok( AIRLIFT_ACTION_IDS.conversionRecord, ORG, WORKER, isResidue ? { conversionId: start.data.conversionId, objectId, outcome: 'failed', detail: 'Unsupported dynamic SQL construct.', } : { conversionId: start.data.conversionId, objectId, outcome: 'converted', artifactRef: `volumes/target/${objectId}.sql`, outputDigest: 'e'.repeat(64), }, `cookbook:cnv-record:${objectId}`, ); if (!isResidue) { // Batch completion requires every converted object to carry an immutable // target artifact whose digest matches the recorded conversion output. await ok( AIRLIFT_ACTION_IDS.artifactRegister, ORG, AGENT, { engagementId, objectId, kind: 'target_code', name: `Converted SQL for ${objectId}`, artifactRef: { system: 'databricks', type: 'volume_object', id: `target/${objectId}.sql`, digest: 'e'.repeat(64), }, digest: 'e'.repeat(64), mediaType: 'application/sql', toolVersion: 'lakebridge@0.14.2', }, `cookbook:artifact:${objectId}`, ); } } // ── 8. Residue lane: create → estimate → assign → resolve → review ─────────── // Residue is the human-remediation lane for what automation could not convert. // Resolution requires a registered artifact and validation evidence ref; review // must come from a different natural person than the resolver (separation of // duties) — demonstrated below in the denials section. const residueObjectId = objectIds[2]; const residue = await ok( AIRLIFT_ACTION_IDS.residueCreate, ORG, AGENT, { engagementId, objectId: residueObjectId, category: 'unsupported_construct', lane: 'human', summary: 'Rewrite dynamic SQL for Databricks SQL', requiredSkills: ['T-SQL', 'Databricks SQL'], commercialAttribution: 'managed_service', }, 'cookbook:residue', ); const residueId = residue.data.residueId; await ok( AIRLIFT_ACTION_IDS.residueEstimate, ORG, AGENT, { residueId, estimateMinutes: 240, requiredSkills: ['T-SQL', 'Databricks SQL'] }, 'cookbook:residue-estimate', ); await ok( AIRLIFT_ACTION_IDS.residueAssign, ORG, OPERATOR, { residueId, assignedToRef: { system: 'techfabric', type: 'engineer', id: 'migration-sql-specialist' }, workRef: { system: 'tower', type: 'work_item', id: 'work-42' }, }, 'cookbook:residue-assign', ); // The batch can complete once every failed object has an ACTIVE residue case — // completion reports `completed_with_residue`; remediation continues after it. await ok( AIRLIFT_ACTION_IDS.conversionBatchComplete, ORG, WORKER, { conversionBatchId }, 'cookbook:batch-complete', ); const repairedArtifact = await ok( AIRLIFT_ACTION_IDS.artifactRegister, ORG, OPERATOR, { engagementId, objectId: residueObjectId, kind: 'target_code', name: 'Repaired view SQL', artifactRef: { system: 'databricks', type: 'volume_object', id: 'target/vw_sales_by_region.sql', }, digest: 'f'.repeat(64), mediaType: 'application/sql', toolVersion: 'human-remediation@1', }, 'cookbook:residue-artifact', ); await ok( AIRLIFT_ACTION_IDS.residueResolve, ORG, OPERATOR, { residueId, resolvedArtifactId: repairedArtifact.data.artifactId, resolutionEvidenceRef: { system: 'experiments', type: 'validation_run', id: 'validation-residue-1', digest: 'a'.repeat(64), }, actualMinutes: 180, resolutionNotes: 'Replaced unsupported syntax and passed the scenario suite.', }, 'cookbook:residue-resolve', ); // ── Expected denial: separation of duties on residue review ────────────────── console.log('\n— Governed denials (first-class outcomes) —'); await denied( 'separation of duties: the resolver cannot review their own residue', { stage: 'handler', errorIncludes: 'The resolver cannot independently review the same residue.' }, AIRLIFT_ACTION_IDS.residueReview, ORG, OPERATOR, { residueId, decision: 'approved', reviewNotes: 'Self review must fail.' }, ); await ok( AIRLIFT_ACTION_IDS.residueReview, ORG, APPROVER, { residueId, decision: 'approved', reviewNotes: 'Artifact lineage and evidence reviewed.' }, 'cookbook:residue-review', ); // The failed conversion sent the object to `rework`; the reviewed repair is // recorded as a fresh governed conversion (outside the completed batch) // referencing the repaired artifact. const reconvert = await ok( AIRLIFT_ACTION_IDS.conversionStart, ORG, WORKER, { objectId: residueObjectId, method: 'manual', toolVersion: 'human-remediation@1', }, 'cookbook:cnv-start:residue-repair', ); await ok( AIRLIFT_ACTION_IDS.conversionRecord, ORG, WORKER, { conversionId: reconvert.data.conversionId, objectId: residueObjectId, outcome: 'converted', artifactRef: 'target/vw_sales_by_region.sql', outputDigest: 'f'.repeat(64), }, 'cookbook:cnv-record:residue-repair', ); // ── Expected denial: agents cannot approve waves ───────────────────────────── // Authorization is fail-closed at entitlement: wave_approve is outside the // bounded agent action set, so the agent is not entitled to the module at all // for this action — the denial is the generic entitlement error, not a // wave-specific message. await denied( 'agent actor attempting wave_approve (approvals are outside the bounded agent action set)', { stage: 'handler', errorIncludes: 'is not enabled for tenant' }, AIRLIFT_ACTION_IDS.waveApprove, ORG, AGENT, { waveId }, ); // ── 9. Validation evidence, readiness, certification per object ────────────── // For each object: validation profile → validation run (admitted principal + // verifier + immutable run refs) → readiness track records → parity certificate // (honest depth fields) → business acceptance → migration certificate mint. console.log('\n— Validation, readiness, and certification —'); function honestParityCertificate() { const now = new Date().toISOString(); return { // Depth honesty (ADR-0003): the certificate states its own evidence depth and // must carry the evidence for every depth it claims — policy blocks overclaims. evidenceDepth: 'sampled_rows', rowCountSource: 1_204_331, rowCountTarget: 1_204_331, aggregateChecksums: { 'sum(amount)': 'a1b2c3', 'count(*)': '1204331' }, sampledRowCount: 10_000, toolVersion: 'lakebridge@0.14.2', validationRunRef: VALIDATION_RUN_REF, sourceSnapshotAt: now, targetSnapshotAt: now, }; } // ── Expected denial: overclaimed parity depth ──────────────────────────────── const overclaiming = honestParityCertificate(); overclaiming.aggregateChecksums = undefined; // claims sampled_rows depth without aggregate evidence await denied( 'parity certificate claiming a depth without carrying its evidence', { stage: 'policy', errorIncludes: 'missing aggregateChecksums' }, AIRLIFT_ACTION_IDS.parityCertify, ORG, WORKER, { objectId: objectIds[0], certificate: overclaiming }, ); const certifiedObjects = []; for (const objectId of objectIds) { const object = runtime.db.objects.get(ORG, objectId); const profile = defaultValidationProfile(object.objectType); await ok( AIRLIFT_ACTION_IDS.validationProfileAssign, ORG, OPERATOR, { objectId, profileId: profile.profileId, profileVersion: profile.version, reason: 'Assign the default validation profile for the object type.', }, `cookbook:profile:${objectId}`, ); // The run is admitted only because the whole tuple — producer, provider, run ref, // evidence ref and digest, plus the artifact digest and verdict — matches the // immutable registry entry configured at startup. That is binding, not attestation: // the manifest establishes no provider provenance, which is sufficient here because // this Synapse estate is outside the governed hazard profile set. const run = await ok( AIRLIFT_ACTION_IDS.validationRunRecord, ORG, WORKER, { objectId, provider: 'experiments', providerRunRef: VALIDATION_RUN_REF, evidenceRef: VALIDATION_EVIDENCE_REF, evidenceDigest: VALIDATION_EVIDENCE_DIGEST, verdict: 'passed', toolVersion: 'databricks-testkit@0.9.0', artifactDigest: VALIDATION_ARTIFACT_DIGEST, sourceWatermark: 'source:v1', targetSnapshot: 'target:v1', completedAt: new Date().toISOString(), }, `cookbook:validation:${objectId}`, ); // Readiness tracks: admit the validation run as evidence for every required // track except business acceptance (a human decision, below) and cutover // readiness (out of scope for a local demo — see the boundary note). for (const requirement of profile.requirements) { if ( requirement.applicability === 'required' && requirement.track !== 'business_acceptance' && requirement.track !== 'cutover_readiness' ) { await ok( AIRLIFT_ACTION_IDS.readinessRecord, ORG, WORKER, { objectId, track: requirement.track, validationRunId: run.data.validationRunId, reason: 'Validation evidence admitted for the track.', }, `cookbook:readiness:${objectId}:${requirement.track}`, ); } } await ok( AIRLIFT_ACTION_IDS.parityCertify, ORG, WORKER, { objectId, certificate: honestParityCertificate() }, `cookbook:parity:${objectId}`, ); await ok( AIRLIFT_ACTION_IDS.businessAccept, ORG, APPROVER, { objectId, evidenceRef: `decisions/${objectId}`, evidenceDigest: 'c'.repeat(64), decision: 'accepted', reason: 'Business owner accepted parity and validation evidence.', }, `cookbook:accept:${objectId}`, ); // Minting binds the certificate to the exact profile and readiness digests the // caller observed — stale expectations are rejected. const assignment = runtime.db.profileAssignments.get(ORG, objectId); const snapshot = readinessSnapshot(runtime.db, ORG, objectId); const minted = await ok( AIRLIFT_ACTION_IDS.migrationCertificateMint, ORG, WORKER, { objectId, expectedProfileDigest: assignment.profileDigest, expectedReadinessDigest: snapshot.digest, }, `cookbook:mint:${objectId}`, ); certifiedObjects.push({ objectId, certificateId: minted.data.certificateId }); } // ── 10. Wave approval by a distinct natural person ─────────────────────────── await ok(AIRLIFT_ACTION_IDS.waveApprove, ORG, APPROVER, { waveId }, 'cookbook:wave-approve'); // ── 11. Offline certificate verification ───────────────────────────────────── // The minted envelope carries an Ed25519 signature over its content digest, so // it verifies offline with nothing but the public key. This script verifies // in-process; the printed `fa certificate verify` command is the offline // follow-up for you to run against the exported files. console.log('\n— Offline verification —'); const envelope = runtime.db.migrationCertificates.activeForObject(ORG, objectIds[0]).envelope; const verifier = createEd25519EvidenceVerifier({ [SIGNING_KEY_ID]: publicKeyPem }); if (!(await verifyMigrationCertificateEnvelope(envelope, verifier))) { console.error('✗ minted certificate failed offline verification'); process.exit(1); } console.log('✓ minted migration certificate verifies offline (in-process)'); const outDir = process.argv[2] ?? mkdtempSync(join(tmpdir(), 'airlift-cookbook-')); mkdirSync(outDir, { recursive: true }); // ── 12. Governed evidence export ───────────────────────────────────────────── // `evidence_export` assembles the replayable evidence pack for one wave (or one // object) from the org's event log — assessment, conversions, parity // certificates, approvals, everything addressed to or about the subject — plus // the governance revision in force at export time. Unlike every worker-driven // step above, this action requires a natural person holding // `airlift:evidence:export` (admin/operator/approver/validator — the cookbook's // operator is an admin); it is deliberately // NOT in the trusted-worker action set, so the conversion worker is denied even // though it drove the conversions the pack now reports on. console.log('\n— Evidence export —'); await denied( 'system worker invoking evidence_export (natural-person-only; not a trusted-worker action)', { stage: 'handler', errorIncludes: 'is not enabled for tenant' }, AIRLIFT_ACTION_IDS.evidenceExport, ORG, WORKER, { waveId, reason: 'Automated export attempt from the conversion worker.' }, ); const evidenceExport = await ok( AIRLIFT_ACTION_IDS.evidenceExport, ORG, OPERATOR, { waveId, reason: 'Assemble the replayable evidence pack for the approved wave.' }, 'cookbook:evidence-export', ); const evidenceReportFile = join(outDir, 'evidence-export.report.json'); const evidenceMetaFile = join(outDir, 'evidence-export.meta.json'); // `reportJson` is the exact string the action digested — write it byte for byte // so `reportDigest` remains verifiable against the file on disk. writeFileSync(evidenceReportFile, evidenceExport.data.reportJson); writeFileSync( evidenceMetaFile, `${JSON.stringify( { waveId: evidenceExport.data.waveId, exportKey: evidenceExport.data.exportKey, reportDigest: evidenceExport.data.reportDigest, reason: evidenceExport.data.reason, }, null, 2, )}\n`, ); console.log(`✓ evidence pack exported: ${evidenceReportFile}`); console.log(` exportKey: ${evidenceExport.data.exportKey}`); console.log(` reportDigest: ${evidenceExport.data.reportDigest}`); const certificateFile = join(outDir, 'migration-certificate.json'); const keysFile = join(outDir, 'verify-keys.json'); writeFileSync(certificateFile, `${JSON.stringify(envelope, null, 2)}\n`); writeFileSync(keysFile, `${JSON.stringify({ [SIGNING_KEY_ID]: publicKeyPem }, null, 2)}\n`); console.log(`\nCertificate envelope: ${certificateFile}`); console.log(`Verification keys: ${keysFile}`); console.log(`Verify offline with:\n fa certificate verify ${certificateFile} --keys ${keysFile}`); // ── Pre-cutover boundary ───────────────────────────────────────────────────── console.log(` — Pre-cutover boundary — This local demo stops at certified objects and an approved wave. It deliberately does NOT claim: cutover execution or rehearsal, data transfer, workspace job runs, deployment evidence, or production operational evidence. Those stages require the durable runtime, real effectors, and independently produced evidence — a local in-memory walkthrough proving them would be an overclaim, which is exactly what the governance model exists to prevent.`); console.log( `\nDone: ${certifiedObjects.length} objects certified, wave approved, 1 residue reviewed, evidence exported, 5 denials demonstrated.`, ); if (failures > 0) process.exit(1); ``` {/* GENERATED:COOKBOOK-LISTING:END */} # Quickstart The first path is the Databricks App. Open it, create an engagement, and follow the guided journey — the App walks each governed gate in order, shows why progress stopped and who acts next, and never asks you to paste a digest or author evidence by hand. ## Start in the Databricks App [#start-in-the-databricks-app] 1. Open the Airlift Databricks App and choose **Engagements → New engagement**. Enter a program name, accountable owner, and the delivery services being requested, then **Create draft**. The draft is only the governed container; it does not connect to a source, run code, or claim readiness. 2. Follow **Guided setup** for the engagement: register the source estate, bind a secret-safe connection reference, verify source access with retained evidence, activate the engagement, request an assessment, and review and accept the discovered scope. Each step names the evidence it creates and unlocks only when its governed event is recorded. 3. Continue with the [guided migration journey](/docs/getting-started/guided-migration-journey), which walks every phase — discover, plan, convert, move data, validate, certify, cut over — with its gate evidence and next actor. The App and the CLI call the same Fabric Platform actions and read the same governed projections, so anything created in one surface appears in the other. See [guided client onboarding](/docs/getting-started/client-onboarding) for the full setup walkthrough and [App walkthroughs](/docs/getting-started/ui-walkthroughs) for the stage-by-stage UI. ## Automation [#automation] Use the CLI and SDK when you need the same governed operations in CI, scripts, or an integration service. This appendix is the automation entry point; the complete authenticated surface is documented under [Airlift CLI](/docs/cli), including [authenticated automation](/docs/cli/remote-automation) for remote-endpoint identity, idempotency, and evidence recipes. ### Install the CLI [#install-the-cli] Prerequisites: Node.js **22+** and npm, pnpm, or another Node.js package manager. ```bash npm install --global @fabricorg/airlift-cli fa help ``` Use a pinned, installation-free invocation in CI: ```bash npx --yes --package @fabricorg/airlift-cli@0.18.4 fa sources --json ``` ### Generate a source migration plan [#generate-a-source-migration-plan] List the source profiles and generate a deterministic project contract before configuring credentials: ```bash fa sources fa source inspect synapse fa source plan synapse --json > synapse-plan.json ``` The plan covers inventory, conversion, transfer, validation, security, downstream consumers, cutover, and modernization. Each step names its Airlift actions, expected outputs, and exit criteria. Planning commands never connect to a source or mutate an Airlift deployment. ### Connect to a deployed Airlift App [#connect-to-a-deployed-airlift-app] After an administrator admits your Databricks principal to one Airlift organization, configure the authenticated remote endpoint: ```bash export AIRLIFT_API_URL="https://" export DATABRICKS_TOKEN="$(your-secret-provider read databricks-token)" ``` Create `engagement.json`: ```json { "name": "Synapse modernization", "services": ["discovery", "migration_factory", "modernization"], "owner": "data-platform" } ``` Then create and inspect the governed engagement: ```bash fa engagement create \ --file engagement.json \ --idempotency-key synapse-program-engagement \ --json fa engagement list ``` The token authenticates the request but is not the organization selector. Airlift derives the actor and organization at the App boundary and rejects ambiguous memberships. ### Add the SDK to an integration [#add-the-sdk-to-an-integration] Install the typed source registry, action contracts, schemas, and runtime composition surface in your Node.js project: ```bash npm install @fabricorg/airlift ``` ```ts import { AIRLIFT_ACTION_IDS, createSourceMigrationPlan, resolveSourceSystemProfile, } from '@fabricorg/airlift'; const source = resolveSourceSystemProfile('synapse'); const migrationPlan = createSourceMigrationPlan(source.id); console.log(source.workloadSurfaces); console.log(migrationPlan.steps.map((step) => ({ id: step.id, actions: step.airliftActions, exitCriteria: step.exitCriteria, }))); console.log(AIRLIFT_ACTION_IDS.assessmentRecord); ``` Your application invokes governed actions for assessment, inventory, waves, conversion attempts, validation runs, readiness observations, acceptance, certificate minting, cutover, rollback, and evidence export. Illegal transitions are rejected structurally; authorization and policy fail closed with an attributable reason. ### Connect project adapters [#connect-project-adapters] The SDK is the governance and evidence layer—not a source credential manager. A complete project supplies narrow adapters for: 1. Lakebridge assessment and conversion jobs in the Databricks workspace. 2. Source-specific snapshot and incremental transfer with restart checkpoints. 3. Independent parity and performance validation. 4. Client-approved cutover checkpoint, apply-once, verification, and rollback effects. Production cutover composition requires a client-certified `CutoverEffector`; selecting a runtime mode or setting an environment variable cannot enable an unverified effector. Use the [source developer workflow](/docs/sources/developer-workflow) to implement those adapters, then follow the guide for your [source system](/docs/sources). Production deployment of the Airlift application and workers is delivered as a Databricks project; it is not installed from the private application repository. ## Next commands [#next-commands] ```bash fa profiles fa actions --json fa doctor --profile production fa docs sources/synapse ``` Continue with the [complete CLI reference](/docs/cli/command-reference) and the [Synapse developer guide](/docs/sources/synapse), or select another supported source. # Transfer and deploy a wave # Transfer and deploy a wave [#transfer-and-deploy-a-wave] This tutorial starts after assessment acceptance and conversion. You should already have an engagement, estate, accepted objects, verified connections, and immutable converted artifacts. ## The end-to-end path [#the-end-to-end-path] Follow these developer guides in order: 1. [Build a resumable data transfer](/docs/migration/transfer) to create a transfer specification, plan scope, run, pause/resume, and reconcile. 2. [Deploy with Fabric Runway](/docs/integrations/runway) to register a deployment manifest, request the desired state, and observe the Runway result. 3. [Validation](/docs/migration/validation) to record independent data and business evidence. 4. [Cutover](/docs/migration/cutover) only after the required readiness tracks pass. ## Read the deployment page before you act [#read-the-deployment-page-before-you-act] The **Requirements**, **Awaiting observation**, **Matched**, and **Blocked or uncertain** counters describe a reconciliation workflow; they are not deployment-job counts. Four zeros mean that no desired release has been declared for the selected engagement. Choose **Create the first deployment requirement** and select the exact immutable artifacts that form the release. For a normal first release, choose **Deploy a new release**, the environment where validation will run, and **Release succeeded**. Airlift computes the desired digest; it does not deploy the artifacts itself. The next-action card then gives you the exact `fr deploy` command. Run it from the Runway artifact directory, copy the returned deployment ID and staged artifact SHA-256, and choose **Connect Runway deployment**. The connection is only a pointer. Choose **Check Runway result now** to make admitted automation query Runway and independently verify the release. After Runway executes the release, Airlift admits the verified release reference and compares artifact digest, environment, operation, and terminal state. A match unlocks validation. Missing or different evidence remains awaiting, blocked, or uncertain with a plain-language next action. If verification is disabled, configure the Runway observer; do not interpret the disabled button as a successful deployment. See [Follow the guided migration journey](/docs/getting-started/guided-migration-journey) for every phase screen and its evidence semantics. ## What to automate in CI [#what-to-automate-in-ci] ```bash fa transfer status "$TRANSFER_ID" --json > transfer-status.json fa deployment status "$DEPLOYMENT_REQUIREMENT_ID" --json > deployment-status.json ``` Fail your release when the transfer is not `completed`, the deployment reconciliation outcome is not `matched`, or a required readiness track is stale. Do not infer readiness from a job exit code. ## What to test regularly [#what-to-test-regularly] * duplicate transfer starts attach to one Temporal execution; * restart resumes from the latest increasing checkpoint; * pause is respected without losing an in-flight checkpoint; * excessive lag and row-count mismatch fail reconciliation; * a person cannot record runner or Runway evidence; * deployment digest drift blocks readiness; * unknown actual state remains uncertain; and * all governed action replays collapse under the same idempotency key. # App walkthroughs # Understand the App before running a migration [#understand-the-app-before-running-a-migration] For the primary App-first walkthrough, follow the [guided migration journey](/docs/getting-started/guided-migration-journey/): one synthetic Azure Synapse to Databricks Lakehouse engagement, artifact by artifact, with the next action per phase. This page is the annotated screenshot catalog that supports it — each image paired with the `fa` commands that read the same governed projection. Every screenshot in this documentation is an automated capture from a governed, public synthetic Airlift workspace. The images contain no client data, credentials, workspace hostnames, or internal deployment references. Select an image to open the full-size view. Each annotated image answers two questions in text as well as visually: * **What you are seeing** explains which governed projection or workbench produced the UI. * **What to do next** names the developer action and evidence needed to continue. The text is intentional. Screen readers, `llms.txt`, `llms-full.txt`, and developers who cannot inspect the image receive the same workflow explanation. ## 1. Choose the migration [#1-choose-the-migration] ```bash fa engagement list --json fa engagement status "$ENGAGEMENT_ID" --json ``` ## 2. Read the evidence gates [#2-read-the-evidence-gates] ```bash fa engagement status "$ENGAGEMENT_ID" fa residue list --engagement-id "$ENGAGEMENT_ID" --json ``` ## 3. Inspect the exact migration output [#3-inspect-the-exact-migration-output] ```bash fa artifact list --engagement-id "$ENGAGEMENT_ID" --json fa artifact show "$ARTIFACT_ID" --json ``` ## 4. Validate independently [#4-validate-independently] ```bash fa validation run --file validation-request.json \ --idempotency-key "$ENGAGEMENT_ID-validation-v1" fa validation list --engagement-id "$ENGAGEMENT_ID" --json ``` ## Screenshot transcript [#screenshot-transcript] The complete documented UI catalog covers these developer-visible states: | Area | What the image proves | Developer action | | ---------------- | ------------------------------------------------------------------- | --------------------------------------- | | Engagements | Scope, owner, phase, blockers, and next action are visible together | Choose or create the governed migration | | Source workspace | Navigation exposes only sources attached to the engagement | Run and admit the source assessment | | Discovery | Accepted inventory and dependencies have immutable digests | Review and accept scope | | Planning | Target design and waves are bound to accepted inventory | Freeze the selected plan | | Conversion | Attempts, residues, and target artifacts retain lineage | Repair residue and register output | | Transfer | Watermarks, checkpoints, rejects, and reconciliation are visible | Complete restartable movement | | Deployment | Airlift requirement is separate from Runway observation | Deploy the same digest with `fr` | | Validation | Experiments evidence is independent of conversion | Resolve failed discrepancies | | Assurance | Readiness tracks and certificates are evidence-derived | Complete missing tracks and acceptance | | Cutover | Frozen scope, rehearsal, approvals, health, and rollback must agree | Execute only after the gate passes | For a source-specific sequence, open [Azure Synapse](/docs/sources/synapse/) or [SQL Server](/docs/sources/sql-server/). Other source pages use clearly labeled representative workflow images until that source has its own admitted workspace evidence. # What leaders approve # What leaders approve [#what-leaders-approve] Most of a migration job runs without ceremony: assessments record, conversions start, transfers checkpoint, and validation runs execute as ordinary work. A small number of moments are different. They change what the organization is committed to, and Airlift requires a named human being — never an agent, never a script — to make each one through a governed Platform action. This page covers the common leader decisions. For the canonical action and permission catalog, full role model, principal types, and tenant configuration, see [roles and separation of duties](/docs/operations/roles-and-separation). ## Common human-only decisions [#common-human-only-decisions] | Decision | Plain-language meaning | When it appears | | -------------------- | -------------------------------------------------------------------------- | -------------------------------------------------------------- | | **Accept scope** | "These discovered objects and dependencies are the real scope of the job." | After inventory and dependency review, before planning | | **Freeze the plan** | "Lock this plan; waves are cut against it." | Before conversion at scale | | **Approve a wave** | "This wave's scope is certified and may proceed toward go-live." | Per wave, once its evidence is green | | **Request a waiver** | "Ask to carry a known residual risk, with an expiry." | Only when a blocked item cannot be cleared on the default path | | **Approve a waiver** | "A second, distinct person accepts that residual risk." | After a waiver request; never by the requester | | **Business accept** | "The business accepts this scope for go-live." | Before the certificate and cutover path | | **Execute cutover** | "Open the production cutover for this wave." | Only when the cutover gate is green | One adjacent moment is deliberately **not** a human decision: **certificate minting**. A migration certificate is system-minted from independently admitted evidence once policy thresholds are met. No person authors or edits a certificate payload — a leader cannot mint one by hand, and neither can an agent. The leader's job is the business acceptance and cutover decisions around the certificate, not the certificate itself. ## How a decision appears in the App [#how-a-decision-appears-in-the-app] A decision card renders only when policy actually requires a human, and it answers the same four questions as every other guidance surface: what this decision commits to, why it is required now, who may make it, and what evidence it is based on. There is one primary action per card. The App never shows a leader a form asking for a digest, an idempotency key, or a raw action name — the button on the card invokes the same governed Platform action the CLI invokes. ## Separation of duties is enforced, not suggested [#separation-of-duties-is-enforced-not-suggested] Approval authority is checked again at execution time, and it is independent of submission authority: * **Distinct approvers.** Waivers and go-live approvals require a second, distinct natural person. The requester cannot approve their own waiver; the person who submits a cutover cannot be its only approver. * **A resolver cannot review the same case.** The person who repaired a residue case never reviews it; review belongs to a different authorized person. * **No inherited approval.** An agent, converter, worker, or submitter cannot manufacture or inherit a natural person's approval from a workflow signal, a form field, or a CLI flag. Approval exists only as governed Platform state recorded by that person. * **Automation is bounded operationally.** The `automation` role may record or execute explicitly cataloged assessment, inventory, planning, transfer, validation, discrepancy, observation, and evidence work. It never carries approval, waiver, certificate, policy, membership, or cutover-execution authority. The exact list is machine-checked in [organization membership](/docs/reference/organization-membership/). ## Agents never decide [#agents-never-decide] The agent boundary is absolute and worth stating plainly: a bounded Harness agent may record assessments and objects, propose dependency graphs and plans, and start and record conversions. It **cannot approve, certify, waive, execute cutover**, and it cannot deploy. Agents also never accept assessment or dependency truth, select or freeze plans, submit readiness evidence, accept on behalf of the business, mint or invalidate certificates, roll back a cutover, or configure policy — and nothing in the App, the CLI, or an AI assistant gives an agent an alternative mutation path around these rules. When an assistant explains a decision card to you, it reads the same deterministic guidance you do. It may summarize and link; it may never invent a different next action or make the decision for you. ## Use the CLI with the same truth [#use-the-cli-with-the-same-truth] Leaders who prefer the terminal exercise the exact same governed actions — there is no lighter-weight approval path in the CLI: ```bash fa plan freeze --idempotency-key plan-freeze- --json fa cutover approve --idempotency-key cutover-approval- --json fa cutover start \ --window 2030-01-15T02:00:00Z/2030-01-15T04:00:00Z \ --reason "Approved production change window" \ --json ``` Scope acceptance, wave approval, waivers, and business acceptance are exercised from the decision cards in the App; every one of them records the same governed Platform action with the same actor and evidence. The App and `fa` read the same projections, so an approval made in one surface is immediately visible in the other, with the same actor, evidence, and audit record. ## Where to go deeper [#where-to-go-deeper] * [Roles and separation of duties](/docs/operations/roles-and-separation) — the five tenant roles, worker and agent principal lanes, and independent approval checks. * [Waivers and staleness](/docs/operations/waivers-and-staleness) — waiver expiry, evidence freshness, and what a waiver can never cover. * [Cutover control room](/docs/operations/cutover-control-room) — the go-live ceremony, the cutover gate, and rollback. * [When something fails](/docs/getting-started/when-something-fails) — the Fix list work that precedes most waiver requests. ## Trust & certificates [#trust--certificates] Two reference pages close the loop on what these approvals protect: * [Parity and migration certificates](/docs/parity-certificates) — what a signed certificate actually claims, the evidence depth it was minted from, and how to verify it offline. * [Security model](/docs/reference/security) — the identity, tenancy, evidence, and external-effect boundaries every approval on this page is enforced through. # When something fails (the Fix list) # When something fails [#when-something-fails] Something will fail. Unsupported source behavior, a failed independent check, a deployment observation that does not match the requirement — every real migration produces work that automation cannot finish alone. Airlift treats that work as a first-class part of the job, not as an error state: ```text Connect → Inventory → Scope → Build → Move → Prove → Go-live ↘ Fixes (residue / discrepancies) ↗ ``` This page explains the Fix list: how a failure becomes a work item, how the App tells you **why progress stopped** and **who acts next**, and the recovery action that clears each item. Follow the [guided migration journey](/docs/getting-started/guided-migration-journey/) first if you have not seen the eight stage workspaces this page builds on. ## Failure is a work item, not a dead end [#failure-is-a-work-item-not-a-dead-end] Two kinds of failure land in the Fix list: * **Residue.** Source behavior the converter could not carry across — an unsupported transaction pattern, a temporary-table idiom, an orchestration construct with no target equivalent. Each case is a named item with an owner, a diagnosis, and the evidence needed to close it. * **Discrepancies.** A failed required check from an independent validation run against an exact artifact digest. Each discrepancy has a governed disposition path, not a retry button. Residue and discrepancies are **normal specialist work** on any real estate. Airlift makes them explicit, assignable, measurable, and independently reviewable instead of hiding them inside an automation percentage. A Fix list with open items is not a broken migration; it is a migration telling the truth about the work that remains. ## Read one Fix list item [#read-one-fix-list-item] Every item in the Fix list answers the same questions the stage guidance panel answers: 1. **Why progress stopped.** The blocked reason in plain language — what automation could not do, or which independent check failed, and the concrete migration risk of carrying on anyway. Ledger codes and digests stay in the collapsed evidence drawer. 2. **Who acts next.** The named owner in a plain role: you, an assigned teammate, the system, or a lead. 3. **Recovery action.** One primary action — the exact repair, disposition, or re-observation that can clear the item. 4. **Evidence that clears it.** The immutable artifact, passing run, review, or provider observation Airlift must admit. Closing an external ticket never clears the ledger. ## Who acts next [#who-acts-next] The Fix list names an owner for every item. The same role model applies across stages: | Owner | Acts when | Typical recovery | | -------------------------------------- | ---------------------------------------------------------------------------- | ---------------------------------------------------------------------------- | | **You** (the migrator running the job) | The item needs a repair only a person can design | Implement the fix, register the immutable artifact, request validation | | **An assigned teammate** | The item needs a specialist — a stored-procedure redesign, a schema decision | Take the assignment; the item stays on their list until evidence is admitted | | **The system** | A worker or admitted provider can produce the evidence | Starts the job once you trigger it; Airlift records progression | | **A lead or second approver** | The item needs review, a waiver, or a business decision | Reviews independently; the resolver cannot review their own case | Separation of duties is enforced, not suggested: a resolver cannot review the same case, and a bounded Harness agent may triage a discrepancy but can never accept, resolve, verify, waive, or certify it. Nothing in the Fix list gives an agent an alternative mutation path. ## Recovery actions in the App [#recovery-actions-in-the-app] Each failure type has one primary recovery action. The App never asks you to guess it: | Failure | Recovery action | Evidence that clears it | | ------------------------------------------- | ------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- | | Conversion residue | Implement the repair, register the immutable artifact, run validation, request independent review | Repaired artifact digest plus a passing validation run and a distinct reviewer | | Validation discrepancy | Triage, then accept with a bounded waiver, or resolve and verify | A disposition by the owning role plus a verifying re-run | | Blocked or unmatched deployment requirement | Correct the requirement or re-observe the Runway release | A verified observation matching digest, environment, operation, and terminal state | | Stale or expired evidence | Refresh the evidence from its source | A new admitted observation bound to the same scope | Each action invokes the same governed Platform action the CLI invokes. A UI click never advances the ledger by itself, and a reopened item keeps its full history. ## The honesty rule: a failed refresh never renders fresh [#the-honesty-rule-a-failed-refresh-never-renders-fresh] When evidence is refreshed from a provider and the refresh **fails or returns only part of the picture**, the App keeps showing the last known state as what it is: stale, with the blocked reason attached. A failed or partial evidence refresh never renders as "fresh". You always see one of three honest states: * **Observed** — admitted evidence bound to this exact scope and digest. * **Stale or blocked** — the last admitted state, plus the reason the refresh could not complete and who acts next. * **Not yet observed** — no evidence admitted; a zero means nothing was recorded, not that something succeeded. Configuration is never evidence. A connected-looking setup, a green provider console, or a closed external ticket does not change what the App renders until Airlift admits the matching observation. ## The first path never asks you to paste a digest [#the-first-path-never-asks-you-to-paste-a-digest] Humans repair things; they do not transcribe hashes. The App derives every digest, idempotency key, and scope binding from governed records. The primary recovery path is always a button on the item itself — register this artifact, request this validation, assign this owner. Copying a digest out of one system and pasting it into a form is the advanced fallback, never the first path, and the evidence drawer offers copy-to-clipboard when an integration genuinely needs the value. ## Use the CLI with the same truth [#use-the-cli-with-the-same-truth] The App and `fa` read the same governed projections, so the Fix list is identical from either surface: ```bash fa residue list --engagement-id --json fa residue show --json fa residue resolve --file residue-resolution.json --idempotency-key residue-resolution- --json fa discrepancy list --engagement-id --json fa discrepancy triage --file discrepancy-triage.json --idempotency-key discrepancy-triage- --json ``` Use the returned owner, blocked reason, and next action; do not compute a second readiness model in automation. See [remediation and residue](/docs/migration/residue) for the full case lifecycle and [validation and readiness](/docs/migration/validation) for the discrepancy disposition path. ## AI assistance boundary [#ai-assistance-boundary] An assistant may explain a Fix list item in more conversational language, summarize the blocked reason, or link the deeper developer article. It reads the same deterministic guidance card you do and must never invent a different next action, approve a disposition, waive a check, certify an object, or close an item without the owning governed Platform action and its required evidence. # Databricks application-kit commands # Databricks application-kit commands [#databricks-application-kit-commands] Use `fa application-kit` when a migration or Databricks-native modernization should produce an operational application, not only tables, pipelines, and dashboards. Airlift turns the opportunity into a typed delivery and evidence contract. It does not generate a success claim from a checklist and it does not deploy the release. An application kit has two layers: * a shared foundation: Databricks Apps, Lakebase, Unity Catalog, portable runtime bindings, an immutable release intent, an isolated preview, and a restore rehearsal; * one or more domain modules: schemas, grants, synthetic fixtures, BDD scenarios, and module-specific evidence requirements. ## List available modules [#list-available-modules] ```bash fa application-kit module list fa application-kit module list --json ``` | Module | Typical application behavior | | ------------------------- | --------------------------------------------------------------------------------- | | `customer_intelligence` | customer profiles, consent, governed features, and service workflows | | `agent_operations` | agent memory, checkpoints, review queues, tool audit, and evaluation handoff | | `risk_compliance` | alerts, cases, policy retrieval, separation of duties, and decision evidence | | `intelligent_operations` | assets, work orders, operational scores, alerts, and maintenance workflows | | `migration_control_plane` | migration scope, conversion, validation, release references, and cutover evidence | ## Initialize a manifest [#initialize-a-manifest] ```bash fa application-kit init \ --name "Customer operations" \ --module customer_intelligence \ --cloud azure \ > application-kit.json ``` `--cloud` accepts `aws`, `azure`, or `gcp`. The generated manifest uses logical resource names and opaque references. It never contains a workspace ID, database connection string, token, or secret value. Every Databricks App runs with a dedicated service principal whose grants are resolved per environment. The generated foundation includes: ```json { "foundation": { "databricksApp": true, "lakebase": true, "unityCatalog": true, "declarativeAutomationBundle": true, "dedicatedServicePrincipal": true, "runtimeBindings": [ { "name": "application-state", "resourceType": "lakebase", "permission": "write", "required": true, "source": "valueFrom" }, { "name": "governed-data", "resourceType": "unity_catalog_schema", "permission": "read", "required": true, "source": "valueFrom" } ], "syncedTables": [], "preview": { "isolatedBranch": true, "restoreRequired": true, "rehearsalRequired": true } } } ``` Add Synced Table contracts when application serving needs governed lakehouse data in Lakebase. Each row declares a direction, opaque source and target references, business keys, and a freshness SLO. Airlift records the requirement; the admitted Databricks adapter and release own materialization. ## Inspect and plan [#inspect-and-plan] ```bash fa application-kit validate --file application-kit.json fa application-kit inspect --file application-kit.json fa application-kit plan \ --file application-kit.json \ --json > application-plan.json ``` The plan is deterministic. It expands each module into required BDD and assurance scenarios and produces explicit handoffs: * Fabric Platform: governed application mutations and audit; * Fabric Harness: bounded agent execution and Databricks transport; * Fabric Experiments: BDD, A/B, performance, quality, and agent/model evaluations; * Fabric Runway: bundle validation, preview, deployment, promotion, and rollback; * Fabric Radar: SLO and operational evidence; * Fabric Tower: work references only; and * Fabric Airlift: application-modernization scope, requirements, and evidence decision. The plan contains `deploymentCli: "fr"`. There is deliberately no `fa application-kit deploy` command. ## Qualify evidence [#qualify-evidence] Without evidence, only the contract can be proven: ```bash fa application-kit qualify \ --file application-plan.json \ --level contract_only ``` Hermetic qualification requires a passing bundle-validation reference, secret scan, synthetic-fixture digest, BDD contract, and every required runtime binding: ```bash fa application-kit qualify \ --file application-plan.json \ --evidence application-evidence.json \ --level hermetic_proven ``` Workspace qualification additionally requires: * a successful Runway deployment for the qualified artifact digest; * a passing Experiments execution for that same artifact digest; * an isolated branch preview and tested restore reference; and * a healthy Radar observation when the plan declares an operational SLO. ```bash fa application-kit qualify \ --file application-plan.json \ --evidence application-evidence.json \ --level workspace_proven \ --json > application-qualification.json ``` Digest drift, failed scenarios, missing bindings, or missing required operational evidence blocks. Airlift stores only provider references and digests; it does not copy the provider's release, test, or monitoring state. ## Register the immutable plan [#register-the-immutable-plan] ```bash fa application-kit register \ --file application-plan.json \ --engagement-id \ --estate-id \ --artifact-id \ --idempotency-key ``` Registration invokes governed `airlift.artifact_register` with media type `application/vnd.fabric.airlift.application-kit+json`. Store the plan body in an admitted immutable artifact location; Airlift records its reference, digest, producer, and audit event. Query only registered application-kit artifacts without filtering the general artifact ledger yourself: ```bash fa application-kit list --engagement-id fa application-kit show --json ``` `module list` is the only module-discovery spelling. Generate the complete current command surface with `fa commands --json`. ## Execute through the owning products [#execute-through-the-owning-products] ```bash fr validate --dir generated fr deploy --dir generated --environment preview fx apply experiments/ fx report ``` Use Runway for deploy, promote, and rollback. Use Experiments for executable BDD, A/B, performance, and quality evidence. Application-kit qualification is a read-only Airlift decision over those foreign results. See [Operational application modernization](/docs/migration/operational-applications) for the complete developer lifecycle and proof model. # Enterprise application-pack commands # Enterprise application-pack commands [#enterprise-application-pack-commands] `fa application-pack` is for migrations where the unit of scope is a business object or data product, not a source table. The executable generation supports SAP Business Data Cloud and Dynamics 365 / Dataverse. The compiler carries these requirements with each entity: * business and alternate keys; * entity relationships and cardinality; * snapshot, delta cursor, delete, and restart behavior; * effective dating and history; * currency, unit, and hierarchy meaning; * field-level and row-level authorization; * privacy classifications; * business control totals; and * downstream consumer transitions. ## Create a credential-free manifest [#create-a-credential-free-manifest] ```json { "schemaVersion": 1, "source": { "system": "dynamics_365_dataverse", "productVersion": "current", "snapshotAt": "2030-01-15T12:00:00.000Z", "connectionRef": "databricks-connection://migration/dataverse" }, "estate": { "name": "Customer service" }, "entities": [ { "entityId": "account", "name": "Account", "kind": "business_object", "keys": ["accountid"], "relationships": [], "changeBehavior": { "mode": "managed_connector", "cursorField": "versionnumber", "deleteMode": "tombstone", "effectiveDating": false }, "semantics": { "currencyFields": ["revenue"], "unitFields": [], "hierarchyFields": ["parentaccountid"], "controlTotals": ["active_account_count"] }, "authorization": { "fieldSecurity": false, "rowSecurity": false, "privacyClassifications": ["customer"] }, "consumers": ["customer-360"], "sourceFragment": { "logicalName": "account" }, "provenance": { "sourceRef": "dataverse://metadata/account", "extractor": "your-dataverse-exporter", "extractorVersion": "1.0.0", "observedAt": "2030-01-15T12:00:00.000Z" }, "unsupportedReasons": [] } ] } ``` `connectionRef` is an opaque reference. The source adapter or managed connector owns authentication and ingestion. Airlift does not broker OAuth or store source tokens. ## Inspect and plan [#inspect-and-plan] ```bash fa application-pack inspect --file application-manifest.json fa application-pack plan \ --file application-manifest.json \ --json > application-plan.json ``` The plan routes each entity to `deterministic`, `agent_repairable`, or `human_only` and retains the exact source fragment and provenance digest. It defines the managed-source checkpoint contract, Experiments validation requirements, Runway deployment handoff, business-owner acceptance, consumer transitions, and remediation acceptance criteria. Connector completion is never business-semantic proof. An entity remains blocked when delete behavior, effective dating, currencies, units, hierarchies, authorization, control totals, or consumers are unresolved. ## Register [#register] ```bash fa application-pack register \ --file application-plan.json \ --engagement-id \ --estate-id \ --artifact-id \ --idempotency-key ``` This invokes governed `airlift.artifact_register`. The artifact body remains in your admitted store; Airlift records the reference, digest, producer generation, and media type. SAP S/4HANA, ECC, BW/4HANA, HANA, Datasphere, other Dynamics profiles, Salesforce, Workday, and ServiceNow remain cataloged profiles. They are not executable application packs until their source contracts and recurring evidence ship. # Capability commands # Capability commands [#capability-commands] `fa capability` operates the tenant-scoped capability registry. It does not install a converter, deploy an application, or certify client objects. ## Query [#query] ```bash fa capability list fa capability list --source snowflake --variant snowflake_enterprise fa capability list --source redshift --capability incremental_cdc --json fa capability list --source synapse --construct merge_statement --artifact-kind source fa capability show cap_ --json fa capability matrix --source synapse --variant synapse_dedicated_sql ``` `matrix` overlays active evidence on the installed source profile and prints each capability's delivery mode, proof level, provider, and provider version. Missing cells are reported as `unproved`. Construct filters select registry-v2 cells. Text matrix output lists these cells separately and states that they do not aggregate into variant proof. ## Mutate through governed actions [#mutate-through-governed-actions] All lifecycle mutations use a JSON parameter file and a stable idempotency-key prefix: ```bash fa capability propose --file proposal.json --idempotency-key proposal-v1 fa capability evidence --file evidence.json --idempotency-key evidence-v1 fa capability promote --file promotion.json --idempotency-key promotion-v1 fa capability expire --file expiry.json --idempotency-key expiry-v1 fa capability revoke --file revocation.json --idempotency-key revocation-v1 fa capability reconcile --file reconciliation.json --idempotency-key reconcile-v1 ``` The API derives organization and identity from the authenticated principal. Parameter files cannot select another organization or supply an approving actor. | Command | Required authority | Result | | ----------- | --------------------------------- | -------------------------------------------------- | | `propose` | human operator | proposed source/capability/provider generation | | `evidence` | admitted system | immutable proof reference and digest | | `promote` | different human reviewer | active claim at or below proven strength | | `expire` | admitted system or human operator | inactive time-bounded claim | | `revoke` | human reviewer | explicitly withdrawn claim | | `reconcile` | admitted system | matched observation or fail-closed expiry on drift | See [Capability registry](/docs/sources/capability-registry) for schemas and the proof ladder. ## Diagnose a source or engagement [#diagnose-a-source-or-engagement] ```bash fa source doctor snowflake --variant snowflake --level executable fa source limitations redshift --variant redshift_serverless fa source certify synapse --variant synapse_dedicated_sql --level certifiable fa engagement preflight eng_ ``` `doctor` and `certify` query active, unexpired tenant evidence and exit with status `1` when blocked. Their text output includes a reason and next action for each blocker; `--json` returns the stable diagnosis contract for CI. `source certify` checks whether provider capabilities are strong enough for the requested outcome; it does not mint a migration certificate. Migration certificates are still minted only from object-specific admitted validation evidence through `fa certificate mint`. An engagement preflight derives its required level from the selected services: | Engagement service | Capability target | | -------------------------- | ------------------------------------------- | | `discovery` | `assessable` | | `factory_pilot` | `executable` | | `migration_factory` | `certifiable` | | `managed_migration_office` | `certifiable` | | `cutover_assurance` | `cutover_certified` | | `modernization` | adds an evidenced modernization requirement | # Catalog commands # Catalog commands [#catalog-commands] The CLI exposes two different catalogs: * `fa sources` describes source-system migration routes and common workload areas; * `fa profiles` describes object-type evidence requirements used to mint a certificate. Do not substitute one for the other. A source profile helps construct the project; a validation profile is enforced by certificate policy. ## Actions [#actions] ```bash fa actions fa actions --json ``` The command reads the installed action definitions and prints action ID, version, required permissions, policies, and emitted events. Use it to detect documentation or deployment drift; it does not invoke an action. ## Validation profiles [#validation-profiles] ```bash fa profiles fa profiles --json ``` The command prints built-in profile ID/version, object type, canonical SHA-256 digest, and required readiness tracks. The digest is computed from canonical JSON using the same function as certificate minting. Catalog JSON is deterministic and suitable for release manifests or policy review. See [source planning commands](/docs/cli/sources) for the source catalog. # Certificate commands # Certificate commands [#certificate-commands] ## Inspect [#inspect] ```bash fa certificate inspect migration-certificate.json ``` Inspection validates the envelope schema and reports certificate, organization, estate, object, wave, profile, issuer, key ID, and computed unsigned-envelope digest. It shows whether the digest matches the signed digest but does not claim signature validity. ## Verify [#verify] Create a JSON file that maps key IDs to Ed25519 public PEM values: ```json { "airlift-migration-cert-v1": "-----BEGIN PUBLIC KEY-----\n...\n-----END PUBLIC KEY-----\n" } ``` Then verify: ```bash fa certificate verify migration-certificate.json --keys public-keys.json ``` Without `--keys`, verification reads `AIRLIFT_EVIDENCE_VERIFY_KEYS_JSON`. The command recomputes canonical content, requires the signed digest to match, selects the declared key ID, and verifies the Ed25519 signature. Malformed input, tampering, unknown keys, and invalid signatures return non-zero. # Command reference # Command reference [#command-reference] The examples assume `fa` resolves to the installed CLI. Run `fa help` to print the same command inventory from the installed generation. ## Global automation options [#global-automation-options] `--format text|table|json|yaml|jsonl` may appear on any command that produces structured data. `--json` remains an alias for `--format json`. JSONL emits one item per line when the result is an array. Human-readable text or tables remain the default. Use `--file -` to read a JSON manifest or mutation request from stdin. Use `--correlation-id ` to propagate a trace identifier into a governed mutation and `--timeout ` to bound remote calls. Neither option can override identity, organization, policy, or approvals. Run `fa help ` for focused help, `fa commands --json` for the machine-readable command contract, and `fa completion bash|zsh|fish` for shell completion. The [generated command index](/docs/cli/generated-command-index) is built from that same manifest. Remote commands require `AIRLIFT_API_URL` and `DATABRICKS_TOKEN`. Mutation commands also require a stable, non-secret `--idempotency-key` prefix. See [authenticated automation](/docs/cli/remote-automation) for identity and retry semantics. Modernization and measurement commands are documented together in [Modernization and value commands](/docs/cli/modernization-and-value). Source capability commands and their evidence-authority split are documented in [Capability commands](/docs/cli/capabilities). The executable Synapse manifest compiler and governed registration flow are documented in [Synapse golden-path commands](/docs/cli/synapse). Executable SQL Server, Snowflake, Redshift, Oracle, Teradata, and Hadoop manifest compilation, governed registration, and certification are documented in [migration-pack commands](/docs/cli/migration-packs). Native ADF import and file generation, plus normalized-manifest routing for SSIS, PowerCenter, SAS, DataStage, Talend, ODI, dbt, and Airflow, are documented in [pipeline import and generation](/docs/cli/migration-ir). Business-semantic SAP BDC and Dataverse compilation is documented in [enterprise application-pack commands](/docs/cli/application-packs). Existing Databricks estate modernization is documented in [Databricks-native commands](/docs/cli/databricks-native). Portable Databricks Apps and Lakebase foundation/module planning, same-digest evidence qualification, and the `fa` versus `fr` boundary are documented in [application-kit commands](/docs/cli/application-kits). Qualification, reusable delivery kits, connector policies, external value claims, activation evidence, and public-content checks are documented in [delivery and claim commands](/docs/cli/delivery). Deployment-access inspection and admitted preflight recording (`fa access`) are documented in [Databricks access and integrations](/docs/integrations/databricks-access). Evaluation-candidate freeze, rehearsal, and window inspection (`fa evaluator`) are documented in [Evaluator readiness](/docs/integrations/evaluator-readiness). Unity Catalog evidence recording (`fa uc-evidence`) is documented in [Unity Catalog evidence](/docs/integrations/uc-evidence). Organization provisioning and retirement are platform-installer-only actions in the `airlift-platform` tenant; no tenant CLI command exists. See the [local lifecycle cookbook](/docs/getting-started/local-lifecycle-cookbook). Parity certification is recorded by the admitted worker through `airlift.parity_certify`; no caller-authored parity command exists. See [Parity certificates](/docs/parity-certificates). Waiver requests and approvals are App decision-card surfaces; the CLI deliberately exposes no waiver subcommand. See [Waivers and staleness](/docs/operations/waivers-and-staleness). ## `fa organization membership set` [#fa-organization-membership-set] ```bash fa organization membership set \ --file membership-changes.json \ --idempotency-key membership-grant-2026-08 ``` Apply governed organization membership grants and revocations through `airlift.organization_membership_set`. The file holds one tenant-free `changes` array; the authenticated API derives the organization from the caller's membership. Requires the `admin` role (`airlift:membership:manage`) and refuses to leave an organization without an active admin. See [Organization membership](/docs/reference/organization-membership). ## `fa engagement list|show` [#fa-engagement-listshow] ```bash fa engagement list fa engagement show eng_01ARZ3NDEKTSV4RRFFQ69G5FAV --json ``` Query engagements in the organization derived from the authenticated Databricks principal. There is no organization selection flag. ## `fa engagement preflight` [#fa-engagement-preflight] ```bash fa engagement preflight eng_01ARZ3NDEKTSV4RRFFQ69G5FAV fa engagement preflight eng_01ARZ3NDEKTSV4RRFFQ69G5FAV --json ``` Evaluate every scoped source variant against the evidence level implied by the engagement services. Exit status `1` means the engagement is blocked. The result includes missing or stale claims, next actions, governed human lanes, and the requested support level. ## `fa engagement create|update` [#fa-engagement-createupdate] ```bash fa engagement create \ --file engagement.json \ --idempotency-key project-42-create fa engagement update \ --file engagement-update.json \ --idempotency-key project-42-scope-v2 ``` The create file accepts `name`, one or more `services`, `owner`, and optional target dates, estate IDs, and structured external references. The update file includes `engagementId` plus at least one changed field. The API validates referenced estates and rejects edits after scope is frozen. ## `fa engagement activate|freeze` [#fa-engagement-activatefreeze] ```bash fa engagement activate eng_01ARZ3NDEKTSV4RRFFQ69G5FAV \ --idempotency-key project-42-activate fa engagement freeze eng_01ARZ3NDEKTSV4RRFFQ69G5FAV \ --idempotency-key project-42-freeze ``` Activation moves a draft scope to active. Freeze computes and records the canonical scope digest. A frozen engagement rejects later edits. ## `fa connection list|register` [#fa-connection-listregister] ```bash fa connection list --engagement-id eng_01ARZ3NDEKTSV4RRFFQ69G5FAV fa connection register \ --file source-binding.json \ --idempotency-key project-42-synapse-binding ``` The registration file contains the engagement ID, display name, direction, capabilities, and an opaque `credentialRef` such as `databricks-connection://migration/synapse-metadata`. It never contains a password, token, private key, or connection string. ## `fa connection verify|retire` [#fa-connection-verifyretire] ```bash fa connection verify bnd_01ARZ3NDEKTSV4RRFFQ69G5FAV \ --digest aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa \ --idempotency-key project-42-synapse-verify fa connection retire bnd_01ARZ3NDEKTSV4RRFFQ69G5FAV \ --reason "Credential rotated" \ --idempotency-key project-42-synapse-retire ``` Verification records an admitted SHA-256 evidence digest and authenticated verifier. Retirement is terminal for that binding; register a new binding for a rotated reference. ## `fa estate list|show|register` [#fa-estate-listshowregister] ```bash fa estate list --json fa estate show est_01ARZ3NDEKTSV4RRFFQ69G5FAV fa estate register --file estate.json --idempotency-key estate-synapse-1 ``` Register a source estate or query the tenant-scoped estate projection. Registration accepts a source profile ID, owner, environment, priority, and optional opaque connection reference. ## `fa assessment list|status|start|record|accept|export` [#fa-assessment-liststatusstartrecordacceptexport] ```bash fa assessment list --estate-id est_01ARZ3NDEKTSV4RRFFQ69G5FAV fa assessment status asm_01ARZ3NDEKTSV4RRFFQ69G5FAV --json fa assessment start --file assessment-start.json --idempotency-key assess-start-1 fa assessment record --file assessment-record.json --idempotency-key assess-record-1 fa assessment accept --file assessment-accept.json --idempotency-key assess-accept-1 fa assessment export --file assessment-export.json --idempotency-key assess-export-1 ``` Start and record compose the configured worker/source-adapter lifecycle. Acceptance is a human decision that binds normalized inventory and accepted dependency digests. Export returns a content-digested pack containing the accepted assessment, inventory, dependencies, and applicable plan scenarios. ## `fa inventory list|register` [#fa-inventory-listregister] ```bash fa inventory list --estate-id est_01ARZ3NDEKTSV4RRFFQ69G5FAV \ --assessment-id asm_01ARZ3NDEKTSV4RRFFQ69G5FAV --json fa inventory register --file object.json --idempotency-key object-sales-1 ``` Inventory registration normalizes one source object into the governed migration ledger. Use repeated idempotent calls for objects emitted by an assessment adapter. Filter by `--assessment-id` whenever an estate has multiple snapshots so results cannot be mixed. ## `fa inventory graph-*` [#fa-inventory-graph-] ```bash fa inventory graph list --assessment-id asm_01ARZ3NDEKTSV4RRFFQ69G5FAV fa inventory graph show dpg_01ARZ3NDEKTSV4RRFFQ69G5FAV --json fa inventory graph start --file graph-start.json --idempotency-key graph-start-1 fa inventory graph record --file graph-batch.json --idempotency-key graph-batch-1 fa inventory graph accept --file graph-accept.json --idempotency-key graph-accept-1 ``` One graph declares its expected edge count. Each record call accepts at most 500 edges. A human operator accepts the complete graph and Airlift computes its digest and cycle count. ## `fa plan list|show|compare|generate|select|freeze` [#fa-plan-listshowcomparegenerateselectfreeze] ```bash fa plan list --engagement-id eng_01ARZ3NDEKTSV4RRFFQ69G5FAV fa plan compare --engagement-id eng_01ARZ3NDEKTSV4RRFFQ69G5FAV --json fa plan show pln_01ARZ3NDEKTSV4RRFFQ69G5FAV fa plan generate --file plan.json --idempotency-key plan-parity-1 fa plan select pln_01ARZ3NDEKTSV4RRFFQ69G5FAV --idempotency-key plan-select-1 fa plan freeze pln_01ARZ3NDEKTSV4RRFFQ69G5FAV --idempotency-key plan-freeze-1 ``` Generation maps accepted objects to Databricks target patterns, topologically groups candidate waves, detects cycles, and calculates transparent effort/value estimates. Freeze requires a selected scenario, frozen engagement scope, and no blocking issues. ## `fa wave list` [#fa-wave-list] ```bash fa wave list fa wave list --estate-id est_01ARZ3NDEKTSV4RRFFQ69G5FAV --json ``` Query materialized execution waves. Plan candidates remain part of the frozen plan until the operator materializes execution waves through governed wave actions. ## `fa cutover` and `fa hypercare` [#fa-cutover-and-fa-hypercare] ```bash fa cutover list fa cutover status wav_01J00000000000000000000000 --json fa cutover freeze --file wave-freeze.json --idempotency-key wave-freeze-v1 fa cutover runbook --file runbook.json --idempotency-key runbook-v1 fa cutover rehearse --file rehearsal.json --idempotency-key rehearsal-v1 fa cutover observe --file observation.json --idempotency-key parallel-run-v1 fa cutover certify-effector --file effector.json --idempotency-key effector-v1 fa cutover approve wav_01J00000000000000000000000 --idempotency-key approval-1 fa cutover start wav_01J00000000000000000000000 --window 2030-09-14T02:00Z --reason "Approved production window" fa cutover workflow-status airlift-v2-wav_ fa hypercare start --file start.json --idempotency-key hypercare-v1 fa hypercare observe --file observation.json --idempotency-key hypercare-observe-1 fa hypercare complete --file decision.json --idempotency-key hypercare-decision-1 fa hypercare decommission --file disposition.json --idempotency-key disposition-1 ``` These commands operate Airlift-owned migration cutover policy and its durable workflow. They do not deploy or promote release artifacts and do not create monitoring state. See the complete [cutover command reference](/docs/cli/cutover). ## `fa artifact list|show|register` [#fa-artifact-listshowregister] ```bash fa artifact list --engagement-id eng_01J00000000000000000000000 fa artifact list --object-id obj_01J00000000000000000000000 --json fa artifact show art_01J00000000000000000000000 fa artifact register --file artifact.json --idempotency-key artifact-v1 ``` Query or register immutable artifact references. Registration requires the artifact store reference, SHA-256 digest, media type, producer generation, and engagement/object lineage. Artifact bodies and credentials are never uploaded to Airlift. ## `fa conversion list|show|diff` [#fa-conversion-listshowdiff] ```bash fa conversion list --object-id obj_01J00000000000000000000000 fa conversion show cnv_01J00000000000000000000000 --json fa conversion diff obj_01J00000000000000000000000 --json ``` Read the conversion-attempt ledger. `diff` returns the same object's ordered attempt records for comparing method, tool/model/prompt generation, content digests, diagnostic, and validation reference. Fetch code bodies from the artifact store. ## `fa conversion batch-*` [#fa-conversion-batch-] ```bash fa conversion batch list --engagement-id eng_01J00000000000000000000000 fa conversion batch show cbh_01J00000000000000000000000 --json fa conversion batch create --file batch.json --idempotency-key batch-1 fa conversion batch start --file batch-start.json --idempotency-key batch-1-start fa conversion batch complete --file batch-start.json --idempotency-key batch-1-complete ``` Create an accepted-scope batch, admit worker attempts, and reconcile terminal results. Completion derives counts and blocks on missing attempts, missing artifact lineage, or failed objects without residue. ## `fa conversion attempt-*|retry` [#fa-conversion-attempt-retry] ```bash fa conversion attempt start --file attempt-start.json --idempotency-key object-1-start fa conversion attempt record --file attempt-result.json --idempotency-key object-1-result fa conversion retry --file retry-start.json --idempotency-key object-1-retry-2 ``` `retry` is an explicit alias for starting another conversion attempt. Use a new stable key and an eligible `rework` object. Airlift preserves prior attempts. ## `fa residue list|show|create|estimate|assign|resolve|review|cancel` [#fa-residue-listshowcreateestimateassignresolvereviewcancel] ```bash fa residue list --engagement-id eng_01J00000000000000000000000 fa residue show res_01J00000000000000000000000 --json fa residue create --file residue.json --idempotency-key residue-1 fa residue estimate --file estimate.json --idempotency-key residue-1-estimate fa residue assign --file assignment.json --idempotency-key residue-1-assign fa residue resolve --file resolution.json --idempotency-key residue-1-resolve fa residue review --file review.json --idempotency-key residue-1-review fa residue cancel --file cancellation.json --idempotency-key residue-1-cancel ``` Operate the remediation lifecycle. Assignment works directly from an `open` case and, like review, requires natural-person authority. `estimate` is optional delivery-planning metadata; it is not required before assignment. Resolution requires an artifact from the same object plus a validation evidence reference. The resolver cannot review the same case. In the App, cancellation is disclosed separately and requires a reason plus explicit confirmation. ## `fa transfer list|status|plan|run|checkpoint|pause|resume|reconcile|reconcile-record|fail|cancel` [#fa-transfer-liststatusplanruncheckpointpauseresumereconcilereconcile-recordfailcancel] ```bash fa transfer list --engagement-id eng_01J00000000000000000000000 fa transfer status xfr_01J00000000000000000000000 --json fa transfer plan --file transfer-plan.json --idempotency-key transfer-plan-v1 fa transfer run xfr_01J00000000000000000000000 --idempotency-key transfer-run-v1 fa transfer pause xfr_01J00000000000000000000000 --reason "Source maintenance" --idempotency-key transfer-pause-1 fa transfer resume xfr_01J00000000000000000000000 --idempotency-key transfer-resume-1 fa transfer reconcile xfr_01J00000000000000000000000 --idempotency-key transfer-reconcile-1 ``` `plan`, `run`, `pause`, `resume`, `reconcile`, and `cancel` are operator controls. `checkpoint`, `reconcile-record`, and `fail` submit runner evidence and require admitted automation; a natural-person invocation is rejected. See [data transfer and reconciliation](/docs/migration/transfer) for input schemas and the Temporal execution boundary. ## `fa deployment list|status|require` [#fa-deployment-liststatusrequire] ```bash fa deployment list --engagement-id eng_01J00000000000000000000000 fa deployment status dpr_01J00000000000000000000000 --json fa deployment require --file deployment-requirement.json --idempotency-key wave-3-release ``` `require` declares the Runway operation, environment, immutable artifacts, and terminal state needed by the migration. Airlift computes the desired digest. The admitted Runway integration records observations and reconciliation; caller-authored `observe`, `sync`, and `reconcile` commands do not exist. Use `fr` for deployment execution. See [Deploy with Fabric Runway](/docs/integrations/runway). ## `fa migration-ir qualify` [#fa-migration-ir-qualify] ```bash fa migration-ir qualify \ --file generated/artifact-set.json \ --root generated \ --bindings release-bindings.json \ --resolutions residue-resolutions.json \ --proof hermetic_proven \ --output qualification.json ``` The command verifies generated bytes, invokes Databricks bundle validation, checks runtime binding and residue evidence, and optionally evaluates matching Runway and Experiments workspace evidence. It is read-only and never writes a provider verdict to the Airlift ledger. See [release qualification](/docs/migration/release-qualification). ## `fa validation list|status|runs|readiness|run|cancel` [#fa-validation-liststatusrunsreadinessruncancel] ```bash fa validation run --file validation-request.json --idempotency-key wave-2-validation-v3 fa validation list --engagement-id "$ENGAGEMENT_ID" fa validation status "$VALIDATION_EXECUTION_ID" --json fa validation runs --object-id "$OBJECT_ID" --json fa validation readiness --object-id "$OBJECT_ID" --json fa validation cancel "$VALIDATION_EXECUTION_ID" --reason "Superseded suite" --idempotency-key validation-cancel-v1 ``` `run` creates Airlift validation scope; the durable worker delegates the generated suite to Experiments. `runs` shows admitted provider runs. `readiness` shows derived track observations. Cancellation is governed and leaves external provider state subject to reconciliation. ## `fa discrepancy list|show|create|triage|accept|resolve|verify` [#fa-discrepancy-listshowcreatetriageacceptresolveverify] ```bash fa discrepancy list --object-id "$OBJECT_ID" fa discrepancy show "$DISCREPANCY_ID" --json fa discrepancy create --file discrepancy.json --idempotency-key discrepancy-42-create fa discrepancy triage --file triage.json --idempotency-key discrepancy-42-triage fa discrepancy accept --file triage-acceptance.json --idempotency-key discrepancy-42-accept fa discrepancy resolve --file resolution.json --idempotency-key discrepancy-42-resolution fa discrepancy verify --file verification.json --idempotency-key discrepancy-42-verify ``` Human operators can triage, accept eligible low/medium differences, and submit remediation. `create` and `verify` are admitted automation operations. Verification takes a `validationRunId` and succeeds only for a passing Experiments run belonging to the same object and matching the resolution evidence identity. ## `fa certificate list|show|mint|invalidate` [#fa-certificate-listshowmintinvalidate] ```bash fa certificate list --object-id "$OBJECT_ID" fa certificate show "$CERTIFICATE_ID" --json fa certificate mint --file certificate-mint.json --idempotency-key object-42-mint-v1 fa certificate invalidate --file certificate-invalidate.json --idempotency-key object-42-stale-v1 ``` List and show are authenticated reads. Mint and invalidate call the system-owned certificate actions and normally require the admitted worker principal. They do not replace offline `certificate inspect` and `certificate verify`. ## `fa evidence export` [#fa-evidence-export] ```bash fa evidence export --file evidence-export.json --idempotency-key wave-2-evidence-v1 ``` Creates a content-digested governed evidence pack for an object or wave. The export contains references and policy evidence, not artifact bodies or credentials. ## `fa help` [#fa-help] ```bash fa help fa --help fa -h ``` Print every command and the CLI mutation boundary. Exit `0`. ## `fa version` [#fa-version] ```bash fa version fa --version fa -V ``` Print the installed CLI version. Exit `0`. ## `fa docs [topic]` [#fa-docs-topic] ```bash fa docs fa docs sources/sql-server fa docs migration/validation --json ``` Print `https://airlift.fabric.pro/docs`, optionally with the URL-encoded topic appended. This command prints a URL; it does not launch a browser. ## `fa sources` [#fa-sources] ```bash fa sources fa sources --json ``` List the installed source profiles with archetype, evidence-derived level, implementation routing (non-evidence), variants, applicable Lakebridge capabilities, program track, and migration-area metadata. ## `fa sources export --plans` [#fa-sources-export---plans] ```bash fa sources export --plans --format jsonl fa sources export --plans --json ``` Export one canonical plan record per source × variant in the installed registry. Each record wraps a compiled plan with `sourceSystem`, `sourceVariant`, `implementationRoutingLevel`, `externalGates`, a `docsUrl`, and a `contentDigest` computed over the plan alone, so unchanged plans keep the same digest across CLI releases for incremental re-indexing. The default text output emits the same JSONL form as `--format jsonl`; `--json` emits a single canonical array instead. `--plans` is required; omitting it returns exit `2`. ## `fa source inspect ` [#fa-source-inspect-source] ```bash fa source inspect sql_server fa source inspect mssql --json fa source inspect teradata ``` Resolve a profile ID or alias and print its variants, archetype, implementation routing (non-evidence), adapter contract, workload surfaces, applicable Lakebridge routing, transfer strategy, validation checks, residue, and modernization targets. Unknown sources fail with exit `1`. Run `fa sources --json` for canonical IDs and variants. ## `fa source plan ` [#fa-source-plan-source] ```bash fa source plan synapse fa source plan dynamics_365 --variant dynamics_365_finance_operations --json ``` Generate the deterministic archetype-aware plan. Each phase contains specialist adapter commands or boundaries, exact governed Airlift action IDs, outputs, and exit criteria. `--variant` selects a registered specialization. There is no legacy schema selector. Generating a plan does not create governed state. ## `fa source certification-check ` [#fa-source-certification-check-file] ```bash fa source certification-check source-certification.json fa source certification-check source-certification.json --json ``` Validate an immutable live-run manifest and report evidence missing for its requested support level. Eligible returns `0`; missing required evidence returns `1`. The command does not promote the registry or mint a certificate. ## `fa doctor` [#fa-doctor] ```bash fa doctor fa doctor --profile local fa doctor --profile production --json ``` Check Node, persistence, authorization, evidence registry, signing and verification, Databricks forwarded identity, Lakebridge job binding, Temporal mode, and the cutover effector boundary. Production profile failures return exit `1`; warnings do not. `doctor` validates configuration shape. It cannot prove connectivity, validate client data, or certify a cutover effector. ## `fa actions` [#fa-actions] ```bash fa actions fa actions --json ``` Print every governed action with its version, required permissions, policies, and emitted events. The command reads the installed contract; it never invokes an action. ## `fa profiles` [#fa-profiles] ```bash fa profiles fa profiles --json ``` Print built-in object validation profiles, their canonical SHA-256 digests, and required readiness tracks. These are certificate profiles, not source-system profiles. ## `fa certificate inspect ` [#fa-certificate-inspect-file] ```bash fa certificate inspect migration-certificate.json fa certificate inspect migration-certificate.json --json ``` Parse the certificate schema and show its identity, profile, issuer, key, signed digest, computed digest, and digest match. Inspection does not verify the signature. ## `fa certificate verify ` [#fa-certificate-verify-file] ```bash fa certificate verify migration-certificate.json --keys public-keys.json AIRLIFT_EVIDENCE_PUBLIC_KEYS_JSON='{"airlift-prod":"...PEM..."}' \ fa certificate verify migration-certificate.json ``` Verify both envelope digest and Ed25519 signature. `--keys` accepts a JSON object mapping key IDs to public PEM strings. Without it, the command reads Airlift's verifier configuration from the environment. Valid returns `0`; invalid returns `1`. ## Usage failures [#usage-failures] Unknown commands, missing required arguments, or unsupported subcommands print a concise message plus `Run 'fa help' for usage.` and return exit `2`. Remote authentication returns `3`, not found returns `4`, blocked/conflict returns `5`, and dependency outage returns `6`. Malformed local files, unknown source profiles, and failed diagnostics return `1`. # Cutover and hypercare commands # Cutover and hypercare commands [#cutover-and-hypercare-commands] `fa cutover` manages migration cutover truth and starts Airlift's durable workflow. `fa deployment` does not deploy releases; use the Runway CLI for release execution and then admit the resulting immutable reference through an Airlift deployment requirement. Set the authenticated API boundary once: ```bash export AIRLIFT_API_URL=https:// export DATABRICKS_TOKEN= ``` The API derives organization and identity from authentication. No cutover command accepts an actor or organization override. ## Inspect control state [#inspect-control-state] ```bash fa cutover list --estate-id est_01J00000000000000000000000 fa cutover status wav_01J00000000000000000000000 --json ``` The response includes the frozen digest, runbook, rehearsal, operational evidence, effector certification, incidents, hypercare, source disposition, and any stale reason. ## Freeze exact scope [#freeze-exact-scope] ```json title="wave-freeze.json" { "engagementId": "eng_01J00000000000000000000000", "waveId": "wav_01J00000000000000000000000", "transfers": { "applicability": "required", "ids": ["xfr_01J00000000000000000000000"] }, "deployments": { "applicability": "required", "ids": ["dpr_01J00000000000000000000000"] }, "releaseEvidenceRefs": [ { "system": "runway", "type": "promoted_release", "id": "release-v42", "digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" } ], "reason": "Freeze the certified production scope for the approved window." } ``` ```bash fa cutover freeze --file wave-freeze.json --idempotency-key wave-freeze-v1 ``` For a genuinely inapplicable transfer or deployment track, use `{"applicability":"not_applicable","reason":"..."}`. Airlift requires a reason; an empty list cannot silently bypass the track. ## Configure and rehearse the runbook [#configure-and-rehearse-the-runbook] ```json title="runbook.json" { "waveId": "wav_01J00000000000000000000000", "version": "2026.09.14-1", "reason": "Pin the approved timed procedure and restore path.", "steps": [ { "stepId": "checkpoint", "name": "Create routing checkpoint", "ownerRole": "cutover_operator", "offsetMinutes": -15, "expectedDurationMinutes": 5, "actionKind": "governed_action", "actionRef": "effector:checkpoint" }, { "stepId": "switch", "name": "Apply consumer routing once", "ownerRole": "cutover_operator", "offsetMinutes": 0, "expectedDurationMinutes": 10, "actionKind": "governed_action", "actionRef": "effector:apply", "rollbackStepId": "restore" }, { "stepId": "restore", "name": "Restore the checkpoint", "ownerRole": "cutover_operator", "offsetMinutes": 15, "expectedDurationMinutes": 10, "actionKind": "governed_action", "actionRef": "effector:compensate" } ] } ``` ```bash fa cutover runbook --file runbook.json --idempotency-key runbook-v1 fa cutover rehearse --file rehearsal-result.json --idempotency-key rehearsal-v1 ``` `rehearse` requires an admitted automation principal and immutable evidence reference. A human cannot author a passing rehearsal verdict. ## Admit operational evidence [#admit-operational-evidence] ```json title="parallel-run.json" { "waveId": "wav_01J00000000000000000000000", "phase": "parallel_run", "providerRef": { "system": "radar", "type": "observation_window", "id": "window-42", "digest": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" }, "evidenceDigest": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", "sloProfileId": "airlift.cutover.production_slo.v1", "observedFrom": "2030-09-13T00:00:00.000Z", "observedUntil": "2030-09-14T00:00:00.000Z", "verdict": "passed" } ``` ```bash fa cutover observe --file parallel-run.json --idempotency-key parallel-run-v1 ``` Only Radar or an admitted external monitor may provide operational evidence. Airlift retains the foreign reference, digest, window, SLO profile, and verdict; the monitor definition and raw observations remain with the provider. ## Certify the effector and approve the wave [#certify-the-effector-and-approve-the-wave] ```bash fa cutover certify-effector \ --file effector-certification.json \ --idempotency-key effector-cert-v1 fa cutover approve wav_01J00000000000000000000000 \ --note "Change authority approval" \ --idempotency-key wave-approval-1 ``` Certification requires a passing rehearsal and a natural-person certifier. The file identifies the implementation and proves `checkpoint`, `apply_once`, `verify`, and `compensate`. Approval is a separate governed action and follows separation of duties. ## Start the durable workflow [#start-the-durable-workflow] ```bash fa cutover start wav_01J00000000000000000000000 \ --window 2030-09-14T02:00Z \ --reason "Approved customer change window" fa cutover workflow-status airlift-v2-wav_ fa cutover wake airlift-v2-wav_ ``` `start` calls the dedicated workflow endpoint. It does not invoke `airlift.cutover_execute` from the CLI process. The worker re-reads governed readiness, waits durably for approvals, performs checkpoint/apply-once/verify, and records the outcome through Platform actions. ## Incidents and hypercare [#incidents-and-hypercare] ```bash fa cutover incident-open --file incident.json --idempotency-key incident-1 fa cutover incident-resolve --file incident-resolution.json --idempotency-key incident-1-resolve fa hypercare start --file hypercare-start.json --idempotency-key hypercare-v1 fa hypercare observe --file hypercare-observation.json --idempotency-key hypercare-observe-1 fa hypercare complete --file hypercare-decision.json --idempotency-key hypercare-accept-1 fa hypercare decommission --file source-disposition.json --idempotency-key source-disposition-1 ``` An open incident blocks readiness or hypercare acceptance. Resolution requires immutable evidence. Hypercare completion and source disposition are human decisions with separation-of-duties checks; agents and monitors cannot accept or decommission. ## Exit behavior [#exit-behavior] | Code | Meaning | | ---- | ----------------------------------------------------- | | `0` | request completed or attached to an existing workflow | | `2` | invalid command arguments | | `4` | authentication failed | | `5` | authorization failed | | `6` | dependency unavailable or invalid API response | | `7` | governed readiness conflict | # Databricks-native commands # Databricks-native commands [#databricks-native-commands] Use `fa databricks-native` when the customer already runs Databricks. This is not a pretend migration from an external system. The source estate is `databricks`, and the program starts from an immutable current-state baseline. ## Register the source profile [#register-the-source-profile] ```bash fa source inspect databricks fa source plan databricks --variant databricks_unity_catalog ``` Available variants are `databricks_unity_catalog`, `databricks_workspace_consolidation`, and `databricks_cost_performance`. ## Create the native manifest [#create-the-native-manifest] ```json { "schemaVersion": 1, "estate": { "name": "Analytics platform", "variant": "databricks_unity_catalog", "workspaceRefs": ["workspace://analytics-prod"], "snapshotAt": "2030-01-15T12:00:00.000Z" }, "reason": "unity_catalog_upgrade", "baselineEvidenceRef": "evidence://experiments/native-baseline-v1", "assets": [ { "assetId": "sales-database", "name": "Sales Hive database", "kind": "hive_metastore_object", "dependencies": [], "owner": "sales-data", "currentStateDigest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "issues": ["Uses workspace-local storage paths."], "metrics": { "tables": 42 } } ] } ``` Inventory may cover workspaces, metastores, Hive Metastore objects, Unity Catalog assets, jobs, pipelines, notebooks, warehouses, clusters, policies, permissions, dashboards, models, endpoints, shares, lineage, cost observations, and operational dependencies. Store identities, digests, aggregate metrics, and issues—not credentials or raw client data. Typed reasons include Unity Catalog upgrades, workspace consolidation, permission and path modernization, serverless or Lakeflow adoption, Delta and liquid clustering, runtime upgrades, dashboard transitions, and cost/performance optimization. ## Inspect and plan [#inspect-and-plan] ```bash fa databricks-native inspect --file native-manifest.json fa databricks-native plan \ --file native-manifest.json \ --json > native-plan.json ``` Every asset becomes a deterministic, advisory, or human recommendation. The plan requires baseline behavior, allow-and-deny permission tests, performance and cost guardrails, immutable release and rollback evidence, an operational observation window, and owner acceptance. ## Register [#register] ```bash fa databricks-native register \ --file native-plan.json \ --engagement-id \ --estate-id \ --artifact-id \ --idempotency-key ``` The plan records the family execution contract: * Airlift owns the baseline, modernization intent, recommendations, evidence references, promotion policy, and value record. * Harness may produce bounded advice or repair candidates. * Experiments owns A/B, BDD, parity, performance, and cost evaluations. * Runway owns preview, deployment, promotion, reconciliation, and rollback. * Radar owns operational observations. * Tower owns work references. * Platform owns governed mutations and audit. Use `fa` to inspect Airlift migration/modernization status and request governed changes. Use `fr` to execute releases. Airlift may block on Runway or Radar evidence; it never copies their state or provides a second deploy command. # Delivery and claim commands # Delivery and claim commands [#delivery-and-claim-commands] `fa delivery` makes the migration method repeatable without turning internal delivery state into public documentation or unsupported claims. These commands are local, deterministic checks; governed engagement mutations still use the normal authenticated commands. ## Qualify a source engagement [#qualify-a-source-engagement] Create a credential-reference-only qualification file with source and target authority, representative-data approval, object count, required/excluded surfaces, network shape, RPO/RTO, and cutover/rollback owners. ```bash fa delivery preflight --file qualification.json ``` The command returns assessment and cutover-design readiness separately. Missing source authority, target authority, or representative-data approval blocks assessment. Missing cutover or rollback owners blocks cutover design. Private-link and offline shapes add a connectivity rehearsal instead of pretending online dependencies are available. ## Generate the delivery kit [#generate-the-delivery-kit] ```bash fa delivery kit \ --file qualification.json \ --json > delivery-kit.json ``` The generated kit contains contracts for: * source qualification; * assessment report; * target blueprint; * wave plan; * human-residue estimate; * migration certificate; * cutover and rollback runbook; and * measured value report. It also lists factory setup, client adapter, human remediation, cutover assurance, modernization, and hypercare playbooks plus backup/restore, signer rotation, connector rotation, retention/export, disaster recovery, and upgrade/rollback operations. ## Validate a connector operating envelope [#validate-a-connector-operating-envelope] ```bash fa delivery connector-policy-check --file connector-policy.json ``` A promoted connector policy must bound concurrency, rate, quota, cost, timeout, retry classes, and circuit breaking. Checkpoint and reconciliation are mandatory. The schema fixes non-idempotent effects at one attempt. ## Gate an external value claim [#gate-an-external-value-claim] ```bash fa delivery claim-check --file external-claim.json ``` Eligibility requires at least three engagements, at least ten comparable observations, medium or high confidence, observed methods beyond expert estimates alone, a versioned cohort and release generation, exclusions, limitations, a reviewed methodology, and an immutable evidence digest. A rejected packet exits nonzero and emits exact blockers. This is stricter than an engagement-scoped client value report. It exists for aggregate external language. Never hard-code “60% faster” into product material. ## Check activation evidence [#check-activation-evidence] ```bash fa delivery activation-check --file activation-evidence.json ``` Requirements are data, not hard-coded program assumptions. Record each credential category's current required and observed counts, approved outcome references, thought-leadership references, repeatable demo evidence, reviewer, and evidence digest. Repository tests do not count as customer outcomes. ## Scan public developer content [#scan-public-developer-content] ```bash fa delivery public-check --file public-document.md ``` The scanner blocks known private path, deployment, evidence-ledger, credential, and client-identity patterns. It complements review and CI; it does not declassify a document or replace client approval. # fa doctor # `fa doctor` [#fa-doctor] ```bash fa doctor --profile local fa doctor --profile production --json ``` Doctor checks Node generation, store selection and binding shape, authorization directory, evidence registry, Ed25519 signer/verifier configuration, Databricks Apps identity gates, Lakebridge job configuration, Temporal selection, and the client-effector boundary. It also checks the observation-only Runway API binding and the Experiments case-module/evidence-store binding used by release qualification. Production profile fails when durable store, authorization, identity, signing, live Lakebridge, Runway observation, or Experiments validation structure is missing. Temporal absence and the composition-injected cutover effector remain explicit warnings because connectivity and client certification cannot be proven from environment shape. Doctor prints variable names, key IDs, counts, and dispositions. It does not print connection strings, private keys, tokens, authorization-directory contents, or other secret values. A passing shape check is not deployment or client certification evidence. # Live evidence commands # Live evidence commands [#live-evidence-commands] Airlift admits immutable artifact references into the migration ledger. It does not turn a JSON file into a validation verdict, deployment result, or cutover authorization. Fabric Experiments, Runway, Radar, and client-approved adapters retain ownership of their results; Airlift stores their identifiers, content digests, producer generations, and lineage. ## 1. Inspect before admission [#1-inspect-before-admission] ```bash fa evidence inspect live-evidence.json fa evidence inspect live-evidence.json --json > .airlift/evidence-inspection.json ``` Inspection is local. It validates the secret-free schema, timestamp order, source identity, run references, artifact digests, and disallowed credential-like keys. A historical failed run remains valid evidence—it is reported as failed rather than erased. ## 2. Admit artifact references [#2-admit-artifact-references] ```bash fa evidence admit \ --file live-evidence.json \ --engagement-id \ --estate-id \ --idempotency-key source-workspace-run-v1 \ --json > .airlift/evidence-admission.json ``` The command invokes the governed `airlift.artifact_register` action once for the manifest and once for each referenced artifact. It records no file bodies and no credentials. The JSON result includes `providerVerdictsAdmitted: false`; provider evidence must still enter through its admitted principal and policy path before readiness can pass. Artifact admission cannot manufacture an Experiments verdict. ## 3. Inspect the admitted ledger [#3-inspect-the-admitted-ledger] ```bash fa evidence list --engagement-id fa evidence show --json fa artifact list --engagement-id ``` In the Databricks App, open **Engagements → active engagement → Artifacts** for immutable references and **Runs** for assessment, conversion, transfer, deployment, and validation execution references. ## 4. Export governed evidence [#4-export-governed-evidence] ```bash fa evidence export \ --file evidence-export.json \ --idempotency-key wave-evidence-export-v1 ``` An evidence export is a content-digested view of the governed event trail and current projection. The digest proves byte identity; only a valid signing envelope proves signer identity. Retain the export with the application build, source snapshot, provider runs, certificates, approvals, and observation windows that supported the decision. ## Automation outcomes [#automation-outcomes] | Exit | Meaning | | ---- | ---------------------------------------------------------------- | | `0` | schema inspection, admission, query, or export request succeeded | | `1` | the governed action or policy rejected the request | | `2` | command usage or the local manifest is invalid | | `4` | the requested remote resource was not found | | `6` | authentication, transport, or service availability failed | # Generated command index # Generated command index [#generated-command-index] This page is generated from Fabric Airlift CLI 0.18.4. The same manifest drives `fa help`, group help, shell completion, tests, and this reference. It describes developer commands only; it does not include credentials, tenant data, deployment state, or internal delivery notes. Use resource-first paths: `fa `. For a nested resource, continue the path: `fa inventory graph list`, `fa conversion batch create`, and `fa application-kit module list`. There are no deprecated spellings or hidden aliases. Global automation options: * `--format text|table|json|yaml|jsonl`; `--json` is the concise JSON form; * `--file -` reads a JSON request or manifest from stdin; * `--timeout ` bounds remote requests and explicit waits; and * `--correlation-id ` carries a caller trace ID into governed mutations. Generate completion with `fa completion bash`, `fa completion zsh`, or `fa completion fish`. Emit this command model for tooling with `fa commands --json`. ## `fa organization` [#fa-organization] Govern the projected organization membership registry. | Command | Behavior | | ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------ | | `fa organization membership set --file --idempotency-key ` | Apply governed membership grants and revocations (admin role; last-admin protected). | ## `fa engagement` [#fa-engagement] Create and govern migration engagements. | Command | Behavior | | --------------------------------------------------------------------------- | ----------------------------------------------------------- | | `fa engagement list` | List engagements visible to the authenticated principal. | | `fa engagement show ` | Show one engagement. | | `fa engagement status ` | Show the derived eight-phase migration status and blockers. | | `fa engagement create\|update --file --idempotency-key ` | Create or update an engagement. | | `fa engagement activate\|freeze --idempotency-key ` | Advance the engagement lifecycle. | | `fa engagement preflight ` | Evaluate source-capability and human-lane blockers. | ## `fa connection` [#fa-connection] Register source connection references without storing credentials. | Command | Behavior | | -------------------------------------------------------------------------- | ---------------------------------------------------------------------------- | | `fa connection list [--engagement-id ]` | List connection bindings. | | `fa connection register --file --idempotency-key ` | Register a secret-backed connection reference. | | `fa connection verify --digest --idempotency-key ` | Verify a binding with the digest of a recorded connectivity diagnostic. | | `fa connection retire --reason --idempotency-key ` | Retire a connection binding. | | `fa connection test [--json]` | Report observed connectivity state for a binding; exit 0 only when observed. | | `fa connection diagnose --file --idempotency-key ` | Record an admitted connectivity diagnostic (system principal). | ## `fa access` [#fa-access] Inspect derived Databricks deployment access and record admitted access preflights. | Command | Behavior | | ------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------ | | `fa access show [--json]` | Report the derived access-preflight state and five target-integration states; exit 0 only when access is observed. | | `fa access record --file --idempotency-key ` | Record an admitted access preflight (system principal). | ## `fa evaluator` [#fa-evaluator] Inspect derived evaluation readiness and record admitted candidate freezes and rehearsals. | Command | Behavior | | ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------- | | `fa evaluator window [--json]` | Report derived evaluation readiness and the evaluation-window manifest; exit 0 only when evaluation\_ready is true. | | `fa evaluator freeze --file --idempotency-key ` | Freeze an evaluation candidate (admitted system principal; in-process path). | | `fa evaluator rehearsal --file --idempotency-key ` | Record an evaluator rehearsal against the frozen candidate (admitted system principal; in-process path). | ## `fa uc-evidence` [#fa-uc-evidence] Inspect the derived Databricks-native evidence state and record admitted projection runs and reconciliations. | Command | Behavior | | ---------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- | | `fa uc-evidence show [--engagement-id ] [--json]` | Report the derived UC evidence state; exit 0 only when the evidence is current. | | `fa uc-evidence record --kind projection-run\|reconciliation --file --idempotency-key ` | Record an admitted UC projection run or target reconciliation (admitted system principal; in-process path). | ## `fa estate` [#fa-estate] Register and inspect source estates. | Command | Behavior | | ----------------------------------------------------------------- | ------------------- | | `fa estate list` | List estates. | | `fa estate show ` | Show an estate. | | `fa estate register --file --idempotency-key ` | Register an estate. | ## `fa assessment` [#fa-assessment] Run and accept governed source assessments. | Command | Behavior | | ------------------------------------------------------------------------------------------ | -------------------------------- | | `fa assessment list [--estate-id ]` | List assessments. | | `fa assessment status ` | Show assessment status. | | `fa assessment start\|record\|accept\|export --file --idempotency-key ` | Mutate the assessment lifecycle. | ## `fa inventory` [#fa-inventory] Register inventory and build dependency graphs. | Command | Behavior | | --------------------------------------------------------------------------------------- | ------------------------------------ | | `fa inventory list [--estate-id ] [--assessment-id ]` | List migration objects. | | `fa inventory register --file --idempotency-key ` | Register inventory. | | `fa inventory graph list [--assessment-id ]` | List dependency graphs. | | `fa inventory graph show ` | Show graph edges. | | `fa inventory graph start\|record\|accept --file --idempotency-key ` | Build and accept a dependency graph. | ## `fa plan` [#fa-plan] Generate, compare, select, and freeze migration plans. | Command | Behavior | | --------------------------------------------------------------- | ------------------------ | | `fa plan list\|compare [--engagement-id ]` | List or compare plans. | | `fa plan show ` | Show a migration plan. | | `fa plan generate --file --idempotency-key ` | Generate a plan. | | `fa plan select\|freeze --idempotency-key ` | Select or freeze a plan. | ## `fa wave` [#fa-wave] Inspect dependency-aware migration waves. | Command | Behavior | | --------------------------------- | ------------------- | | `fa wave list [--estate-id ]` | List cutover waves. | ## `fa conversion` [#fa-conversion] Run conversion batches and govern remediation residue. | Command | Behavior | | ------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------- | | `fa conversion list [--object-id ]` | List conversion attempts with their hazard-scan state (clean, hazardous, or unassessed). | | `fa conversion show ` | Show a conversion attempt; hazardous attempts block certification until their conversion\_hazard residue is reviewed. | | `fa conversion diff ` | Compare attempts for an object. | | `fa conversion batch list [--engagement-id ]` | List conversion batches. | | `fa conversion batch show ` | Show a conversion batch. | | `fa conversion batch create\|start\|complete --file --idempotency-key ` | Mutate a conversion batch. | | `fa conversion attempt start\|record --file --idempotency-key ` | Start or record a conversion attempt. | | `fa conversion retry --file --idempotency-key ` | Retry through the governed conversion action. | ## `fa residue` [#fa-residue] Estimate, assign, and resolve human or agent remediation. | Command | Behavior | | ----------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | | `fa residue list [--engagement-id ] [--object-id ]` | List remediation residue. | | `fa residue show ` | Show one residue item; conversion\_hazard cases name their exact origin conversion and assessment digest. | | `fa residue create\|estimate\|assign\|resolve\|review\|cancel --file --idempotency-key ` | Mutate the residue lifecycle. After an approved review, retry certification with a FRESH idempotency key — a denied key stays denied. | ## `fa transfer` [#fa-transfer] Plan, operate, and reconcile data movement. | Command | Behavior | | --------------------------------------------------------------------------------------------------- | ----------------------------------- | | `fa transfer list [--engagement-id ] [--estate-id ]` | List transfer plans. | | `fa transfer status ` | Show transfer status. | | `fa transfer plan\|checkpoint\|reconcile-record\|fail --file --idempotency-key ` | Record transfer plans and evidence. | | `fa transfer run\|resume\|reconcile --idempotency-key ` | Operate a transfer workflow. | | `fa transfer pause\|cancel --reason --idempotency-key ` | Pause or cancel a transfer. | ## `fa validation` [#fa-validation] Run Experiments-backed parity validation and inspect readiness. | Command | Behavior | | ------------------------------------------------------------------- | ---------------------------------------- | | `fa validation list [--engagement-id ] [--estate-id ]` | List validation executions. | | `fa validation status ` | Show validation status. | | `fa validation runs\|readiness [--object-id ]` | Inspect admitted evidence and readiness. | | `fa validation run --file --idempotency-key ` | Request validation. | | `fa validation cancel --reason --idempotency-key ` | Cancel validation. | ## `fa certificate` [#fa-certificate] Inspect, verify, mint, and invalidate migration certificates. | Command | Behavior | | ------------------------------------------------------------------------------ | --------------------------------------- | | `fa certificate list [--object-id ]` | List certificates. | | `fa certificate show ` | Show a governed certificate record. | | `fa certificate inspect ` | Inspect a certificate envelope locally. | | `fa certificate verify [--keys ]` | Verify digest and signature locally. | | `fa certificate mint\|invalidate --file --idempotency-key ` | Mutate governed certificate state. | ## `fa cutover` [#fa-cutover] Freeze, approve, execute, and observe governed cutover. | Command | Behavior | | -------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------- | | `fa cutover list [--estate-id ]` | List cutover controls. | | `fa cutover status ` | Show cutover readiness. | | `fa cutover freeze\|runbook\|rehearse\|observe\|certify-effector --file --idempotency-key ` | Record cutover control evidence. | | `fa cutover incident-open\|incident-resolve --file --idempotency-key ` | Govern a cutover incident. | | `fa cutover approve [--note ] --idempotency-key ` | Approve through the authenticated mutation boundary. | | `fa cutover start --window --reason [--wait] [--timeout ]` | Start or attach to the durable cutover workflow. | | `fa cutover workflow-status\|wake ` | Query or wake a durable cutover workflow. | ## `fa hypercare` [#fa-hypercare] Record post-cutover observation and decommission evidence. | Command | Behavior | | -------------------------------------------------------------------------------------------------- | ------------------------------- | | `fa hypercare start\|observe\|complete\|decommission --file --idempotency-key ` | Mutate the hypercare lifecycle. | ## `fa artifact` [#fa-artifact] Register and inspect content-digested migration artifacts. | Command | Behavior | | ------------------------------------------------------------------- | ----------------------------------------- | | `fa artifact list [--engagement-id ] [--object-id ]` | List artifacts. | | `fa artifact show ` | Show one artifact. | | `fa artifact register --file --idempotency-key ` | Register an immutable artifact reference. | ## `fa deployment` [#fa-deployment] Declare and inspect deployment requirements; Runway executes releases. | Command | Behavior | | ------------------------------------------------------------------------- | ----------------------------------------------------------------- | | `fa deployment list [--engagement-id ] [--estate-id ]` | List deployment requirements. | | `fa deployment status ` | Show a deployment requirement. | | `fa deployment require --file --idempotency-key ` | Declare a required Runway outcome. | | `fa deployment connect --file --idempotency-key ` | Connect an existing requirement to a Runway deployment reference. | ## `fa modernization` [#fa-modernization] Separate baseline migration from measurable Databricks modernization. | Command | Behavior | | ------------------------------------------------------------------------------------------------------------------------- | ---------------------------- | | `fa modernization list [--engagement-id ] [--object-id ]` | List modernization items. | | `fa modernization show ` | Show one modernization item. | | `fa modernization recommend\|decide\|plan\|start\|evidence\|promote\|rework --file --idempotency-key ` | Mutate modernization state. | ## `fa value` [#fa-value] Record baselines and publish measurable outcome reports. | Command | Behavior | | ---------------------------------------------------------------------------------- | ----------------------- | | `fa value observations\|summaries\|reports [--engagement-id ]` | Inspect value evidence. | | `fa value record\|summarize\|publish --file --idempotency-key ` | Mutate value evidence. | ## `fa capability` [#fa-capability] Inspect and govern source-capability evidence. | Command | Behavior | | -------------------------------------------------------------------------------------------------------------------------------- | ----------------------------- | | `fa capability list [--source ] [--variant ] [--capability ] [--construct ] [--artifact-kind ]` | List capability entries. | | `fa capability show ` | Show one capability entry. | | `fa capability matrix --source [--variant ]` | Render the capability matrix. | | `fa capability propose\|evidence\|promote\|expire\|revoke\|reconcile --file --idempotency-key ` | Mutate capability evidence. | ## `fa source` [#fa-source] Inspect, plan, diagnose, and qualify supported source profiles. | Command | Behavior | | ------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------- | | `fa source inspect ` | Inspect one source profile. | | `fa source plan [--variant ]` | Create a source migration plan. | | `fa source recipe [--variant ]` | Render the complete developer command and artifact sequence for one source. | | `fa source constructs [--variant ] [--construct ] [--artifact-kind ]` | List repository-owned construct routing without implying provider proof. | | `fa source doctor\|certify [--variant ] [--level ]` | Diagnose or certify capability evidence. | | `fa source limitations [--variant ]` | List explicit source limitations. | | `fa source certification-check ` | Evaluate source certification evidence. | ## `fa migration-pack` [#fa-migration-pack] Compile and qualify source-specific migration packs. | Command | Behavior | | ----------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------- | | `fa migration-pack inspect\|plan --file ` | Inspect or compile a migration pack. | | `fa migration-pack register --file --engagement-id --estate-id --artifact-id --idempotency-key ` | Register a compiled pack. | | `fa migration-pack certification-check --file ` | Evaluate pack certification evidence. | ## `fa migration-ir` [#fa-migration-ir] Import, compile, generate, and qualify ETL migration IR. | Command | Behavior | | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------- | | `fa migration-ir import --source --file --estate-name --snapshot-at [--product-version ] [--credential-map ] [--output ]` | Import a supported source export. | | `fa migration-ir inspect\|compile --file ` | Inspect or compile migration IR. | | `fa migration-ir generate --file --out-dir [--artifact-set ] [--engagement-id --estate-id --artifact-ref-prefix ]` | Generate Databricks artifacts. | | `fa migration-ir materialize --file --out-dir ` | Materialize generated files. | | `fa migration-ir validate --file [--root ]` | Validate generated files. | | `fa migration-ir qualify --file --root [--proof ] [--bindings ] [--resolutions ] [--workspace-evidence ] [--output ] [--databricks-cli ]` | Qualify a release candidate. | | `fa migration-ir register --file --engagement-id --estate-id --artifact-id --idempotency-key ` | Register migration IR. | ## `fa lakebase` [#fa-lakebase] Plan and qualify SQL Server migrations to Databricks Lakebase. | Command | Behavior | | --------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------- | | `fa lakebase inspect\|plan --file ` | Inspect or compile a Lakebase target plan. | | `fa lakebase register --file --engagement-id --estate-id --artifact-id --idempotency-key ` | Register the immutable plan with an engagement. | | `fa lakebase qualification-check --file ` | Evaluate digest-bound compatibility or managed Lakebase evidence. | ## `fa application-pack` [#fa-application-pack] Compile enterprise application migration packs. | Command | Behavior | | ------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------- | | `fa application-pack inspect\|plan --file ` | Inspect or compile an application pack. | | `fa application-pack register --file --engagement-id --estate-id --artifact-id --idempotency-key ` | Register an application pack. | ## `fa application-kit` [#fa-application-kit] Plan and qualify Databricks operational applications. | Command | Behavior | | ---------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | | `fa application-kit module list` | List supported application modules. | | `fa application-kit init --name --module [--cloud aws\|azure\|gcp]` | Create an application-kit manifest. | | `fa application-kit validate --file ` | Validate a manifest without compiling it. | | `fa application-kit inspect\|plan --file ` | Inspect or compile a portable plan. | | `fa application-kit qualify --file [--evidence ] [--level ]` | Evaluate qualification evidence. | | `fa application-kit list [--engagement-id ]` | List registered application-kit artifacts. | | `fa application-kit show ` | Show a registered application-kit artifact. | | `fa application-kit register --file --engagement-id --estate-id --artifact-id --idempotency-key ` | Register a plan reference. | ## `fa databricks-native` [#fa-databricks-native] Plan modernization for estates already on Databricks. | Command | Behavior | | -------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------- | | `fa databricks-native inspect\|plan --file ` | Inspect or compile a native modernization plan. | | `fa databricks-native register --file --engagement-id --estate-id --artifact-id --idempotency-key ` | Register a native plan. | ## `fa delivery` [#fa-delivery] Build delivery kits and validate external claims. | Command | Behavior | | ------------------------------------------------------------------ | -------------------------------------------------- | | `fa delivery preflight\|kit --file ` | Evaluate or package delivery readiness. | | `fa delivery connector-policy-check --file ` | Validate a connector execution policy. | | `fa delivery claim-check\|activation-check --file ` | Evaluate an external claim or activation evidence. | | `fa delivery public-check --file ` | Reject internal-only language in public material. | ## `fa synapse` [#fa-synapse] Run the Synapse golden-path helpers. | Command | Behavior | | ---------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------- | | `fa synapse inspect\|plan --file ` | Inspect or compile a Synapse plan. | | `fa synapse compile-adf --file ` | Compile an ADF export. | | `fa synapse register --file --engagement-id --estate-id --artifact-id --idempotency-key ` | Register a Synapse plan. | | `fa synapse certification-check --file ` | Evaluate Synapse certification evidence. | ## `fa discrepancy` [#fa-discrepancy] Triage and resolve validation discrepancies. | Command | Behavior | | ----------------------------------------------------------------------------------------------------- | ------------------------- | | `fa discrepancy list [--engagement-id ] [--estate-id ] [--object-id ]` | List discrepancies. | | `fa discrepancy show ` | Show a discrepancy. | | `fa discrepancy create\|triage\|accept\|resolve\|verify --file --idempotency-key ` | Mutate discrepancy state. | ## `fa evidence` [#fa-evidence] Export content-digested migration evidence. | Command | Behavior | | -------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- | | `fa evidence inspect ` | Inspect a secret-free live-evidence manifest locally. | | `fa evidence admit --file --engagement-id --estate-id --idempotency-key ` | Register verified immutable artifact references without manufacturing provider verdicts. | | `fa evidence list [--engagement-id ]` | List admitted evidence artifacts. | | `fa evidence show ` | Show one admitted evidence artifact. | | `fa evidence export --file --idempotency-key ` | Request an evidence export. | ## Utilities [#utilities] | Command | Behavior | | | | ----------------------------- | ---------------------------------------------------- | ---------------------------------------- | -------------------------- | | `fa help []` | Show global or group help. | | | | \`fa --help | -h\` | Show global help. | | | `fa actions` | List governed action contracts. | | | | `fa profiles` | List validation profiles. | | | | `fa sources` | List source profiles. | | | | `fa sources export --plans` | Export one canonical plan record per source variant. | | | | \`fa doctor \[--profile local | production]\` | Check local or production configuration. | | | `fa commands` | Emit the machine-readable command manifest. | | | | \`fa completion bash | zsh | fish\` | Generate shell completion. | | `fa docs []` | Print the documentation URL. | | | | \`fa version | --version | -V\` | Print the CLI version. | ## Ownership boundary [#ownership-boundary] `fa deployment require/list/status` governs the deployment outcome required by a migration. Fabric Runway's `fr` command executes deployment, promotion, rollback, and release reconciliation. Airlift may block on a verified Runway reference; it never re-owns Runway state. # Automation (Airlift CLI) # Automation with the Airlift CLI [#automation-with-the-airlift-cli] Humans run the guided migration journey in the Databricks App. This section is the **Automation** surface: `@fabricorg/airlift-cli` installs one executable, `fa`, for scripts and CI. The name follows the Fabric family convention: Harness uses `fh`, Runway uses `fr`, and Airlift uses `fa`. Use `fa` to operate a migration or modernization engagement: assess scope, plan waves, convert code, transfer data, evaluate readiness, and inspect evidence. It is not a general Databricks deployment CLI. Fabric Runway's `fr` command owns release deployment, promotion, rollback, and reconciliation. Authenticated remote commands use the same governed Airlift actions as the Databricks App; local catalog and verification commands never mutate engagement state. Every mutation below names the governed action it invokes — there is no second mutation path. ## Common automation recipes [#common-automation-recipes] Each recipe names its governed action and permission. All remote mutations take a stable, non-secret `--idempotency-key` prefix: the CLI appends a canonical digest of the action and parameters, so an exact retry collapses to the original invocation and changed content cannot reuse the old content-bound key. Persist the prefix with your pipeline job — never mint a fresh timestamp or random value per retry — and use a fresh key after a policy denial, because a denied key stays denied. See [authenticated automation](/docs/cli/remote-automation) for identity, tenant derivation, and the full retry contract. ### Create an engagement [#create-an-engagement] ```bash fa engagement create --file engagement.json --idempotency-key project-42-engagement ``` Invokes `airlift.engagement_create` (`airlift:engagement:create`). Admitted for the service-principal `automation` role, so CI can onboard engagements unattended. ### Register a connection binding [#register-a-connection-binding] ```bash fa connection register --file source-binding.json --idempotency-key project-42-binding ``` Invokes `airlift.connection_binding_register` (`airlift:connection:register`); automation-admitted. The file carries an opaque secret reference, never credentials. A binding becomes `verified` only against a recorded connectivity diagnostic — follow the digest recipe in [authenticated automation](/docs/cli/remote-automation). ### Accept an assessment [#accept-an-assessment] ```bash fa assessment accept --file assessment-accept.json --idempotency-key project-42-assess-accept-1 ``` Invokes `airlift.assessment_accept` (`airlift:assessment:accept`). Acceptance is a human decision: it requires the `operator` role on an authenticated person. The `automation` service-principal role is denied — it may start, record, and export assessments but never accept scope. ### Freeze a plan [#freeze-a-plan] ```bash fa plan freeze pln_01ARZ3NDEKTSV4RRFFQ69G5FAV --idempotency-key project-42-plan-freeze-1 ``` Invokes `airlift.plan_freeze` (`airlift:plan:freeze`); human `operator` role, denied to `automation`. Freeze requires a selected scenario, frozen engagement scope, and no blocking issues. ### Export evidence [#export-evidence] ```bash fa evidence export --file evidence-export.json --idempotency-key project-42-wave-2-evidence ``` Invokes `airlift.evidence_export` (`airlift:evidence:export`; human `operator`, `approver`, or `validator` role). The export is a content-digested governed pack of references and policy evidence — never artifact bodies or credentials. ## Install [#install] ```bash npm install --global @fabricorg/airlift-cli fa help ``` For an ephemeral or CI-pinned invocation, use `npx`: ```bash npx --yes --package @fabricorg/airlift-cli@0.18.4 fa sources --format json ``` ## What can I do with it? [#what-can-i-do-with-it] | Task | Command | | ------------------------------------------------------------- | -------------------------------------------------------------------------------- | | list migration engagements in your authenticated organization | `fa engagement list` | | create an engagement from a versioned JSON input | `fa engagement create --file engagement.json --idempotency-key onboarding-42` | | activate or freeze its scope | `fa engagement activate --idempotency-key activate-42` | | register an opaque source connection reference | `fa connection register --file source-binding.json --idempotency-key binding-42` | | verify or retire a connection binding | `fa connection verify --digest --idempotency-key verify-42` | | create and reconcile a scoped conversion batch | `fa conversion batch create --file batch.json --idempotency-key batch-42` | | inspect attempts for an object | `fa conversion diff --json` | | register an immutable converted artifact | `fa artifact register --file artifact.json --idempotency-key artifact-42` | | estimate, assign, resolve, and review engineering residue | `fa residue list --engagement-id ` | | plan, run, pause, resume, and reconcile data movement | `fa transfer status --json` | | declare and inspect a required Runway outcome | `fa deployment status --json` | | run object-specific Experiments validation | `fa validation run --file validation.json --idempotency-key validation-42` | | triage and resolve required-check failures | `fa discrepancy list --object-id ` | | inspect governed readiness and certificates | `fa certificate list --object-id ` | | list supported source profiles | `fa sources` | | import a native ADF or Synapse Pipelines export | `fa migration-ir import --source adf-synapse --file adf-export/ ...` | | generate and byte-validate Databricks implementation files | `fa migration-ir generate --file migration-ir.json --out-dir generated` | | materialize an artifact set received from CI | `fa migration-ir materialize --file artifact-set.json --out-dir generated` | | inspect one source's tools, surfaces, transfer, and residue | `fa source inspect sql_server` | | generate its complete migration plan | `fa source plan sql_server` | | check a live source-pack manifest | `fa source certification-check source-certification.json` | | check local or production configuration | `fa doctor --profile production` | | inspect governed actions and permissions | `fa actions` | | inspect object validation profiles and digests | `fa profiles` | | inspect a certificate without trusting it | `fa certificate inspect certificate.json` | | cryptographically verify a certificate | `fa certificate verify certificate.json --keys keys.json` | | print a documentation URL | `fa docs sources/sql-server` | Use `--format text|table|json|yaml|jsonl` on data-producing commands. `--json` is the concise JSON form. JSON is canonical and deterministic; JSONL emits one array item per line. Both are safe to diff in CI or feed into project scaffolding: ```bash fa source plan snowflake --json > .airlift/snowflake-plan.json fa actions --json > .airlift/action-contract.json ``` Read JSON from stdin with `--file -`. Use `--correlation-id ` to connect a governed mutation to your pipeline trace, and `--timeout ` to bound remote calls. These options never change identity, organization, policy, or approval authority. The help system is resource-first: ```bash fa help inventory fa inventory graph list --assessment-id asm_ fa conversion batch create --file batch.json --idempotency-key batch-42 fa application-kit module list fa commands --json fa completion zsh ``` The CLI has no deprecated command aliases. Use the exact resource hierarchy printed by `fa commands`; obsolete flat or verb-first spellings fail with exit status `2`. See the [generated command index](/docs/cli/generated-command-index) for the exact surface shipped by the current CLI. Continue with the [complete command reference](/docs/cli/command-reference), or use [source planning commands](/docs/cli/sources) to start a migration integration. ## Airlift versus Runway [#airlift-versus-runway] Use `fa deployment require` only when a migration engagement needs a particular Runway release outcome. Airlift stores the requirement and verified foreign references; it does not execute the release. If a customer is already using Databricks and is not running a migration or modernization engagement, use Runway directly: ```bash fr catalog fr deploy fr promote ``` Runway's console and API own detailed deployment status and reconciliation. Do not create an Airlift estate solely to obtain deployment commands. ## Exit status [#exit-status] | Status | Meaning | | ------ | ------------------------------------------------------------- | | `0` | command completed or verification passed | | `1` | diagnostic, verification, or internal response failed | | `2` | usage or remote request validation failed | | `3` | authentication, organization binding, or authorization failed | | `4` | remote resource not found | | `5` | replay conflict or governed action blocked | | `6` | remote dependency or transport unavailable | | `7` | asynchronous workflow conflict or request conflict | ## Security boundary [#security-boundary] Set `AIRLIFT_API_URL` to the deployed Airlift Databricks App URL and supply a short-lived `DATABRICKS_TOKEN` through your shell or CI secret provider. The CLI never prints the token. It sends no actor or organization field: Harness authenticates the principal and the API resolves exactly one admitted organization membership. The CLI accepts no actor, tenant, approval, waiver, `--force`, or unsafe override. It never creates a production Airlift runtime or writes the durable store directly. The CLI exposes the same action contract to authorized automation, but it cannot grant itself approval, waiver, certificate, policy, or cutover authority. Platform Host still enforces the caller's admitted role and agent/system bounds. # SQL Server to Lakebase # SQL Server to Lakebase with `fa` [#sql-server-to-lakebase-with-fa] > The current release candidate admits the repeatable hermetic contract only. Historical raw > workspace reports were withdrawn from the candidate; rerun the sanitized producer and admit > its same-digest output before requesting `workspace_proven`. The `fa lakebase` command group turns an accepted SQL Server inventory into a Lakebase-specific migration plan and evaluates digest-bound execution evidence. It does not translate T-SQL, deploy a database, or mark a migration complete by itself. Use an admitted converter or human-reviewed SQL artifact for target DDL. Fabric Runway owns deployment. Airlift records the source scope, target intent, artifact digests, human work, validation requirements, and the proof boundary. ## 1. Compile the target plan [#1-compile-the-target-plan] Start with the same SQL Server manifest accepted by the migration-pack compiler: ```bash fa lakebase inspect --file sql-server-manifest.json fa lakebase plan --file sql-server-manifest.json --json \ > .airlift/sql-server-lakebase-plan.json ``` The plan contains: * source inventory and dependency digests; * a route for every SQL Server object; * PostgreSQL-compatible Lakebase target kinds; * deterministic, agent-repairable, and human-only dispositions; * explicit human redesign for SQL Agent, SSIS, SSRS, and other external workloads; * the validation stages required before stronger proof can be claimed; * a content-derived `planDigest`. The plan deliberately keeps operational database objects separate from orchestration. A SQL Agent job does not silently become a stored procedure in Lakebase; it remains a visible redesign item for Lakeflow Jobs or another approved runtime. ## 2. Register the immutable plan [#2-register-the-immutable-plan] Upload the plan to your approved artifact store, then register its reference: ```bash fa lakebase register \ --file .airlift/sql-server-lakebase-plan.json \ --engagement-id \ --estate-id \ --artifact-id volumes/migrations/sql-server/lakebase-plan.json \ --idempotency-key lakebase-plan-v1 ``` `register` invokes the governed artifact action. It records a reference and matching digest; it does not upload or deploy the file. ## 3. Run the repeatable compatibility demo [#3-run-the-repeatable-compatibility-demo] From an Airlift source distribution, run: ```bash pnpm demo:sql-server-lakebase ``` The test creates an explicitly named ephemeral PostgreSQL 18 container, applies the target schema, loads representative rows, executes validation, writes the reports, and removes the container. It verifies: 1. plan compilation; 2. schema application; 3. data loading; 4. row counts; 5. aggregate parity; 6. decimal, timestamp, and null semantics; 7. identity/sequence behavior; 8. idempotent restart/replay; 9. a negative schema-isolation assertion. The output is compatibility proof. It is not managed Lakebase proof and it is not proof of a client SQL Server estate. ## 4. Evaluate qualification evidence [#4-evaluate-qualification-evidence] ```bash fa lakebase qualification-check \ --file .airlift/sql-server-lakebase-qualification.json ``` Use `requestedProofLevel: hermetic_proven` for the repeatable PostgreSQL-compatible test. `workspace_proven` additionally requires both sides of the migration: * `source_connectivity` from a connected SQL Server engine; and * `target_connectivity` from an actual managed Databricks Lakebase workspace run. Every evidence row must carry the same build, plan, and dataset digests, including both connectivity rows and every compatibility stage. A target-only run, a PostgreSQL container standing in for SQL Server, a mismatched digest, or a missing stage blocks promotion. The workspace runner or your delivery automation writes one qualification request. Run the same public evaluator before admitting it: ```bash fa lakebase qualification-check \ --file .airlift/sql-server-lakebase-workspace-qualification.json \ --json > .airlift/sql-server-lakebase-workspace-report.json ``` The report's `boundaries.sourceConnected` must be `true`, `eligible` must be `true`, and `maximumProofLevel` must be `workspace_proven`. Those fields prove the engineering fixture only; they do not convert the run into client or production evidence. ## 5. Admit the connected workspace report [#5-admit-the-connected-workspace-report] Store the report bytes in an approved Unity Catalog Volume or another immutable artifact store. Then register only its reference and SHA-256 digest with the engagement: ```bash fa artifact register \ --file sql-server-workspace-artifact.json \ --idempotency-key sql-server-workspace-evidence-v1 ``` `sql-server-workspace-artifact.json` identifies the engagement and estate, uses `kind: "evidence"`, points `artifactRef` at the provider-owned report, repeats the exact digest in both digest fields, and records the producer generation. It never contains a database password, OAuth token, or connection string. In the App, open **Engagement → Artifacts**. Registration first shows **Workspace evidence not corroborated**. An admitted provider must record a passing governed validation run whose `subjectArtifactId` names this platform-minted artifact row and whose `artifactDigest` matches its SHA-256 digest. Only then is the row labeled **Connected workspace evidence**; reusing the digest of another passing run does not qualify. Open the row to copy the digest and provider reference, inspect whether independent validation and a Runway requirement are bound, and read the proof boundary. Airlift still will not infer client approval or production cutover authority from its file name. ## 6. Read the status in automation and the App [#6-read-the-status-in-automation-and-the-app] ```bash fa engagement status fa engagement status --json ``` Open the engagement in the Airlift App and select **Migration status**. The route reads **SQL Server → Databricks Lakebase** when the engagement contains the governed target reference `databricks:migration_target:lakebase`. The page displays all eight migration phases, active blockers, human residue, evidence counts, certificates, and the exact next action. ## Proof labels [#proof-labels] | Label | Meaning | | ------------------ | -------------------------------------------------------------------------------------------------- | | `contract_only` | schemas and denial policies are implemented, but the required test set is incomplete | | `hermetic_proven` | the supplied artifact and representative dataset passed the PostgreSQL-compatible test | | `workspace_proven` | the same digests passed a connected SQL Server run and a managed Databricks Lakebase workspace run | None of these labels means client-proven or production-certified. Client evidence must come from the accepted client scope, and production cutover still requires the normal certificate and cutover gates. # Pipeline import and generation # Pipeline import and generation [#pipeline-import-and-generation] `fa migration-ir` gives migration engineers a reproducible path from source orchestration metadata to reviewable Databricks implementation files. It does not treat a list of target paths as generated code and it does not call a routing policy a native compiler. The command group has two capability levels: | Source | Input accepted today | Output available today | | ------------------------------------------------------------------------------------- | -------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | | ADF and Synapse Pipelines | Native ARM template, one Git JSON file, or an exported Git directory | Lossless IR plus concrete Workflow YAML, Python, BDD, configuration, and Asset Bundle files | | SSIS, PowerCenter, SAS, DataStage, Talend, Oracle Data Integrator (ODI), dbt, Airflow | Credential-free normalized Airlift manifest | Lossless routing, artifact declarations, tests to implement, and remediation packs | The eight normalized-manifest profiles are useful for inventory, dependency ordering, effort routing, and human remediation. Their artifact declarations are not file bodies. Only ADF/Synapse currently has the native importer and concrete file generator described below. ## ADF end-to-end workflow [#adf-end-to-end-workflow] ```bash fa migration-ir import \ --source adf-synapse \ --file factory-export/ \ --estate-name "Commerce integration" \ --snapshot-at 2030-01-15T12:00:00Z \ --output migration-ir.json fa migration-ir inspect --file migration-ir.json fa migration-ir generate \ --file migration-ir.json \ --out-dir generated fa migration-ir validate \ --file generated/artifact-set.json \ --root generated ``` The generated directory contains the actual bytes referenced by the artifact set: ```text generated/ ├── artifact-set.json ├── databricks.yml ├── resources/ │ └── adf-migration.yml ├── src/adf/ │ ├── copy_runtime.py │ └── review_required.py ├── generated/adf_synapse/ │ └── ... content-digested node files └── tests/ └── ... Experiments-compatible BDD specifications ``` `artifact-set.json` includes every file body, SHA-256 digest, byte length, source-node provenance, required runtime binding, limitation, and executable flag. You can transport that one JSON document through CI and materialize it later: ```bash fa migration-ir materialize \ --file artifact-set.json \ --out-dir generated ``` ## Import native ADF exports [#import-native-adf-exports] Use an ARM template, a single ADF Git JSON file, or the root of an ADF Git export: ```bash fa migration-ir import \ --source adf-synapse \ --file adf-export/ \ --estate-name "Finance pipelines" \ --snapshot-at 2030-01-15T12:00:00Z \ --product-version 2018-06-01 \ --output migration-ir.json ``` The importer reads `pipeline/`, `dataset/`, `linkedService/`, and `trigger/` documents. It creates stable node identifiers, resolves activity and dataset dependencies, preserves source fragments and locations, and creates opaque connection bindings. Provide explicit connection references when the generated defaults do not match your connection registry: ```json title="credential-refs.json" { "WarehouseSource": "databricks-connection://migration/warehouse-source", "LandingStorage": "databricks-secret://migration/landing-storage" } ``` ```bash fa migration-ir import \ --source adf-synapse \ --file adf-export/ \ --estate-name "Finance pipelines" \ --snapshot-at 2030-01-15T12:00:00Z \ --credential-map credential-refs.json \ --output migration-ir.json ``` Never add credentials to the export. Parameter, Key Vault, and secure-string references become opaque bindings. A plaintext password, token, connection string, account key, or client secret fails import before an IR is produced. ## Inspect and compile routing [#inspect-and-compile-routing] ```bash fa migration-ir inspect --file migration-ir.json fa migration-ir compile --file migration-ir.json --json > migration-ir-bundle.json ``` Inspection validates IDs, dependencies, and cycles, then reports routing counts. Generic compilation produces: * one typed IR node per accepted source construct; * `deterministic`, `agent_repairable`, `human_only`, `excluded`, or `blocked` disposition; * source fragments, locations, extractor generation, and content digests; * artifact declarations with `materialization: descriptor_only`; * behavioral-test requirements and remediation packs; and * ownership contracts for Harness, Experiments, Platform, and Runway. Unknown nodes are not dropped. Airlift retains their safe source fragment and routes them to remediation. A new rule can therefore be developed against the exact accepted source construct without rescanning the estate. ## Generate concrete ADF files [#generate-concrete-adf-files] ```bash fa migration-ir generate \ --file migration-ir.json \ --out-dir generated \ --artifact-set generated/artifact-set.json ``` Generation currently admits the common ADF patterns it can express safely: * pipeline and dataset configuration; * one-input, one-output Copy activities; * pipeline dependencies and retry counts; * supported schedule triggers; * explicit target-job bindings for pipeline invocation; * a Databricks Asset Bundle root and Workflow resource; and * one BDD specification per deterministic node. ADF expressions, Mapping Data Flow semantics, conditions, loops, tumbling-window behavior, external calls, and unknown activities do not disappear. They remain repairable or human work until an implementation artifact is independently reviewed and validated. The generated copy runtime requires Unity Catalog `source_identifier` and `target_identifier` bindings. It covers table-to-table materialization. Snapshot, CDC, delete, file-layout, and schema-drift behavior belongs to the selected Airlift transfer profile and must be validated separately. ## Validate bytes [#validate-bytes] Validate the artifact-set envelope by itself: ```bash fa migration-ir validate --file generated/artifact-set.json ``` Also compare every materialized file with its recorded body, digest, and byte length: ```bash fa migration-ir validate \ --file generated/artifact-set.json \ --root generated ``` Validation fails on a modified, missing, duplicated, or path-escaping artifact. It does not claim that Databricks runtime behavior is correct. Use Runway preview and deployment, then execute the generated BDD and parity scenarios through Fabric Experiments. ## Register immutable lineage [#register-immutable-lineage] After validation, store the artifact set in an admitted immutable store and register its reference: ```bash fa migration-ir register \ --file generated/artifact-set.json \ --engagement-id \ --estate-id \ --artifact-id \ --idempotency-key ``` Registration invokes `airlift.artifact_register` through the authenticated Platform action API. Airlift records the engagement, estate, reference, media type, tool generation, and artifact-set digest. It does not put file bodies or credentials in the migration ledger. ## Qualify a release candidate [#qualify-a-release-candidate] `qualify` turns byte verification, runtime bindings, residue decisions, Databricks bundle validation, and optional live provider evidence into one fail-closed report: ```bash fa migration-ir qualify \ --file generated/artifact-set.json \ --root generated \ --bindings release-bindings.json \ --resolutions residue-resolutions.json \ --proof hermetic_proven \ --output qualification.json ``` For workspace proof, add `--workspace-evidence workspace-evidence.json` and request `--proof workspace_proven`. The evidence must contain the governed Airlift artifact, deployment requirement, validation execution, per-object validation runs, and matching Databricks workspace references. A missing or mismatched Runway/Experiments reference, artifact digest, tenant, object verdict, or workspace identity blocks the command. Validation runs exported with a `conversionId` require `"schemaVersion": 2` on the document; untagged documents are read as version 1 and keep working unchanged. Read [Qualify a generated release](/docs/migration/release-qualification) for the binding, residue, Runway, Experiments, and workspace-evidence contracts. ## Deploy and validate [#deploy-and-validate] The generated bundle is a candidate release: 1. Use `fa migration-ir validate` and `fa migration-ir qualify` to prove byte integrity, bundle validation, runtime bindings, and residue disposition. 2. Use Airlift conversion and remediation actions to resolve every non-deterministic node. 3. Use `fr` to preview, deploy, promote, reconcile, or roll back the immutable release. 4. Run generated behavioral scenarios with Fabric Experiments. 5. Return Runway and Experiments references and digests to Airlift readiness. `fa` owns migration intent, artifact lineage, readiness, and cutover gates. `fr` owns release execution. There is no `fa migration-ir deploy` command. # Migration-pack commands # Migration-pack commands [#migration-pack-commands] `fa migration-pack` compiles an exported source inventory into an executable migration bundle. It does more than generate a checklist: the compiler validates dependencies, routes every object to an automation or remediation lane, defines restartable data movement, selects validation suites, and produces immutable target requirements. Use `fa source` to inspect the broader capability catalog or draft a generic engagement plan. Use `fa migration-pack` when you have a real, versioned inventory for one of these executable packs: | Source | Supported variants | Data movement checkpoint | | --------------- | ------------------------------------------------ | --------------------------------------------------------------------- | | Microsoft SQL | SQL Server, Azure SQL Database, Managed Instance | LSN, change tracking, or application watermark | | Snowflake | account and warehouse metadata | unload manifest plus stream, timestamp, or application watermark | | Amazon Redshift | provisioned, Multi-AZ, serverless | S3 UNLOAD manifest plus sequence, timestamp, or application watermark | | Oracle | Database, Exadata, Autonomous Database | consistent snapshot plus SCN or timestamp watermark | | Teradata | Vantage and appliance estates | utility restart state plus source watermark | | Hadoop | Cloudera, Hortonworks, Apache Hadoop | file manifest plus ingestion watermark | ## Manifest contract [#manifest-contract] The input is credential-free JSON. Keep connection secrets in the execution system, not in the manifest. ```json { "schemaVersion": 1, "estate": { "name": "Commerce Oracle estate", "sourceSystem": "oracle", "variant": "oracle_database", "sourceVersion": "19c", "sourceSnapshot": "2025-01-15T12:00:00.000Z" }, "inventory": [ { "sourceId": "package:order_api", "name": "ORDER_API", "kind": "package", "sourcePath": "exports/packages/order_api.sql", "owner": "order-platform", "dependencies": ["table:orders"], "hardCases": ["plsql_package", "exception_handling"] } ] } ``` The schema rejects duplicate source IDs, missing dependency nodes, invalid source and variant combinations, unknown hard-case labels, and dependency cycles. It deliberately has no credential, token, or connection-string fields. ## Inspect [#inspect] ```bash fa migration-pack inspect --file source-manifest.json fa migration-pack inspect --file source-manifest.json --json ``` Inspection prints scope, dependency order, routing totals, required hard-case coverage, missing cases, and the canonical bundle digest. Missing hard cases remain visible because a small inventory must not silently certify a larger source capability. ## Plan [#plan] ```bash fa migration-pack plan \ --file source-manifest.json \ --json > migration-plan.json ``` The generated `application/vnd.fabric.airlift.migration-pack+json` bundle contains: * the accepted inventory digest and stable dependency order; * deterministic, bounded-agent, and human-remediation routes per object; * source-specific snapshot, catch-up, restart, and reconciliation requirements; * required Experiments validation suites and discrepancy return paths; * Unity Catalog, Delta, Lakeflow, Databricks SQL, and runtime target mappings; * a Runway-owned immutable deployment requirement; and * a canonical SHA-256 digest. The same manifest produces the same bundle and digest. ## Register [#register] ```bash fa migration-pack register \ --file migration-plan.json \ --engagement-id \ --estate-id \ --artifact-id \ --idempotency-key ``` Registration invokes the governed `airlift.artifact_register` action through the authenticated API. The CLI records the immutable reference and digest. It neither uploads source exports nor creates an alternate mutation path. ## Certification check [#certification-check] Run the recurring, account-independent suite for all six packs: ```bash pnpm certify:migration-packs ``` Evaluate a generated evidence request independently: ```bash fa migration-pack certification-check \ --file reports/f9-oracle-evidence.json \ --json ``` The evaluator requires a complete source hard-case corpus and one bound build and dataset across all stages. It accepts only `hermetic_proven` and `workspace_proven`. Client acceptance and production certification require external evidence and cannot be requested through this command. See [migration-pack certification](/docs/operations/migration-pack-certification) for the evidence contract. # Modernization and value commands # Modernization and value commands [#modernization-and-value-commands] The `fa` CLI owns migration-program intent and evidence. Use `fr` for artifact deployment, promotion, and rollback. Use Experiments to execute A/B and parity checks; record their immutable references in Airlift. ```bash fa modernization list [--engagement-id ] [--object-id ] [--json] fa modernization show [--json] fa modernization recommend|decide|plan|start|evidence|promote|rework \ --file --idempotency-key [--json] fa value observations|summaries|reports [--engagement-id ] [--json] fa value record|summarize|publish \ --file --idempotency-key [--json] ``` All mutations call the authenticated Airlift API. The CLI never accepts an organization or actor override; Databricks identity and configured membership determine authority. See [Modernization studio](/docs/migration/modernization) and [Measure engagement value](/docs/operations/value-measurement) for payloads and gates. # Authenticated automation # Authenticated automation [#authenticated-automation] The CLI includes a versioned authenticated command/query transport for engagement, connection, assessment, inventory, dependency, and planning operations. Configure it with the Databricks App URL and a short-lived token admitted by that App's ingress: ```bash export AIRLIFT_API_URL="https://" export DATABRICKS_TOKEN="$(your-secret-provider read databricks-token)" fa engagement list --json ``` Do not put tokens in JSON inputs, command arguments, committed environment files, or shell history. The examples use placeholders; connect your normal CI secret provider. For unattended automation, use an OAuth service principal with `CAN_USE` on the App and an explicit Airlift `automation` membership. A generic workspace user OAuth token or PAT is not a substitute for App ingress identity and can return HTTP 401 even when it works against workspace APIs. Interactive developers can perform the same governed operations in the App with Databricks SSO. Airlift does not provide a second login flow or accept caller-authored identity headers. ## Identity and organization binding [#identity-and-organization-binding] The request body contains action, parameters, correlation metadata, and a content-bound idempotency key. It does not contain actor or organization. Harness authenticates the Databricks principal at the App boundary, then Airlift requires exactly one configured organization membership before the Platform action can run. Human users receive their configured tenant role. A service principal must have the narrow `automation` role. The automation role may onboard engagements/connections, record assessment and inventory output, propose dependency graphs, export assessment packs, and generate plan scenarios. The automation role cannot accept dependency graphs or assessment scope, select/freeze plans, approve waves, accept business results, waive evidence, mint certificates, configure policy, execute cutover, or roll back. ## Retry contract [#retry-contract] Supply a stable, non-secret prefix with `--idempotency-key`. The CLI adds a canonical digest of the action and parameters. An exact retry collapses to the original action invocation and events; changed content cannot reuse the old content-bound key. ```bash fa engagement create \ --file engagement.json \ --idempotency-key project-42-engagement ``` Persist the prefix with your deployment job. Do not generate a new timestamp or random value on each retry. ## Verify a connection binding with recorded evidence [#verify-a-connection-binding-with-recorded-evidence] A binding becomes `verified` only against a connectivity diagnostic recorded by an admitted system principal through `airlift.connection_diagnostic_record`. The handler derives the diagnostic digest; a caller-chosen digest is rejected. Record the diagnostic, then verify the binding with that recorded digest: ```bash fa connection diagnose --file probes.json --idempotency-key project-42-diagnose fa connection test --json fa connection verify \ --digest \ --idempotency-key project-42-verify ``` This recipe is for admitted automation. Interactive operators perform the same step in the App with **Verify binding**, which attaches the recorded diagnostic automatically and never asks anyone to paste a digest. ## Remote migration operations [#remote-migration-operations] Local source planning, diagnostics, catalog inspection, and certificate verification remain read-only. Authenticated remote commands cover engagement and connection onboarding, assessment, inventory, planning, conversion batches, artifact lineage, remediation, transfer control, and deployment requirements. Authority remains role-bounded. Operators can plan/control a transfer and declare a required Runway outcome. Admitted automation records checkpoints, runner failures, reconciliation, and Runway observations. The CLI transport does not turn a human token into runner authority, and it does not make a deployment requirement equivalent to Runway success. An automation client remains a transport client. It must not accept `--actor` or `--tenant` as authority, write the Airlift database directly, or provide a bypass around approval, certificate, waiver, or cutover policy. # Source planning commands # Source planning commands [#source-planning-commands] Source commands are read only. They inspect the installed registry, create deterministic project inputs, and check evidence manifests; they do not connect to a source or mutate an Airlift estate. ## List source packs [#list-source-packs] ```bash fa sources fa sources --json | jq '.[] | {id, archetype, evidenceSupportLevel, implementationRoutingLevel, variants}' ``` `evidenceSupportLevel` is derived from admitted capability cells. The unauthenticated local catalog has no tenant evidence, so it reports `cataloged`. `implementationRoutingLevel` describes the deterministic workflow shipped in the package; it is not workspace or client proof. See the [capability registry](/docs/sources/capability-registry). ## Bulk plan export [#bulk-plan-export] ```bash fa sources export --plans --format jsonl fa sources export --plans --json > .airlift/source-plans.json ``` Emits one canonical-JSON record per source × variant in the installed registry, in registry order. Each record wraps a compiled plan with `sourceSystem`, `sourceVariant`, `label`, `archetype`, `implementationRoutingLevel`, `programTrack`, `externalGates`, a `docsUrl` pointing at the source's documentation page, and a `contentDigest` computed over the plan alone — the digest is stable across CLI releases when the underlying plan is unchanged, so downstream indexers can re-index incrementally instead of re-scanning every record. The default text output is the same JSONL form; add `--json` for a single canonical JSON array instead. ## Inspect one pack [#inspect-one-pack] ```bash fa source inspect dynamics_365 fa source inspect dynamics_365 --json > .airlift/dynamics-profile.json ``` The output includes variants, archetype, non-evidentiary implementation routing, Lakebridge capabilities where applicable, inventory inputs, movement options, validation inputs, target services, workload surfaces, residue, and the credential boundary. ## Compile a plan [#compile-a-plan] ```bash fa source plan dynamics_365 \ --variant dynamics_365_finance_operations \ --json > .airlift/dynamics-finance-plan.json fa source plan kafka \ --variant confluent_cloud \ --json > .airlift/confluent-plan.json ``` The command emits the current archetype-specific plan schema. Each step includes `specialistCommands`, `airliftActions`, outputs, and exit criteria. Command templates beginning with `#` name an adapter boundary; they are not fake executables. ## Generate the complete developer recipe [#generate-the-complete-developer-recipe] ```bash fa source recipe snowflake fa source recipe snowflake --variant snowflake --json > .airlift/snowflake-recipe.json ``` The recipe is generated from the installed source registry and CLI contract. It covers source inspection, governed engagement scope, assessment acceptance, any admitted source-specific compiler, conversion and transfer, independent validation, certificates, cutover status, and evidence export. Every stage identifies its expected artifacts and engagement-relative App route. The App does not display a permanent menu for every supported source. **Active sources** is derived from estates assigned to active engagements. Add another source through **Engagements → Add source** or the governed `estate register` plus `engagement update` commands; the source workspace then appears automatically. ## Check a live certification manifest [#check-a-live-certification-manifest] ```bash fa source certification-check source-certification.json fa source certification-check source-certification.json --json ``` The command exits `1` when required immutable runs are absent. `assessable` requires an inventory run; `executable` adds transfer and validation; `certifiable` adds scale; and `cutover_certified` adds a cutover-effector run. A successful check does not mutate the published registry. ## TypeScript equivalent [#typescript-equivalent] ```ts import { createSourceMigrationPlan, evaluateSourcePackCertification, resolveSourceSystemProfile, } from '@fabricorg/airlift'; const profile = resolveSourceSystemProfile('sap_s4hana'); const plan = createSourceMigrationPlan(profile.id, { variant: 'sap_s4hana' }); const result = evaluateSourcePackCertification(manifest); ``` # Synapse golden-path commands # Synapse golden-path commands [#synapse-golden-path-commands] `fa synapse` is the source-specific convenience layer for the reference Synapse-to-Databricks journey. Local compilation is deterministic and needs no API. Registration uses the authenticated Airlift API and the existing governed artifact action. ## `fa synapse inspect` [#fa-synapse-inspect] ```bash fa synapse inspect --file synapse-manifest.json fa synapse inspect --file synapse-manifest.json --json ``` Parses the versioned manifest, validates unique source IDs and dependency references, detects cycles, compiles embedded ADF resources, and prints scope and residue totals. Exit status `0` means the manifest can produce a bundle. Invalid schemas, missing dependencies, and cycles exit `1`. ## `fa synapse compile-adf` [#fa-synapse-compile-adf] ```bash fa synapse compile-adf --file TemplateForWorkspace.json --json ``` Compiles an ADF/Synapse ARM template into ETL IR version 1. The JSON result contains tasks, triggers, linked-service binding requirements, and residue. The compiler does not emit or retain linked-service credentials. ## `fa synapse plan` [#fa-synapse-plan] ```bash fa synapse plan --file synapse-manifest.json --json > synapse-plan.json ``` Emits `application/vnd.fabric.airlift.synapse-golden-path+json`. The document contains: * source snapshot and inventory digest; * topological dependency order; * ETL tasks with dependencies, parameters, retry policy, and disposition; * Databricks target-asset mappings; * snapshot/incremental movement proofs; * Experiments validation suites; * agent-repairable and human-only residue; * completion evidence for discover through modernize; * a canonical SHA-256 `bundleDigest`. The same input produces the same output and digest. ## `fa synapse register` [#fa-synapse-register] ```bash fa synapse register \ --file synapse-plan.json \ --engagement-id \ --estate-id \ --artifact-id \ --idempotency-key ``` Validates the generated bundle and invokes `airlift.artifact_register` with kind `plan`. The command records the supplied immutable reference; it does not upload local content. Configure `AIRLIFT_API_URL` and `DATABRICKS_TOKEN` as described in [remote automation](/docs/cli/remote-automation). Registration fails closed when the engagement or estate belongs to another tenant, the principal lacks artifact authority, the bundle is invalid, or the request digest does not match the idempotency key generated by the CLI. ## `fa synapse certification-check` [#fa-synapse-certification-check] Run the repository-owned, account-independent suite first: ```bash pnpm certify:synapse ``` The runner writes a human-readable report and a versioned evaluator input under `reports/`. Check that input independently or in CI: ```bash fa synapse certification-check \ --file reports/f8c-synapse-evidence.json \ --json ``` Exit status `0` means every stage required by the requested proof level is present, content-digested, tied to the same build and dataset, and completed before the report observation time. Missing, future-dated, wrong-build, or wrong-dataset evidence exits `1`. The account-independent request schema accepts only `hermetic_proven` and `workspace_proven`. It rejects `client_proven` and `production_certified` rather than letting a caller reinterpret local test results as production evidence. Workspace proof additionally requires immutable evidence for the live App API, target materialization, Experiments validation, Runway release, and Temporal rehearsal. A successful App deployment alone is insufficient. # Airlift Migration Assistant # Airlift Migration Assistant [#airlift-migration-assistant] The Airlift Migration Assistant helps a developer answer four immediate questions from any focused migration blocker: 1. What failed, and why did Airlift stop automation? 2. Which person or team acts next? 3. What implementation and diagnostic checks are appropriate? 4. Which immutable artifact and independent evidence clear the gate? The canonical next action always comes from the Airlift ledger. When a model is configured, a finite Fabric Harness agent adds plain-language explanation and up to five diagnostic checks. It has no tools and cannot mutate state, execute SQL, approve evidence, waive policy, deploy, certify, or cut over. ## Runtime layers [#runtime-layers] | Layer | Purpose | Identity | Required? | | ----------------------------- | ------------------------------------------------------------------------ | ----------------------------------------------------------- | --------- | | Airlift governed guidance | Exact blocker, owner, next control, and evidence requirement | Airlift projection | Always | | Migration Assistant reasoning | Developer explanation and bounded diagnostics | Databricks App service principal, or secret-backed provider | Optional | | Databricks Genie analytics | Questions over curated migration projections and optional diagnostic SQL | Signed-in user through OBO | Optional | The product-facing assistant is **Airlift Migration Assistant**. A **Databricks Genie** Agent is an optional analytics specialist. It does not replace the Harness agent runtime. The legacy `/airlift-genie` route and `AIRLIFT_GENIE_*` variables remain compatibility aliases; they do not name the product surface. ## Recommended: Unity AI Gateway [#recommended-unity-ai-gateway] Use Unity AI Gateway when the workspace has an approved Unity Catalog model service. This keeps model access, provider routing, permissions, usage tracking, and inference policy in Databricks. 1. Select an approved model service, normally a fully qualified `system.ai.*` name. 2. Grant the Airlift Databricks App service principal permission to use it. 3. Set the bundle variables and deploy the App: ```bash databricks bundle deploy -t dev \ --var="airlift_genie_provider=unity-ai-gateway" \ --var="airlift_genie_model=system.ai." \ --var="airlift_genie_inference_mode=ai-gateway" ``` The App receives these non-secret settings: ```text AIRLIFT_GENIE_PROVIDER=unity-ai-gateway AIRLIFT_GENIE_MODEL=system.ai. DATABRICKS_MODEL=system.ai. DATABRICKS_INFERENCE_MODE=ai-gateway ``` Open **Advanced tools → Migration Assistant**. **Model reasoning** should read **Unity AI Gateway**. Then open a focused remediation case and select **Ask Migration Assistant**. ## Databricks Model Serving [#databricks-model-serving] For a custom workspace serving endpoint, grant the App service principal query permission and use: ```bash databricks bundle deploy -t dev \ --var="airlift_genie_provider=model-serving" \ --var="airlift_genie_model=airlift-approved-endpoint" \ --var="airlift_genie_inference_mode=serving-endpoints" ``` The Harness Databricks adapter owns OAuth token rotation and the OpenAI-compatible transport. Airlift does not implement a Databricks client or persist the token. ## External provider fallback [#external-provider-fallback] Use a direct provider only when workspace-native inference is unavailable. Do not paste a key into the Airlift UI or place it in an action, event, prompt, source file, or ordinary environment-value field. Create an App-specific secret and add a Databricks App secret resource: ```yaml config: env: - name: AIRLIFT_GENIE_PROVIDER value: openai-compatible - name: AIRLIFT_GENIE_MODEL value: approved-model - name: AIRLIFT_GENIE_BASE_URL value: https://models.example/v1 - name: AIRLIFT_GENIE_API_KEY value_from: airlift-genie-api-key resources: - name: airlift-genie-api-key secret: scope: airlift-genie key: model-api-key permission: READ ``` Supported provider settings are: | Provider | Required non-secret settings | Required secret binding | | ------------------- | -------------------------------------------------------------------------- | ----------------------- | | `openai-compatible` | `AIRLIFT_GENIE_MODEL`, `AIRLIFT_GENIE_BASE_URL` | `AIRLIFT_GENIE_API_KEY` | | `anthropic` | `AIRLIFT_GENIE_MODEL`; optional base URL | `AIRLIFT_GENIE_API_KEY` | | `azure-openai` | `AIRLIFT_GENIE_BASE_URL`, `AIRLIFT_GENIE_DEPLOYMENT`; optional API version | `AIRLIFT_GENIE_API_KEY` | Use a separate secret scope for the App and grant it access to only the required key. The configuration page reports **secret bound** but never returns the value to the browser. ## Optional Genie analytics [#optional-genie-analytics] Set `AIRLIFT_GENIE_AGENT_ID` only when developers need questions over curated migration projections. Enable Databricks Apps user authorization with the `genie` scope, grant the intended user **Can run**, and restrict the agent to approved Unity Catalog sources. Genie analytics runs as the signed-in user. Migration Assistant model reasoning runs as the App service principal. The different identities are intentional: shared guidance uses the approved application model, while data visibility continues to respect each user's existing permissions. ## Failure behavior [#failure-behavior] * Missing model settings: show ledger-derived guidance. * App lacks model permission: show ledger-derived guidance and leave the gate unchanged. * Provider timeout, budget exhaustion, or malformed response: show ledger-derived guidance. * Missing Genie user token or permission: skip Genie analytics. * Generated diagnostic SQL: display for review; never execute automatically. Model absence cannot block migration work, and model success cannot advance it. # Databricks access & integrations # Databricks access & integrations [#databricks-access--integrations] > **DBX1 status:** this release implements and tests the access contract in the repository. > It does not claim live closure until an immutable same-digest App deployment and workspace > preflight are admitted for the target workspace. Airlift renders one observed view of two distinct surfaces: * **Deployment access** — whether the Airlift App can actually reach the workspace resources it operates against: identity, organization membership, App permission, the bound SQL warehouse, and every configured surface (analytics schema, evidence Volume, model route, Genie Agent, system tables). * **Target integrations** — the five construct-independent Databricks integrations: Unity Catalog governance, Unity Catalog lineage, Unity AI Gateway, AI/BI reporting, and Databricks Genie analytics. These surfaces share one observed-state vocabulary but answer different questions. An access preflight proves reachability; it is never integration readiness. ## One shared observed-state vocabulary [#one-shared-observed-state-vocabulary] Both surfaces render through the same five tokens: | State | Meaning | | ---------------- | -------------------------------------------------------------- | | `not_configured` | No admitted configuration for this integration. | | `not_observed` | Configured, but no admitted workspace observation. | | `stale` | The last admitted observation is outside its freshness window. | | `blocked` | A named blocker prevents observation. | | `observed` | A current admitted observation exists. | A preflight's `observed` state means "an admitted probe just proved the deployment reaches the workspace". An integration's `observed` state requires admitted capability evidence of workspace strength admitted through the capability registry — a stricter contract that no configuration, screenshot, dashboard tile, or model output can manufacture. No configuration flag ever renders as readiness. ## Access preflight contract [#access-preflight-contract] The governed action `airlift.access_preflight_record` records one preflight observation: * **Actor bound.** Only an admitted system principal can record. Humans and agents are rejected by the `airlift.access_preflight_admission.v1` policy, and the handler restates the bound. * **Admission.** A missing or unverifiable evidence identity fails closed. The deployment supplies an access-probe verifier seam; absent, the admission policy blocks. * **Required coverage.** Deployment evidence requires the App principal's observable `identity` and `sql_warehouse` checks plus every configured native surface. Participant evidence additionally requires `organization_membership` and `app_permission`, derived from the governed directory and the live delegated App request. Partial coverage, an undeclared check, or a required check marked `not_applicable` is rejected. * **Derived fields.** The pipeline derives the preflight digest, the per-check states, and the overall state. Caller-supplied derived fields are rejected by the strict schema; the CLI strips them before transport as a defense in depth. * **Secret-safe output.** Credential references and userinfo are rejected at both the action params schema and the recorded-event schema. * **Replay.** Malformed or digest-mismatched replays are dropped whole; a capped history keeps the newest records. The retained history also bounds the duplicate-digest guard: an evicted record's digest can be recorded again, though its freshness window will render it `stale` rather than `observed`. * **Production seam.** Governed deployments currently fail closed: no production runtime wires an admitted access-probe verifier yet, so no governed deployment can record a preflight until one is admitted. That is the deliberate, honest default — a permissive production verifier would admit arbitrary evidence. The demo environment seeds one passing preflight behind the development-assurance boundary only. Record a preflight through the admitted probe path: ```bash fa access record \ --file .airlift/access-preflight.json \ --idempotency-key access-preflight- ``` Inspect the derived surface — exit 0 only when access is observed, so it works as a CI gate: ```bash fa access show fa access show --json ``` The App renders the same derivation at the **Databricks access & integrations** page. That page is read-only: recording is an admitted system action, and no browser form escalates a human to a system principal. ## Check my workspace access [#check-my-workspace-access] The deployed Databricks App adds one task-oriented action to the Workspace readiness section: **Check my workspace access**. It exists so an operator can prove reachability before migration work starts instead of discovering a missing grant mid-stage. Pressing the button runs two checks against Databricks directly and records both through the governed system action `airlift.access_preflight_record` under the admitted worker principal — the button never mints its own evidence: * **Participant check** — probes with the signed-in person's delegated Databricks session (on-behalf-of). It proves what *you* can reach: identity, organization membership, App permission, the SQL warehouse, and the configured native surfaces. * **Deployment check** — probes as the App's own service principal. It proves what the *deployment* can reach when no person is signed in. Both preflights are admitted with an idempotency key bound to the evidence digest, so a retry of the same outcome collapses to the original record. The result renders as migration impact, owner, and next step: either "Workspace access is ready for this migration", or a count of items needing attention with each blocker's five-part explanation below the button (what happened, why it blocks migration, who can fix it, the required evidence, and one primary action). A denied or blocked state is a precondition answer, not a malfunction: | Message | Meaning | Recovery | | ----------------------------------------------------------- | ----------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- | | `Workspace sign-in required.` | No authenticated App session. | Sign in to the workspace App and retry. | | `This workspace is not available for your Airlift account.` | Your identity resolves to no membership in this organization. | Ask a tenant admin to grant membership on the [Team access](/docs/reference/organization-membership/) page. | | `Your Airlift role cannot run workspace checks.` | Your role lacks `airlift:access:observe`. | Ask an admin for a role that carries access-observe permission. | | `Databricks did not provide a delegated workspace session.` | The App received no on-behalf-of session, so the participant lane cannot run. | Reopen the App from the workspace so Databricks forwards a delegated identity. | | `N workspace items need attention.` | One or more probe checks did not pass. | Apply the named fix (usually a grant named in the blocker), then press the button again. | | `The workspace check could not finish.` | The probe itself errored. | Retry once; if it persists, ask a workspace admin. | Successful checks are throttled to one run per person per 30 seconds; a failed result is never cached, so a fix-then-retry loop is never blocked by the throttle. The same blocked-state vocabulary appears in the [Fix list](/docs/getting-started/when-something-fails/) — a failed check is normal specialist work with a named owner, never a silent zero. ## Blockers [#blockers] Every non-observed access state renders a five-part blocker: what happened, why it matters, who acts, the required evidence, and one primary action. A denied permission names the grant; a missing check names the coverage; a stale preflight names the window. ## Honest support boundary [#honest-support-boundary] Access observations never raise a Databricks target integration to `observed`. The five integrations stay `not_configured`, `not_observed`, `stale`, or `blocked` until admitted workspace-strength evidence lands through the governed capability registry. The bounded Airlift migration assistant is a separate surface from Databricks Genie analytics; see [Airlift Migration Assistant](/docs/integrations/airlift-genie/) for its model-routing contract. # Evaluator readiness # Evaluator readiness [#evaluator-readiness] `evaluation_ready` is a **derived label, never a stored one**. It is true exactly when a fresh rehearsal against the newest frozen evaluation candidate passed every required route inside an unexpired window. No event, handler, or caller can write it, and it promotes nothing — not an integration, a capability cell, a certificate, a gate, or a cutover decision. Two governed actions build the evidence: * `airlift.evaluator_candidate_freeze` — freezes one evaluation candidate: its content digest, release reference, proof snapshot, change-log reference, known limitations, and window expiry. The frozen candidate is governed state; readiness compares rehearsals against it, never against anything a caller or configuration supplies. * `airlift.evaluator_rehearsal_record` — records one rehearsal of the evaluation journey against the newest admitted freeze. ## Readiness law [#readiness-law] | Condition | Result | | ------------------------------------------------------------------- | ------------------------------------------------------------ | | No candidate frozen | Fails closed — `no_freeze` | | Freeze window expired | Fails closed — `freeze_expired` | | No rehearsal against the freeze | Fails closed — `no_rehearsal` | | Rehearsal against a superseded candidate | Fails closed — `candidate_mismatch` | | Rehearsal older than the freshness window | Fails closed — `stale_rehearsal` | | A required route missing, failed, or not applicable | Fails closed — `route_coverage_incomplete` or `route_failed` | | Fresh, candidate-matched, fully-passing rehearsal, unexpired window | `evaluation_ready: true` | The rehearsal freshness window is a module constant, never organization-configurable — configuration may never buy readiness. The evaluation window's recorded expiry is separately bounded by a module-constant maximum, and it never extends rehearsal freshness. ## Required coverage [#required-coverage] Every rehearsal must cover the two core routes (`app`, `sql_warehouse`) plus every route for a configured surface (`unity_catalog`, `jobs`, `lineage`, `dashboard`, `ai_gateway`, `genie`). Coverage is conjunctive: a missing, undeclared, duplicate, or `not_applicable` required route is rejected. The digest binds the derived outcome, so a flipped replay cannot pass the recompute. ## Recording contract [#recording-contract] Freezing and rehearsing are **admitted system actions**. Recording requires an in-process admitted system principal; the remote transport carries no system actor, so these are the admitted probe's path, never a human's. The App section is read-only. ```bash fa evaluator freeze \ --file .airlift/evaluation-freeze.json \ --idempotency-key evaluator-freeze- fa evaluator rehearsal \ --file .airlift/evaluator-rehearsal.json \ --idempotency-key evaluator-rehearsal- fa evaluator window # exit 0 only when evaluation_ready is true fa evaluator window --json ``` Rehearsal participants are **opaque principal references**, never emails, UPNs, `DOMAIN\user` forms, or credential-shaped strings; identifier-shaped input is rejected at both the action params schema and the recorded-event schema. ## Honest support boundary [#honest-support-boundary] Governed deployments currently fail closed: no production runtime wires an admitted evaluator verifier yet, so no governed deployment can record a freeze or rehearsal until one is admitted. That is the deliberate, honest default. A green label does not imply per-participant access was verified: the access preflight is deployment-scoped and is not yet bound to rehearsal participants. That binding is named in the DBX1 remaining exit. A readiness label is never business proof and promotes nothing. # Fabric Experiments # Connect Fabric Experiments [#connect-fabric-experiments] Fabric Experiments is Airlift's executable validation provider. Airlift decides what must be proven for a migration object and whether admitted evidence satisfies policy. Experiments runs the checks and owns the raw run, results, and evidence manifest. This division prevents two unsafe shortcuts: converter success cannot certify itself, and Airlift does not grow a second test engine. ## Composition contract [#composition-contract] | Concern | Owner | Airlift behavior | | ------------------------------------- | ----------- | ---------------------------------------------------------------------- | | Validation scope and requested tracks | Airlift | Creates `ValidationExecution` and deterministic suite specs | | BDD/A-B/parity/performance execution | Experiments | Runs checks and persists `LiveEvidence` | | Provider run and evidence manifest | Experiments | Returns stable references and SHA-256 digests | | Evidence admission and readiness | Airlift | Verifies producer, reference, digest, artifact, and watermark | | Failed-check disposition | Airlift | Creates governed discrepancies; may delegate classification to Harness | | Certificate policy and signing | Airlift | Mints only from satisfied, current readiness | Airlift stores foreign references and digests, not a copy of Experiments run state. It may block on an Experiments result; it never re-owns the result. ## Worker configuration [#worker-configuration] Create an `ExperimentsValidationAdapter` and an evidence publisher, then inject both into the worker: ```ts const worker = await createAirliftWorker({ mode: 'temporal', runtime, effector, validationAdapter: new ExperimentsValidationAdapter(experimentsRunner), validationEvidencePublisher: evidenceStore, }); ``` The runner receives stable idempotency keys for request creation, every object suite, and manifest finalization. Provider retries must return the same logical run. The publisher must durably persist evidence before returning; otherwise Airlift refuses to admit the validation record. Use Experiments BDD scenarios for business behavior and source-specific semantics, A/B tests for modernization releases against a certified baseline, and live Databricks testkit runs for workspace behavior. Local or synthetic fixtures can exercise adapters and policy, but cannot close a client-specific readiness track. ## Failure behavior [#failure-behavior] * A failed required check becomes a discrepancy and a failed readiness observation. * Provider outage or cancellation leaves the execution uncertain until reconciled. * Duplicate workflow starts collapse to one execution. * Evidence with an unregistered producer, mutable reference, or wrong digest fails closed. * Changing an artifact digest, source watermark, profile, or policy can stale an existing readiness decision and invalidate certificate eligibility. See [Build and run validation suites](/docs/migration/validation) for commands and the discrepancy lifecycle. For generated pipeline candidates, [release qualification](/docs/migration/release-qualification) also requires every requested object to have a passing Experiments run bound to the exact artifact-set digest and the same Databricks workspace used by Runway. ## Modernization comparisons [#modernization-comparisons] A modernization comparison has two named treatments: the certified baseline and modernization treatment. The active migration certificate is the baseline and the separately deployed native artifact is the candidate. The Experiments evaluation reference must include functional guardrails plus at least one declared outcome metric. Airlift refuses readiness when any guardrail or metric fails. Use `fa modernization evidence` to admit the Experiments manifest and evaluation digest; do not paste raw test rows into an Airlift action. Use `paired_execution` benchmark observations when Experiments also measures engineering effort or runtime under comparable conditions. # Harness and Lakebridge # Harness and Lakebridge [#harness-and-lakebridge] Lakebridge is the deterministic analyzer/converter. Airlift invokes its pre-provisioned workspace jobs through the Harness Databricks public adapter and records pinned versions, run references, artifact references, and digests. Harness also defines the finite repair agent used for admitted residue. The agent returns one typed candidate under strict runtime, model, size, iteration, tool, network, filesystem, and command bounds. It never receives decision authority. The in-App **Airlift Migration Assistant** composes the same ownership boundary. Airlift derives the canonical blocker, owner, next action, and required evidence from its projection. A finite Harness guidance agent can add explanation and bounded diagnostics through Unity AI Gateway, Model Serving, or a secret-backed provider. An optional Databricks Genie Agent remains a read-only analytics specialist. The Harness repair agent is the bounded lane that may produce a candidate artifact, and Platform remains the only mutation path. ```text Airlift projection -> deterministic answer (always available) -> Harness Migration Assistant explanation (optional, App principal) -> Genie analytics (optional, user OBO) -> highlighted governed Airlift action -> Harness repair candidate when the selected lane allows it -> Experiments validation -> independent review ``` Harness Temporal owns agent-session durability and common worker/connection plumbing. Airlift owns its application-domain cutover workflow using the Temporal TypeScript SDK; all I/O remains in activities. This ownership distinction is enforced by the family decision map and capability tests. ## Conversion worker contract [#conversion-worker-contract] For each running batch, a worker calls `conversion_start` with `batchId`, `objectId`, `method`, and the exact `toolVersion` pinned on the batch. Airlift rejects mismatches. The worker then calls `conversion_record` with the terminal outcome and provenance. For a successful attempt, it also calls `artifact_register` with the stored output's SHA-256 digest. For a terminal deterministic failure, it calls `residue_create` rather than burying the diagnostic in job output. The repair agent may create and estimate a residue and may submit a resolution after a person assigns the work. It cannot assign work, review its own candidate, certify parity, approve a wave, waive policy, deploy, or cut over. ## Modernization advisor [#modernization-advisor] The bounded modernization advisor receives only governed metadata: source system, object type, candidate Databricks services, and the certified-baseline digest. It returns up to five typed, measurable proposals. It has no tools, network, filesystem, deployment, or approval capability. The worker adapter submits each proposal through `airlift.modernization_recommend` with a stable replay key and immutable Harness run reference. That action is the only authority the advisor receives. A natural person must disposition the proposal, Runway must deploy the separate release, Experiments must compare it with the baseline, and a different person must promote it. # Fabric family composition # Fabric family composition [#fabric-family-composition] The governing law is simple: > Airlift may block on sibling state; it never re-owns it. | Product | Owns | | ----------- | ----------------------------------------------------------------------- | | Airlift | scope, readiness, waves, certificates, cutover policy, migration ledger | | Platform | governed mutations, authorization pipeline, invocation and audit spine | | Harness | bounded agents and Databricks execution plumbing | | Experiments | parity, BDD, evaluation, and quality evidence | | Runway | immutable release deploy, promote, reconcile, rollback | | Radar | operational observations, anomalies, and intervention evidence | | Tower | work, tasks, assignments, and human coordination—not migration truth | Composition uses stable foreign references, digests, idempotency, and reconciliation. Runway, Radar, and Tower are optional. When a sibling is absent, configure an admitted external evidence provider for the required observation or mark the corresponding track not applicable through the versioned validation profile. Never create a look-alike sibling state model inside Airlift. # Fabric Platform # Fabric Platform [#fabric-platform] All 66 externally meaningful state changes are `airlift.*` Platform actions. Human, worker, agent, workflow, administrator, and recovery callers use the same Host boundary. Handlers return pending domain events; they never write projections directly. Platform Host owns durable invocation, idempotency, execution-time reauthorization, policy/state-machine evaluation, adapter evidence, and audit. Airlift owns the domain schemas, policies, events, reducers, and adapters composed into that runtime. Projection readers use the public Platform Host store and checkpoint contracts. Airlift does not query Host internals, fork the invocation ledger, or introduce a second mutation pipeline. # Fabric Radar # Fabric Radar [#fabric-radar] Radar owns monitors, observations, anomalies, interventions, and estate-wide operational views. Airlift requires named observation windows for parallel run, cutover verification, and hypercare, then stores the terminal reference, digest, SLO profile, verdict, and observation timestamps. A critical breach can stale readiness or block cutover under Airlift policy. The monitor definition and live health projection remain in Radar. When Radar is absent, use a typed external evidence profile rather than pretending the integration is installed. ## Submit a Radar observation [#submit-a-radar-observation] An admitted automation principal records the observation after Radar has completed the window: ```bash fa cutover observe --file operational-evidence.json --idempotency-key wave-4-parallel ``` ```json title="operational-evidence.json" { "waveId": "wav_01J00000000000000000000000", "phase": "parallel_run", "providerRef": { "system": "radar", "type": "observation_window", "id": "window-42", "digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" }, "evidenceDigest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "sloProfileId": "airlift.cutover.production_slo.v1", "observedFrom": "2030-09-13T00:00:00.000Z", "observedUntil": "2030-09-14T00:00:00.000Z", "verdict": "passed" } ``` Airlift rejects caller-authored human evidence, mismatched digests, unknown providers, and hypercare observations submitted before hypercare starts. A Radar screen or alert acknowledgement never changes Airlift state by itself. # Use Airlift with Fabric Runway # Use Airlift with Fabric Runway [#use-airlift-with-fabric-runway] The two CLIs have one owner each: | Command | Use it for | Do not use it for | | ------- | ------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | | `fa` | migration scope, conversion, transfer, readiness, and required release outcomes | deploying, promoting, rolling back, or reconciling a Databricks release | | `fr` | immutable Databricks release deployment, gates, promotion, rollback, and release operations | migration inventory, wave scope, transfer, or parity certification | The governing rule is: **Airlift may block on Runway state; it never re-owns it.** Airlift stores a requirement plus verified Runway references and digests. Runway owns the release and environment state machines. ## If the customer already uses Databricks [#if-the-customer-already-uses-databricks] A Databricks-only customer that is not running a migration or modernization engagement does not need Airlift for deployment. Use Runway directly: ```bash fr catalog fr deploy fr promote ``` Use the Runway console or API for detailed deployment state and reconciliation. The current `fr` CLI does not expose separate `status` or `sync` commands; that is a Runway CLI concern, not a reason to create an Airlift estate. Never invent a source estate solely to gain release commands. If the customer later starts a governed Databricks-to-Databricks modernization program, Airlift can track that program once it has an admitted modernization engagement profile. Runway still owns every release operation. ## If Airlift is managing a migration [#if-airlift-is-managing-a-migration] ### 1. Register the immutable target artifacts [#1-register-the-immutable-target-artifacts] ```bash fa artifact register \ --file deployment-manifest-artifact.json \ --idempotency-key sales-bundle-manifest-v7 ``` Accepted artifact kinds are `target_code`, `target_configuration`, `deployment`, and `deployment_manifest`. ### 2. Declare the required Runway outcome [#2-declare-the-required-runway-outcome] ```json title="deployment-requirement.json" { "engagementId": "eng_01J...", "estateId": "est_01J...", "waveId": "wav_01J...", "operation": "deploy", "environment": "staging", "artifactIds": ["art_01J...", "art_01K..."], "requiredState": "succeeded" } ``` ```bash fa deployment require \ --file deployment-requirement.json \ --idempotency-key sales-staging-v7 ``` `require` does not execute the deployment. It records what the migration needs, verifies artifact and engagement scope, and computes `desiredDigest` from the operation, environment, and sorted artifact ID/digest pairs. A production requirement needs a frozen engagement. The Airlift console calls this **Save release requirement**. Saving is successful when the requirement appears as **Awaiting observation**. That status means the desired release is recorded but no verified Runway result is connected yet—not that the button failed. ### 3. Execute the release in Runway [#3-execute-the-release-in-runway] The release engineer or CI system uses Runway: ```bash fr deploy ``` Promotion remains a distinct Runway approval operation: ```bash fr promote ``` The current production adapter is observation-only: `fr` or Runway CI creates the release. Airlift does not submit a second deployment request. ### 4. Connect the Runway result [#4-connect-the-runway-result] Copy the deployment ID and staged artifact SHA-256 printed by `fr deploy`. On the Airlift deployment page, enter them under **Connect the Runway result**, or use the CLI: ```json title="runway-reference.json" { "deploymentRequirementId": "dpr_01J...", "runwayRequestRef": { "system": "runway", "type": "deployment", "id": "dep_01J...", "digest": "<64-character-lowercase-sha256>" } } ``` ```bash fa deployment connect \ --file runway-reference.json \ --idempotency-key sales-staging-v7-runway ``` Connecting the result records a pointer, not proof. A person cannot author Runway truth by typing an ID or digest. Airlift still queries Runway and verifies the deployment before changing migration readiness. ### 5. Verify the Runway result [#5-verify-the-runway-result] The Airlift worker accepts only a narrow Runway observation: ```ts type DeploymentObservation = { requestRef: ForeignReference; releaseRef?: ForeignReference; state: 'pending' | 'running' | 'succeeded' | 'failed' | 'rolled_back' | 'unknown'; actualDigest?: string; observationRef: ForeignReference; observationDigest: string; detail?: string; }; ``` Only admitted automation can record or reconcile this observation. Choose **Check Runway result now** in the Airlift console, or let service automation perform the same check. Configure the observer with `AIRLIFT_DEPLOYMENT_ADAPTER=runway`, `AIRLIFT_RUNWAY_API_URL`, and a secret-backed `AIRLIFT_RUNWAY_API_TOKEN`. Optionally set `AIRLIFT_RUNWAY_APP_URL` so the console offers an **Open Runway** link. It verifies organization, environment, terminal state, and Runway artifact digest before recording evidence. Airlift intentionally exposes no deployment sync, observe, or reconciliation CLI verbs because a caller must not author Runway truth. Use [release qualification](/docs/migration/release-qualification) to bind the resulting Runway observation to the generated Airlift candidate and Experiments verdicts. ### 6. Inspect migration readiness [#6-inspect-migration-readiness] ```bash fa deployment status dpr_01J... --json fa deployment list --engagement-id eng_01J... --json ``` These commands read Airlift's last admitted projection; they do not query Runway live. A pending requirement remains pending until the configured service integration records a verified Runway observation. Investigate the integration or the release in Runway rather than forcing Airlift state from the CLI. Airlift derives the outcome: * `matched` when the observed state equals the required state and any actual digest equals the desired digest; * `drifted` when state or digest differs; * `failed` when Runway reports failure; and * `uncertain` when Runway cannot determine actual state. The Airlift **Deployment requirements** screen shows the requirement, both digests, foreign Runway references, and its effect on migration readiness. Use Runway to inspect or act on the release itself. ## Rollback ownership [#rollback-ownership] Runway executes release rollback. Airlift may declare that a migration recovery path requires `rolled_back`, then it observes the resulting Runway reference and preserves the migration evidence ledger. `fa` never exposes a rollback execution command. # Fabric Tower # Fabric Tower [#fabric-tower] Tower owns tasks, assignments, mission activity, and the operator work queue. Airlift can create or reference work for human-only residue, validation follow-up, waiver review, and cutover rehearsal. Tower completion never advances the migration funnel by itself. The responsible principal must perform the corresponding authenticated Airlift action, and Airlift records the decision and evidence. Airlift's console should provide a readiness matrix with deep links to Tower rather than rebuilding Tower as an engagement workbench. # Unity Catalog evidence spine # Unity Catalog evidence spine [#unity-catalog-evidence-spine] > **DBX2 status:** this release implements and tests the Unity Catalog projection and > reconciliation contracts in the repository. It does not claim live closure until the > exact candidate digest has same-digest workspace Job and reconciliation evidence. The Databricks-native evidence surface is a governed projection of the Airlift event log into the Unity Catalog analytics schema, plus the reconciliation of every Airlift target artifact to its exact target UC identity. It is read-only and fail-closed: missing privilege, a missing system table, lag, or a failed/partial refresh renders a named blocker or `not_observed` — never a misleading zero and never a misleading 100% coverage. A stale projection never displays a current metric. Two governed actions build the evidence: * `airlift.uc_projection_run_record` — records one run of the analytics projection. The run carries the source-event cursor **watermark** it advanced to; freshness derives from that watermark, never from the run's completion time, so a permanently-failing refresh loop can never render "fresh". A failed or partial run never advances the displayed watermark. A watermark that regresses below the newest admitted watermark is rejected. App-triggered refreshes use a Databricks Jobs idempotency token, per-org engagement-scoped single-flight, a 20-second request budget, and cursor-windowed event/artifact scope that never splits equal millisecond timestamps. A still-running or unfinished backlog refresh records `partial`; a terminal job failure records `failed`. * `airlift.uc_target_reconciliation_record` — records the reconciliation of one target artifact, keyed per artifact. An `observed` resolution binds the artifactId, the exact candidate digest, and the exact target UC identity; digest-binding (not name-matching) is required. Every required surface is either native-observed for that exact target or explicitly `notApplicable`; those sets are disjoint and exhaustive. Missing DESCRIBE rows, or an external location/connection that does not match the target, remains blocked. This is what makes target drift detectable. ## Evidence-state law [#evidence-state-law] | Condition | Result | | ------------------------------------------------------------------------ | -------------------------------------------------- | | No projection run recorded | `not_observed` — `no_projection` | | Newest completed watermark older than the freshness window | `stale` — `stale_projection` | | Newest run failed / partial | `failed` — `projection_failed` / `partial_refresh` | | Selected engagement has no accepted artifacts | `no_artifacts` — coverage suppressed | | A reconciliation is blocked (missing grant, unreadable scope) | `coverage_incomplete` — coverage suppressed | | Newest observed reconciliation is bound to a superseded candidate digest | `digest_drift` | | Fresh completed run, all scope readable, every artifact reconciled | `current` | Freshness derives from the watermark (`clock − watermark`), never from run recency. Coverage is suppressed whenever any declared scope is unreadable, so it can never render a misleading 100%. The freshness window is a module constant, never organization-configurable. The derived row names its selected `engagementId`; organization-level callers never display a scope-free watermark or ratio. ## Target identity resolution [#target-identity-resolution] `resolveDatabricksTargetIdentity` is a join over admitted reconciliations — never a function of the artifact alone, because an artifact carries no UC identity and any convention-derived name would be configuration rendering as proof. An `observed` resolution requires an admitted reconciliation binding the artifactId, the exact candidate digest, and the exact target UC identity (`catalog`, `schema`, `name`, `objectType`, and the stable `entityRef`/`entityDigest` when the workspace discloses them). Artifact bytes stay in their provider-owned store; Airlift stores only references and digests. ## Recording contract [#recording-contract] Recording is an **admitted system action** through an in-process admitted system principal; the remote transport carries no system actor, so there is currently no transport by which these commands succeed in a governed deployment. The commands below are the admitted probe's in-process path, never an operator's. ```bash fa uc-evidence record --kind projection-run \ --file .airlift/uc-projection-run.json \ --idempotency-key uc-projection- fa uc-evidence record --kind reconciliation \ --file .airlift/uc-reconciliation.json \ --idempotency-key uc-reconciliation- fa uc-evidence show # exit 0 only when evidence is current fa uc-evidence show --engagement-id # scope to one engagement fa uc-evidence show --json ``` `analytics-projection` is the closed DBX2 projection identifier. Admission rejects caller-invented projection IDs, future watermarks, watermarks later than run completion, and any watermark below the highest admitted cursor for the engagement. ## Refresh migration readiness (App-triggered) [#refresh-migration-readiness-app-triggered] The deployed Databricks App exposes the same pipeline as one operator action in the Databricks-native evidence section: **Refresh migration readiness**. From the operator's perspective the contract is: * **Select an engagement.** The action runs per organization and engagement against an `active` or `frozen` engagement; the derived row always names that engagement. A role must carry `airlift:uc_evidence:observe`; without it the action is denied with "Your Airlift role cannot refresh migration readiness." * **Single-flight.** One refresh per organization and engagement runs at a time. A second press while a refresh is in flight joins the running job rather than starting a duplicate. * **What runs in Databricks.** The App writes a metadata-only projection snapshot of the admitted Airlift events to the configured UC Volume and triggers the native Databricks Workflow job with a Jobs idempotency token derived from the snapshot digest — a retry of the same snapshot collapses to the same job run, so a refresh can never double-apply. * **What gets recorded.** The run and each artifact reconciliation are admitted through `airlift.uc_projection_run_record` and `airlift.uc_target_reconciliation_record` by the admitted worker principal, with idempotency keys bound to the evidence digest. The button never mints evidence directly. * **Bounded scope.** A refresh projects the cursor-windowed event batch since the previous watermark (never splitting equal millisecond timestamps) and reconciles the engagement's current artifacts — superseded candidates are excluded. An engagement with more than 20 current artifacts is refused up front with an instruction to split it into smaller engagements. The button answers with the honest terminal state: | Result | Meaning | Recovery | | ---------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `Databricks refreshed N migration targets.` | Projection completed and every current artifact reconciled `observed`. | None — freshness and coverage now derive from the new watermark. | | `… N need the next fix shown below.` | Projection completed but some reconciliations are blocked (missing grant, unreadable scope, target drift). | Apply the named workspace fix for each blocked target, then refresh again. | | `Databricks started the refresh. Check again shortly…` | The job is still running; the run recorded `partial`. | Wait and re-open the page. Readiness does not advance until the run completes. | | `Airlift projected the next M events. Refresh again…` | The batch hit its window with backlog remaining; the recorded watermark advanced honestly. | Press refresh again to continue from the recorded watermark. | | `Databricks could not complete the projection job.` | Terminal job failure, recorded as `failed`. | Fix the shown workspace blocker and retry. The failure never advances the displayed watermark — [a failed refresh never renders fresh](/docs/getting-started/when-something-fails/). | | `No admitted Airlift events are available to project yet.` | Nothing admitted exists to project and no completed projection exists. | Admit migration evidence (assessment, artifacts) first. | | `Databricks could not finish this refresh.` | The refresh errored outside the job contract. | Retry once; if it persists, ask a workspace admin. | Consistent with the evidence-state law above, a `failed` or `partial` run never renders as current: freshness derives from the newest *completed* watermark, so a broken refresh loop degrades to `stale` or `failed` rather than displaying a misleading current metric. ## Honest support boundary [#honest-support-boundary] Governed deployments currently fail closed: no production runtime wires an admitted UC evidence verifier yet, so no governed deployment can record a projection run or reconciliation until one is admitted. That is the deliberate, honest default. A current evidence state is never integration proof and promotes nothing — not a capability cell, a certificate, a gate, or a target integration's observed state. # Register migration artifacts # Register migration artifacts [#register-migration-artifacts] Artifact registration tells Airlift exactly **what output was produced**, **where its bytes live**, and **which immutable content version** must be validated, deployed, and reviewed. It is not a file upload and it is not a claim that the migration is correct. Airlift always stores four things: 1. the migration engagement and object the output belongs to; 2. an opaque reference to the system that owns the bytes; 3. a SHA-256 fingerprint of the exact bytes; 4. the tool and version that produced them. This prevents a ticket, mutable branch, or converter success message from being treated as proof. Validation, release, review, and certification can all bind to the same digest. Artifacts produced by a normalized source extractor may additionally use the strict v2 workbench envelope. It records construct, automation disposition, target pattern, source-fragment digest, dependencies, warnings, and extractor provenance. Those are declared routing facts, not execution or validation proof. If no active registry cell matches the exact source variant, artifact kind, and construct, the App, API, and `fa` render **no governed capability claim** rather than inferring support. For catalog-governed Synapse constructs, the registration action also resolves the bound estate and requires an exact repository catalog row for source variant, artifact kind, and construct. The declared disposition and target prefix must match that row. This write-time gate prevents newly registered workbench metadata from drifting away from the assessment and conversion route. It does not upgrade the artifact's evidence or support level. Catalog constructs are source-bound: an ADF/Synapse construct cannot be registered against a Teradata or other source estate. Historical catalog claims without an estate binding are also shown as requiring review rather than silently treated as ungoverned. ## What should I register for a repaired conversion? [#what-should-i-register-for-a-repaired-conversion] Register the finalized output that preserves the missing source behavior: * repaired SQL or stored procedure → `target_code`; * repaired Python, Scala, notebook source, or dbt model → `target_code`; * Databricks job, pipeline, cluster, Unity Catalog grant, or other declarative setting → `target_configuration`; * executable regression or parity scenario → `test`; * validation result or signed comparison report → `evidence`. For the remediation flow shown in the App, choose **Target code** unless the repair consists only of configuration. Store the file in a durable provider first—normally a Unity Catalog Volume, a pinned source-control commit, or an immutable release—and then register its reference. ## Artifact kinds [#artifact-kinds] | App option | API value | Use it for | | ---------------------- | ------------------------ | ---------------------------------------------------------------------------------- | | Target code | `target_code` | SQL, Python, Scala, notebooks, dbt models, and other executable target code | | Target configuration | `target_configuration` | Jobs, pipelines, clusters, Unity Catalog objects, permissions, and target settings | | Test | `test` | Executable tests, fixtures, assertions, and BDD scenarios | | Evidence | `evidence` | Validation results, comparison reports, receipts, and other proof | | Transfer specification | `transfer_specification` | Snapshot, CDC, watermark, checkpoint, restart, and reconciliation specifications | | Deployment manifest | `deployment_manifest` | The content-digested set of files and settings in one deployable release | | Deployment receipt | `deployment` | An immutable provider record of a completed deployment | | Runbook | `runbook` | Rehearsal, cutover, verification, recovery, and rollback procedures | | Source snapshot | `source` | Immutable source exports or conversion inputs | | Assessment report | `assessment` | Inventory, dependency, complexity, and readiness reports | | Migration plan | `plan` | Target blueprints, mapping specifications, wave plans, and accepted delivery plans | Kind describes the artifact's role in the migration. It does not describe where it is stored. A workspace-certification media type does not create a workspace claim. The App shows connected workspace evidence only when the artifact is an evidence artifact and a passing governed validation record names its platform-minted artifact ID as the validated subject and corroborates the exact SHA-256 digest. Reusing the digest of another passing run is rejected. Failed runs remain inspectable but do not produce the success-toned workspace label. Otherwise the App displays an uncorroborated-evidence blocker and directs the validation owner to record the missing evidence. ## Field reference [#field-reference] | Field | What to enter | Example | | ----------------------- | -------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | | Migration object | The governed inventory object repaired or implemented by this output | `rebuild customer balance` | | Artifact name | A readable name that distinguishes this immutable version | `Repaired customer balance procedure` | | Artifact kind | The artifact's role in the migration | `target_code` | | Reference system | The provider that owns and serves the bytes | `databricks`, `github`, `azure_devops`, `adls`, `s3`, `runway` | | Reference type | The provider object category needed to interpret the ID | `volume_object`, `workspace_file`, `repo_path`, `object_storage_key`, `runway_release` | | Reference ID | Exact stable path, commit-qualified path, release ID, or object key | `/Volumes////repaired.sql` | | SHA-256 digest | 64 lowercase hexadecimal characters calculated from the final bytes | `6a0f…` | | Media type | Standard MIME type for the bytes | `application/sql`, `application/json`, `text/x-python`, `text/yaml` | | Producer generation | Named tool and pinned version that created the bytes | `lakebridge@0.14.2` or `human-remediation@1` | | `construct` | Normalized source construct emitted by the extractor | `adf.copy` | | `automationDisposition` | One governed disposition; it describes routing, not proof | `deterministic` | | `targetPattern` | Declared target kind and path | `lakeflow_declarative_pipeline:resources/load-orders.yml` | | `sourceFragmentDigest` | SHA-256 of the normalized source fragment | `8d4a…` | | `dependsOnSourceIds` | Bounded source identifiers this construct depends on | `dataset:orders` | | `warnings` | Bounded extractor limitations that remain visible | `CDC semantics require a transfer profile` | | `provenance` | Extractor generation and opaque source reference; never source bodies | `@fabricorg/airlift-adapter-adf@0.3.2 · factory.json#copy-orders` | | `workspaceEvidence` | Read-only App/API/CLI projection from an exact artifact-ID + digest validation binding | `false` until a passing governed run exists | The App renders a neutral **Registry-correlated route** only when the artifact and an active construct cell agree on source variant, artifact kind, construct, extractor/provider generation, target pattern, and automation disposition. That correlation explains the declared route; it is not execution or validation proof. A caller-declared label, mismatched producer, or mismatched route remains uncorroborated. The App separately renders **Catalog-aligned route** when the immutable row still agrees with the current repository catalog. A diverged or removed historical route produces a visible review warning and recovery link; replay is preserved, and the warning never silently edits the original event. Registry correlation and catalog alignment are distinct, non-proof signals. Never put credentials, tokens, signed URLs, connection strings, or code bodies in a reference field. Airlift records lineage; the provider remains responsible for storage and access. ## Calculate the digest [#calculate-the-digest] Calculate SHA-256 after the final edit and from the same bytes addressed by the provider reference. ```bash title="Linux" sha256sum repaired-output.sql ``` ```bash title="macOS" shasum -a 256 repaired-output.sql ``` Copy only the 64-character lowercase digest. If the file changes, calculate a new digest and register a new artifact. Do not update the existing artifact row. ## Register from the App [#register-from-the-app] 1. Open the blocked remediation case. 2. Select **Register repaired artifact**. 3. Confirm the object and choose the artifact kind. 4. Enter the provider reference. 5. Calculate and enter the SHA-256 digest. 6. Record the media type and producer generation. 7. Select **Register immutable reference**. When registration succeeds, the App returns to the same remediation case. The next step changes to independent validation and review; registration alone does not clear the gate. ## Register from the CLI [#register-from-the-cli] Create `artifact.json`: ```json { "engagementId": "eng_01J00000000000000000000000", "objectId": "obj_01J00000000000000000000001", "kind": "target_code", "name": "Repaired customer balance procedure", "artifactRef": { "system": "databricks", "type": "volume_object", "id": "/Volumes////repaired-customer-balance.sql" }, "digest": "YOUR_64_CHARACTER_SHA256", "mediaType": "application/sql", "toolVersion": "human-remediation@1" } ``` Then invoke the governed mutation: ```bash fa artifact register \ --file artifact.json \ --idempotency-key repaired-customer-balance-v1 fa artifact list --object-id obj_01J00000000000000000000001 ``` The idempotency key makes retries safe. The command records the same governed `airlift.artifact_register` action as the App. For the native ADF/Synapse path, emit strict v2 registration files alongside the generated Databricks files: ```bash fa migration-ir generate \ --file migration-ir.json \ --out-dir generated \ --engagement-id "$ENGAGEMENT_ID" \ --estate-id "$ESTATE_ID" \ --artifact-ref-prefix "/Workspace/Shared/airlift/generated" for registration in generated/registration/*.json; do registration_digest=$(sha256sum "$registration" | cut -d' ' -f1) fa artifact register \ --file "$registration" \ --idempotency-key "adf-$registration_digest" done ``` Only artifacts tied to deterministic IR nodes receive registration files. Repairable, human-only, excluded, and blocked nodes stay in remediation; generation never silently promotes them into governed artifacts. ## Complete a blocked repair from the CLI [#complete-a-blocked-repair-from-the-cli] Artifact registration is one step in a governed remediation, not the end of the repair. Use this sequence when the App says that a converted object is waiting for a repaired artifact: ```bash export AIRLIFT_API_URL="https://your-airlift-app.example" export DATABRICKS_TOKEN="$(databricks auth token --profile --output json | jq -r .access_token)" # 1. Read the exact object, reason automation stopped, current owner, and required skills. fa residue show "$RESIDUE_ID" --json # 2. Put the final repaired bytes in durable storage and calculate their digest. databricks fs cp repaired-output.sql \ "dbfs:/Volumes////repairs/repaired-output.sql" \ --profile --overwrite sha256sum repaired-output.sql # 3. Register the immutable provider reference and capture Airlift's artifact ID. fa artifact register \ --file artifact.json \ --idempotency-key "$RESIDUE_ID-artifact-v1" \ --json fa artifact list --object-id "$OBJECT_ID" --json # 4. Request validation against this object and artifact generation. fa validation run \ --file validation-request.json \ --idempotency-key "$RESIDUE_ID-validation-v1" fa validation list --engagement-id "$ENGAGEMENT_ID" --json # 5. After the admitted validation run passes, submit the artifact and evidence pair. fa residue resolve \ --file residue-resolution.json \ --idempotency-key "$RESIDUE_ID-resolution-v1" # 6. A different authorized natural person records the independent review. fa residue review \ --file residue-review.json \ --idempotency-key "$RESIDUE_ID-review-v1" ``` The CLI token is short lived. Do not save it in `artifact.json`, a shell history file, or the Airlift provider reference. The reference identifies the stored bytes; it is never a download credential. If a validation provider is not installed, step 4 fails closed. A developer may still register the artifact and inspect it in the App, but must not substitute a ticket, local test result, or caller-authored `passed` value for admitted validation evidence. ## What appears in the App [#what-appears-in-the-app] Open the engagement and select **Artifacts** to see the readable name, kind, object, provider reference, SHA-256 digest, media type, producer generation, normalized construct, declared route, warnings, registry corroboration, and downstream validation or release requirements. Open **Run ledger** to trace the governed registration and later validation execution. Return to **Resolve conversion issues** to see the next required action. The case advances only after the stored artifact, admitted validation, resolution, and independent review refer to the same governed object. The shared App/API/CLI parity field set is `artifactId`, `kind`, `construct`, `automationDisposition`, `targetPattern`, `digest`, `mediaType`, `toolVersion`, and `workspaceEvidence`. JSON uses these exact names. Text output renders the same values in that order; legacy artifacts render visible unclassified placeholders instead of invented construct or routing data. ## What happens next? [#what-happens-next] The assigned engineer requests an Experiments-backed validation run against this exact digest. After passing evidence is admitted, the engineer submits the remediation case for independent review. A different authorized principal approves or rejects it. Only then can the conversion gate advance. See [Engineering remediation](/docs/migration/residue) for the complete state machine and [Build and run validation suites](/docs/migration/validation) for the validation request, provider contract, and result inspection commands. See [Conversion factory](/docs/migration/conversion) for batch and attempt lineage. ## LLM-readable documentation [#llm-readable-documentation] The concise documentation index is available at [`/llms.txt`](https://airlift.fabric.pro/llms.txt). The complete developer corpus, including this guide and every code example above, is available at [`/llms-full.txt`](https://airlift.fabric.pro/llms-full.txt). Individual pages also expose Markdown through the **Copy Markdown** control. These exports contain public developer guidance only; deployment identifiers, client data, credentials, and internal evidence ledgers are intentionally excluded. # Run the assessment # Run the assessment [#run-the-assessment] The assessment establishes the accepted scope for every later migration step. A worker starts the governed assessment, the Lakebridge adapter submits the pre-provisioned workspace job, and Airlift records the job run, report, inventory, dependency digests, and pinned tool version. Use the authenticated CLI to inspect the lifecycle: ```bash fa assessment list --estate-id est_01ARZ3NDEKTSV4RRFFQ69G5FAV --json fa assessment status asm_01ARZ3NDEKTSV4RRFFQ69G5FAV --json ``` ## Build the source command [#build-the-source-command] Do not hard-code one assessment command for every source. Read the installed profile: ```ts import { createSourceMigrationPlan } from '@fabricorg/airlift'; const assessment = createSourceMigrationPlan('oracle').steps.find( (step) => step.id === 'assess', ); if (!assessment) throw new Error('assessment phase is missing'); console.log(assessment.lakebridgeCommands); console.log(assessment.airliftActions); ``` The CLI exposes the same information without writing code: ```bash fa source plan oracle --json | jq '.steps[] | select(.id == "assess")' ``` For Synapse/ADF construct-level routing, inspect the same versioned catalog consumed by the assessment compiler: ```bash fa source constructs synapse --variant synapse_dedicated_sql --json ``` The assessment lane, Migration IR disposition, and artifact-admission checks derive from that catalog rather than separate hard-coded tables. Its `implementationStatus` says which repository behavior exists (`routing_only`, `descriptor_only`, or `native_generator`); none of those values is validation evidence or client support proof. Your worker runs the listed Lakebridge Profiler/Analyzer command in a pre-provisioned Databricks workspace job. It then invokes `airlift.assessment_record` with immutable report, inventory, and dependency references. The repository's Lakebridge adapter owns job submission and output parsing; the action owns state. ## Required inputs [#required-inputs] * an Airlift estate with an opaque source connection reference; * Databricks App service-principal access to the configured analyzer job; * `AIRLIFT_LAKEBRIDGE_ANALYZER_JOB_ID` and volume root; * the certified Lakebridge version pin; * accepted source scope and exclusions. ## Acceptance review [#acceptance-review] The source owner and migration lead review object counts, dependency coverage, unsupported constructs, complexity assignment, and exclusions. The result must be tied to the source version and observation time. A fixture or local stub proves the adapter contract only; it does not prove a production estate. Acceptance is a separate `airlift.assessment_accept` action. It requires a complete, accepted dependency graph and exact normalized object count, then computes the scope digest from projected objects rather than trusting a caller-authored result. See the [assessment-to-plan tutorial](/docs/getting-started/assessment-to-plan). In the Databricks App, open **Discover → Assessment studio** to review the same runs, digests, inventory, graph, blockers, acceptance, and export controls. When the estate has more than one assessment, the studio compares it with the immediately preceding snapshot. It displays changes in tables, views, routines, ETL jobs, and other objects alongside inventory/dependency digest drift. A missing digest is a review signal; it is never interpreted as an unchanged estate. ## Failure behavior [#failure-behavior] Job failure, version drift, missing output, malformed report, or digest mismatch leaves the estate unassessed. Retrying reuses the logical assessment identity where appropriate; it never fabricates a successful record from partial output. # Build a conversion factory # Build a conversion factory [#build-a-conversion-factory] Airlift does more than track a checklist. It gives your conversion worker a governed execution model: accepted scope becomes a batch; every object gets a reproducible attempt; every successful output gets an immutable artifact identity; every failure becomes visible remediation work. The converter still does the translation. Lakebridge is the deterministic converter, and a bounded Harness agent may propose one repair candidate for admitted residue. Airlift coordinates those tools and prevents their success response from being mistaken for proof of migration readiness. Converter success is an attempt outcome, not a readiness verdict. Airlift requires a matching immutable target artifact before a converted object can satisfy batch reconciliation, and it requires independent validation before certification. ## Runtime model [#runtime-model] ```text accepted assessment │ ▼ conversion batch ──pins── object IDs + method + tool generation │ ├── conversion attempt ──success── immutable target artifact │ │ │ └── independent validation │ └── conversion attempt ──failure── residue case ├── bounded agent candidate └── human engineering ``` `ConversionBatch` is coordination state. It does not contain source code and it does not replace the object ledger. `Artifact` is an immutable reference plus a SHA-256 digest; the artifact body stays in your admitted repository, Unity Catalog volume, or release store. `Conversion` is one attempt against one object. Retrying creates another attempt, which preserves the history engineers need when comparing tool generations. The CLI maps directly to Platform actions: batch create/start/complete invoke `airlift.conversion_batch_create`, `airlift.conversion_batch_start`, and `airlift.conversion_batch_complete`; attempt start/record invoke `airlift.conversion_start` and `airlift.conversion_record`; artifact registration invokes `airlift.artifact_register`. ## 1. Create a batch from accepted scope [#1-create-a-batch-from-accepted-scope] Only objects from the accepted assessment are eligible. Objects must be `planned` or `rework`, and every object in a batch must belong to the same estate and assessment. Create `batch-create.json`: ```json { "engagementId": "eng_01J00000000000000000000000", "assessmentId": "asm_01J00000000000000000000000", "name": "Stored code batch 1", "method": "lakebridge", "toolVersion": "lakebridge@YOUR_PINNED_GENERATION", "objectIds": [ "obj_01J00000000000000000000001", "obj_01J00000000000000000000002" ] } ``` ```bash fa conversion batch create \ --file batch-create.json \ --idempotency-key stored-code-batch-1 fa conversion batch list \ --engagement-id eng_01J00000000000000000000000 ``` Use `lakebridge` for the deterministic pass, `agent` only for an admitted bounded repair pass, and `manual` for engineer-authored conversion. The method is evidence; do not label human work as deterministic conversion. ## 2. Start the batch [#2-start-the-batch] Create `batch-start.json`: ```json { "conversionBatchId": "cbh_01J00000000000000000000000" } ``` ```bash fa conversion batch start \ --file batch-start.json \ --idempotency-key stored-code-batch-1-start ``` Starting admits attempts for the pinned object set. `conversion_start` rejects an object, method, or tool generation that differs from the running batch. This prevents a worker configuration change from silently mixing outputs inside one batch. ## 3. Record one attempt per object [#3-record-one-attempt-per-object] Create `attempt-start.json`: ```json { "objectId": "obj_01J00000000000000000000001", "batchId": "cbh_01J00000000000000000000000", "method": "lakebridge", "toolVersion": "lakebridge@YOUR_PINNED_GENERATION" } ``` ```bash fa conversion attempt start \ --file attempt-start.json \ --idempotency-key stored-code-batch-1-object-1-start ``` After the adapter writes the output to your artifact store, create `attempt-record.json`: ```json { "conversionId": "cnv_01J00000000000000000000000", "objectId": "obj_01J00000000000000000000001", "outcome": "converted", "artifactRef": "volume://catalog/schema/migration/target/view.sql", "inputDigest": "INPUT_SHA256", "outputDigest": "OUTPUT_SHA256", "validationRunRef": "experiments-run-reference" } ``` ```bash fa conversion attempt record \ --file attempt-record.json \ --idempotency-key stored-code-batch-1-object-1-result fa conversion list --object-id obj_01J00000000000000000000001 fa conversion diff obj_01J00000000000000000000001 --json ``` `convert diff` returns the object's attempt history for machine comparison. Compare `inputDigest`, `outputDigest`, `toolVersion`, `modelVersion`, `promptVersion`, and `validationRunRef`; fetch code bodies from the referenced artifact store rather than from Airlift. ## 4. Register the immutable output [#4-register-the-immutable-output] Converted attempts must have a matching `target_code` artifact before the batch can complete. Create `artifact.json`: ```json { "engagementId": "eng_01J00000000000000000000000", "objectId": "obj_01J00000000000000000000001", "kind": "target_code", "name": "Converted customer view", "artifactRef": { "system": "databricks", "type": "volume_object", "id": "catalog/schema/migration/target/view.sql", "digest": "OUTPUT_SHA256" }, "digest": "OUTPUT_SHA256", "mediaType": "application/sql", "toolVersion": "lakebridge@YOUR_PINNED_GENERATION" } ``` ```bash fa artifact register \ --file artifact.json \ --idempotency-key artifact-customer-view-v1 fa artifact list --object-id obj_01J00000000000000000000001 ``` Register a new artifact with `supersedesArtifactId` when content changes. Never mutate a previous artifact row. Airlift rejects a reference digest that differs from the declared artifact digest. ## 5. Route failed attempts [#5-route-failed-attempts] Record a failed attempt with an actionable diagnostic. Then open a residue case before completing the batch: The CLI invokes the governed `airlift.residue_create` action; it does not insert a work item or mutate the object projection directly. ```bash fa residue create \ --file residue-create.json \ --idempotency-key object-2-residue-1 ``` See [Engineering remediation](/docs/migration/residue) for the full state machine. A failed object without active residue blocks batch completion; failure cannot disappear into a log file. ## 6. Reconcile batch completion [#6-reconcile-batch-completion] ```bash fa conversion batch complete \ --file batch-start.json \ --idempotency-key stored-code-batch-1-complete ``` Airlift derives the result instead of trusting caller-supplied counts: * every scoped object needs a terminal attempt; * every converted attempt needs a matching target artifact digest; * every failed attempt needs an active residue case; * every hazardous converted attempt needs its exact-origin `conversion_hazard` case; * a mixture of clean and hazardous/failed objects becomes `completed_with_residue`; * all failed objects becomes `failed`. Hazardous conversions count in the residue population, never the converted population. Completion does not certify parity. Independent validation and readiness policies still decide whether an object can receive a migration certificate. ## Conversion hazards: when "converted" is not safe [#conversion-hazards-when-converted-is-not-safe] A deterministic converter can report success while the output is invalid on Databricks or silently lossy. For qualified source profiles, Airlift rescans every converted artifact and records a digest-sealed **hazard assessment** on the attempt. Assessments carry hazard codes and artifact-relative locations — never SQL text. The Teradata catalog for the exact `lakebridge@0.14.2/sqlglot` engine covers five codes: | Hazard code | Meaning | | -------------------------------------- | ------------------------------------------------------------------------- | | `teradata.primary_index_retained` | `PRIMARY INDEX` retained verbatim — not valid Databricks SQL | | `teradata.column_format_retained` | Teradata column `FORMAT '…'` retained inside `CREATE TABLE` | | `teradata.sample_clause_dropped` | `SAMPLE n` collapsed to an empty `TABLESAMPLE ()` — silently lossy | | `teradata.set_table_semantics_dropped` | `CREATE SET TABLE` duplicate-row elimination lost | | `teradata.update_from_merge_review` | `UPDATE … FROM` rewritten to a multi-table `MERGE … USING` needing review | An attempt is **unassessed**, **scan clean**, or **converted with hazards** — and unassessed is never treated as clean. `fa conversion list` and `fa conversion show` render the state; the raw assessment travels with `--json` output. Findings force a `conversion_hazard` residue case bound to the exact conversion and assessment digest (its `origin`). The `airlift.conversion_hazard_gate.v1` policy blocks parity certification, migration-certificate minting, and cutover until that exact case is resolved with a repaired `target_code` artifact and independently reviewed — reviewing an older attempt's case never unlocks a newer attempt. A denied certification request stays denied under its original idempotency key: after the review is approved, issue the certification again with a **fresh idempotency key**. ## Bounded repair agent [#bounded-repair-agent] The Harness repair agent accepts source SQL, its digest, source dialect, and the deterministic failure. It has no tools, commands, filesystem access, or network access; it produces at most one typed candidate under a fixed source-size, iteration, timeout, retry, and cost bound. Persist that candidate as an artifact, then validate it through Experiments. An agent can create, estimate, and resolve an assigned residue with evidence. It cannot assign commercial work or approve its own resolution. Before promoting a repair strategy, evaluate a versioned dataset in Experiments and pass the digested summary to `evaluateRepairPromotion`. Promotion fails closed on sample size, schema validity, compile rate, parity rate, unsafe-output rate, or regression thresholds. ## Databricks App workflow [#databricks-app-workflow] Open **Conversion** in the Airlift App to create/start/reconcile batches, inspect attempt and artifact lineage, and register artifact references. Open **Remediation** for failed objects. The App and CLI invoke the same action definitions, policies, idempotency rules, event log, and tenant-scoped projections. # Production cutover and rollback # Production cutover and rollback [#production-cutover-and-rollback] Airlift does more than retain a cutover checklist. It computes a production gate from the current migration ledger and then orchestrates the external switch through a durable workflow. The gate fails closed when scope, certificates, transfer reconciliation, release evidence, rehearsal, operational evidence, approvals, or the effector binding does not match. The cutover control room and `fa cutover status` show the same materialized control record. UI state is never the gate. ## Lifecycle [#lifecycle] | Stage | What Airlift records or executes | Required authority | | ---------------------- | -------------------------------------------------------------------------------------------------------- | ---------------------------------------- | | freeze | exact object IDs, active certificate digests, transfer IDs, deployment requirement IDs, and release refs | operator | | timed runbook | versioned steps, offsets, owners, durations, and explicit rollback links | operator | | rehearsal | independently produced evidence for the exercised runbook steps | admitted automation | | effector certification | implementation identity, complete capability set, evidence digest, and expiry | separate human certifier | | parallel run | SLO verdict from Radar or an admitted external monitor | admitted automation | | approval | distinct human approvals under the organization policy | approver | | execute | checkpoint, apply once, verify, and compensate when verification proves a failed effect was applied | authenticated operator plus worker | | hypercare | operational evidence, incidents, acceptance, and source disposition | operator, monitor, and separate acceptor | Any object reassignment, certificate invalidation, transfer change, or deployment requirement drift makes the frozen scope stale. Refreeze a new wave revision; do not edit the prior digest. ## Implement the cutover effector [#implement-the-cutover-effector] Install the worker package and implement the exported interface: ```bash npm install @fabricorg/airlift-worker ``` ```ts import type { CutoverEffector } from '@fabricorg/airlift-worker'; export const effector: CutoverEffector = { profileId: 'airlift.cutover.customer-routing.v1', implementationVersion: 'customer-routing@1.4.2', certificationDigest: process.env.CUTOVER_EFFECTOR_CERTIFICATION_DIGEST!, async createCheckpoint({ waveId, idempotencyKey }) { return { checkpointRef: await checkpointRouting(waveId, idempotencyKey) }; }, async applyCutover({ waveId, checkpointRef, idempotencyKey }) { return applyRoutingOnce({ waveId, checkpointRef, idempotencyKey }); }, async verifyCutover({ waveId, checkpointRef, effectRef, idempotencyKey }) { return observeRouting({ waveId, checkpointRef, effectRef, idempotencyKey }); }, async compensateCutover({ waveId, checkpointRef, effectRef, idempotencyKey }) { return restoreRouting({ waveId, checkpointRef, effectRef, idempotencyKey }); }, }; ``` The three identity fields must equal the active `effector_certification_record` for the wave. The certification must be unexpired and prove all four capabilities: `checkpoint`, `apply_once`, `verify`, and `compensate`. Create the worker with the real effector: ```ts import { createAirliftWorker } from '@fabricorg/airlift-worker'; await createAirliftWorker({ mode: 'temporal', runtime, effector }); ``` Temporal mode refuses to start with the stub. The apply and compensate activities each have one attempt. Retries are safe only around idempotent reads and governed mutations. ## Verification outcomes [#verification-outcomes] Return one of these outcomes from `verifyCutover`: | Outcome | Workflow behavior | | ---------------- | --------------------------------------------------------------------------------------- | | `verified` | record success and move the wave to completed | | `not_applied` | record failure; do not infer or retry the external switch | | `failed_applied` | record failure, compensate once from the checkpoint, then record governed rollback | | `uncertain` | leave the wave executing for manual reconciliation; do not claim completion or rollback | An exception after the apply request is also uncertain because the external system may have accepted the request. Airlift never guesses. ## Start and observe the workflow [#start-and-observe-the-workflow] After the governed evidence and approvals are present: ```bash fa cutover start wav_01J00000000000000000000000 \ --window 2030-09-14T02:00Z \ --reason "Approved customer change window" \ --json fa cutover workflow-status airlift-v2-wav_ --json ``` Duplicate starts for the same wave and window attach to the same workflow ID. A payload-free wake can accelerate the next readiness read after an approval: ```bash fa cutover wake airlift-v2-wav_ ``` The wake is not an approval. It carries no actor, organization, decision, or evidence. ## Hypercare and source disposition [#hypercare-and-source-disposition] Hypercare starts only after the wave is completed and independent cutover-verification evidence is present. Monitors continue submitting `hypercare` observations. Open incidents block acceptance. A separate human records `accepted` or extends the observation window, and a different human attests the final source disposition. See [cutover CLI](/docs/cli/cutover) for request files and [cutover control room](/docs/operations/cutover-control-room) for the operator UI. # Implement the migration lifecycle # Implement the migration lifecycle [#implement-the-migration-lifecycle] Airlift is the system of record and assurance plane around migration tools. Each phase has an executable integration boundary and a governed exit condition. Start by generating the source-specific version of this lifecycle: ```bash fa source plan fa source plan --json > .airlift/plan.json ``` | Phase | Developer integration | Governed result | | ---------------------------------------------------------------------------- | ----------------------------------------------------------------------- | --------------------------------------------------------------------------------------- | | [Assess](/docs/migration/assessment) | Lakebridge Profiler/Analyzer job adapter | accepted estate run and inventory/dependency digests | | [Inventory](/docs/migration/inventory) | normalizer plus dependency builder | scoped objects, exclusions, complexity, residue lane | | [Convert](/docs/migration/conversion) | Morpheus/BladeBridge job and optional bounded repair | immutable conversion attempt and artifact digest | | [Transfer](/docs/migration/transfer) | source-specific checkpointed driver | watermark, lag, restart, counts, reconciliation | | [Validate](/docs/migration/validation) | Reconcile, Experiments testkit, or admitted runner | provider run plus object-profile readiness observations | | [Cut over](/docs/migration/cutover) | Temporal worker plus certified effector | approvals, checkpoint, apply-once, verify, rollback evidence | | [Modernize](/docs/migration/modernization) | separate Runway release or project backlog | native Databricks disposition without rewriting parity history | | [Build an operational application](/docs/migration/operational-applications) | Databricks Apps, Lakebase, Synced Tables, Platform, and family handoffs | portable application-kit plan, synthetic BDD, preview/restore, and same-digest evidence | ## Runtime rule [#runtime-rule] Workers and server handlers do not write projected state. They invoke the action named in the source plan through `runtime.invokeAction(...)` with an authenticated tenant and principal. Adapters return immutable references, digests, versions, verdicts, and watermarks; Airlift validates and records those claims. ## Definition of done [#definition-of-done] The success metric is the percentage of in-scope objects deployed and certified against their object-type evidence profile, with provenance and a tested rollback path. “Converted” is an intermediate state, not a cutover claim. # Inventory and dependencies # Inventory and dependencies [#inventory-and-dependencies] Airlift normalizes the assessment report into tables, views, stored procedures, functions, ETL jobs, notebooks, reports, semantic models, ML assets, security objects, and external dependencies. Every object carries source identity, type, complexity, and dependencies sufficient for planning. The normalized report is content-digested so a scope change is observable. ## Normalizer output [#normalizer-output] Map each upstream record into an Airlift object input with a stable source key, object type, source path/name, complexity, dependency keys, and source profile. Register it with `airlift.object_register`; record intentional omissions with `airlift.object_exclude` and a reviewable reason. Use the source profile's `workloadSurfaces` as the inventory checklist: ```bash fa source inspect hadoop --json | jq '.workloadSurfaces[]' ``` This prevents a SQL-only inventory from silently losing jobs, schedulers, notebooks, security objects, BI consumers, utilities, or external integrations. ## Record the graph [#record-the-graph] The executable graph lifecycle is: 1. `airlift.dependency_graph_start` declares the assessment, immutable graph reference, tool generation, and expected edge count. 2. `airlift.dependency_batch_record` accepts up to 500 typed edges per idempotent call. 3. `airlift.dependency_graph_accept` verifies completeness and computes the canonical graph digest and cycle count. ```bash fa inventory graph list --assessment-id asm_01ARZ3NDEKTSV4RRFFQ69G5FAV fa inventory graph show dpg_01ARZ3NDEKTSV4RRFFQ69G5FAV --json ``` An edge expresses `fromObjectId` depends on `toObjectId`. Kinds are `data`, `execution`, `schedule`, `security`, and `consumer`; confidence and critical-path status remain visible in the projection. ## Automation lanes [#automation-lanes] Classify residue early: * **deterministic** — Lakebridge has a supported conversion path; * **agent-repairable** — deterministic residue fits the bounded repair input/output contract; * **human-only** — semantics, source behavior, or risk requires an engineer. Human-only objects remain in scope and pricing unless explicitly excluded through the governed action. They are not silently removed from an automation denominator. ## Dependency rules [#dependency-rules] Plan upstream data and shared functions before dependent objects. Cycles, external dependencies, dynamic SQL, and late-bound orchestration should be surfaced as blockers or explicit design work. Wave assignment must not make the dependency graph disappear; it records the migration decision made against it. # Modernization studio # Modernization studio [#modernization-studio] Airlift turns modernization into an evidence-gated release loop. It does not append an untracked “optimization” checklist to the migration. Each recommendation is a governed item with a certified migration baseline, a human decision, a Runway release reference, an Experiments comparison, and a final human promotion. Use this after the migrated object has an active migration certificate. Keeping the baseline and native release separate tells you whether a discrepancy came from migration or from an intentional design change. ## Lifecycle [#lifecycle] | State | Meaning | Who can advance it | | ---------------------------------- | ----------------------------------------------------------------------------- | -------------------------------------- | | `proposed` | A rule, developer, or bounded Harness advisor suggested a native change. | Agent, system, or authorized developer | | `accepted`, `deferred`, `rejected` | A person made the investment decision. | Approver | | `planned` | Effort and an immutable Runway release-intent reference are recorded. | Operator | | `implementing` | The separate target artifact is being built. | Operator | | `validating` | Runway deployment reconciled and every declared Experiments guardrail passed. | Admitted worker records evidence | | `promoted` | A different person promoted the evidence-backed release. | Promoter | | `rework` | Evidence failed or changed and another implementation pass is required. | Operator or evidence admission | Agents cannot decide, promote, waive, or publish value claims. Airlift may require state owned by another Fabric product, but stores only its immutable reference and digest. ## Developer workflow [#developer-workflow] List recommendations for one engagement: ```bash fa modernization list --engagement-id "$ENGAGEMENT_ID" --json fa modernization show "$MODERNIZATION_ITEM_ID" --json ``` Record a recommendation from a rule or developer. The baseline certificate must be active and belong to the same object and estate: ```json title="recommendation.json" { "engagementId": "eng_...", "estateId": "est_...", "objectId": "obj_...", "baselineCertificateId": "mct_...", "category": "liquid_clustering", "title": "Evaluate liquid clustering", "rationale": "Compare a target-native layout with the certified baseline.", "expectedValue": "Measure representative query latency and maintenance effort.", "outcomeTargets": [ { "metric": "performance", "direction": "decrease", "target": 15, "unit": "percent" } ], "priority": "high", "risk": "medium", "recommendationSource": "human", "recommendationRef": { "system": "engineering", "type": "design", "id": "DES-42", "digest": "" }, "recommendationDigest": "" } ``` ```bash fa modernization recommend \ --file recommendation.json \ --idempotency-key "modernization-recommend-$OBJECT_ID" ``` Disposition, plan, and start use the same file-command pattern: ```bash fa modernization decide --file decision.json --idempotency-key "modernization-decision-$MODERNIZATION_ITEM_ID" fa modernization plan --file plan.json --idempotency-key "modernization-plan-$MODERNIZATION_ITEM_ID" fa modernization start --file start.json --idempotency-key "modernization-start-$MODERNIZATION_ITEM_ID" ``` `plan.json` must include `effortMinutes` and a digested foreign reference whose `system` is `runway`. The Runway CLI (`fr`) builds, deploys, promotes, and rolls back the artifact; the Airlift CLI (`fa`) records why that release is required by the migration program. ## A/B evidence contract [#ab-evidence-contract] After the candidate is deployed, run the certified baseline and modernization treatment through Experiments. Functional parity is a guardrail; outcome metrics measure whether the change achieved its stated goal. ```json title="modernization-evidence.json" { "modernizationItemId": "mod_...", "targetArtifactId": "art_...", "deploymentRequirementId": "dpr_...", "validationExecutionId": "vex_...", "validationEvidenceRef": { "system": "experiments", "type": "evidence_manifest", "id": "manifest-42", "digest": "" }, "validationEvidenceDigest": "", "comparison": { "experimentRef": { "system": "experiments", "type": "experiment", "id": "exp-42" }, "evaluationRef": { "system": "experiments", "type": "evaluation", "id": "eval-42", "digest": "" }, "evidenceDigest": "", "baselineTreatment": "certified-migration", "modernizationTreatment": "liquid-clustering", "guardrailVerdict": "passed", "metrics": [ { "name": "performance", "baseline": 10, "modernized": 7, "unit": "seconds", "direction": "decrease", "verdict": "passed" } ] }, "reason": "Functional guardrails and the declared performance target passed." } ``` ```bash fa modernization evidence --file modernization-evidence.json --idempotency-key "modernization-evidence-$MODERNIZATION_ITEM_ID" fa modernization promote --file promotion.json --idempotency-key "modernization-promote-$MODERNIZATION_ITEM_ID" ``` Evidence admission fails unless the Runway deployment is reconciled as matched and succeeded, the Experiments validation completed without failed objects, every comparison metric passed, and reference digests agree. A promoter must be distinct from both the recommender and the disposition decision-maker. ## Categories [#categories] Airlift supports Lakeflow, serverless, Unity Catalog, Delta optimization, liquid clustering, materialized views, streaming, data products, AI/BI, MLflow, model serving, and governance recommendations. A category describes the intended change; it does not claim that every source object should receive that change. Next: [Measure engagement value](/docs/operations/value-measurement). # Operational application modernization # Operational application modernization [#operational-application-modernization] A migration can expose a second opportunity: the source estate may contain operational workflows that should become a Databricks-native application. Examples include customer service, agent review, compliance cases, asset operations, and the migration workbench itself. Airlift helps engineering teams define that application as an executable delivery contract instead of leaving it as a slide or backlog item. It answers: * which operational state belongs in Lakebase; * which governed lakehouse data must be served through Synced Tables; * which Databricks App resources must be bound at runtime; * which domain schemas, grants, fixtures, and user journeys are required; * which validation, preview, restore, deployment, and SLO evidence blocks promotion; and * which Fabric product owns each execution step. ## Architecture [#architecture] ```text Unity Catalog governed data -> Synced Tables or admitted movement contract -> Lakebase serving tables Databricks App -> valueFrom runtime bindings -> Lakebase transactional application state -> Platform governed actions -> Harness agents when the domain needs bounded automation Experiments BDD/evaluations + Runway release + Radar observations -> foreign references and exact digests -> Airlift application-kit qualification ``` Lakebase is not used as a substitute for the lakehouse. Lakebase holds transactional application state and low-latency serving projections; Unity Catalog remains the governed source for analytical data, permissions, lineage, and data ownership. ## End-to-end developer journey [#end-to-end-developer-journey] ### 1. Capture the application opportunity [#1-capture-the-application-opportunity] ```bash fa application-kit init \ --name "Risk review" \ --module risk_compliance \ --cloud aws \ > application-kit.json ``` Edit logical bindings, Synced Table contracts, module artifact references, BDD feature references, and the Radar SLO profile. Keep credentials and physical workspace resource IDs in environment-owned Databricks App resources, never in the manifest. ### 2. Compile the family delivery contract [#2-compile-the-family-delivery-contract] ```bash fa application-kit plan \ --file application-kit.json \ --json > application-plan.json ``` The plan expands module behavior into observable acceptance scenarios. A risk and compliance module requires case-state transitions, separation of duties, policy retrieval quality, and immutable decision evidence. Common scenarios cover authenticated and unauthorized users, tenant isolation, governed mutation denial, binding failure, data freshness/recovery, and branch preview/restore. ### 3. Build the application release [#3-build-the-application-release] Implement the application vertical with Platform actions and Harness agents. Resolve all Databricks resources through App bindings. Generate or assemble the Declarative Automation Bundle as a Runway release input. Use `fr`, not `fa`, for the release: ```bash fr validate --dir generated fr deploy --dir generated --environment preview ``` The preview environment uses an isolated Lakebase branch or an equivalently isolated state store. Run the restore rehearsal before promotion. ### 4. Execute behavior and quality evidence [#4-execute-behavior-and-quality-evidence] Write synthetic BDD for every advertised workflow and negative authorization path. Execute it through Fabric Experiments against the same candidate digest: ```bash fx apply experiments/ fx report ``` Agent modules also need versioned datasets, evaluators, budgets, tool-denial tests, and release thresholds. A passing agent answer cannot approve, waive, deploy, certify, or change application state outside Platform. ### 5. Qualify the joined evidence [#5-qualify-the-joined-evidence] ```bash fa application-kit qualify \ --file application-plan.json \ --evidence application-evidence.json \ --level workspace_proven ``` The decision fails closed unless Runway and Experiments identify the same artifact digest. It also requires branch preview/restore evidence and a Radar observation when the kit has an SLO. Provider outages, cross-scope references, digest drift, and failed scenarios are blockers, not warnings. ### 6. Promote, operate, and measure separately [#6-promote-operate-and-measure-separately] Runway promotes or rolls back the release. Radar owns operational health. Airlift may admit their references into a modernization item and measured-value report, but it does not duplicate their state. Measure the application against a reviewed baseline: user handling time, queue time, error/rework rate, latency, cost, or another agreed outcome. Do not publish an acceleration percentage from a synthetic run or a single engagement. ## Proof levels [#proof-levels] | Level | Required evidence | | ---------------------- | ------------------------------------------------------------------------------------------------ | | `contract_only` | valid manifest, deterministic plan, explicit owners | | `hermetic_proven` | bundle validation, secret scan, binding evidence, synthetic fixtures, BDD contract | | `workspace_proven` | same-digest Runway deployment, Experiments execution, branch restore, required Radar observation | | `client_proven` | representative client data and accepted business journeys | | `production_certified` | production rollout, rollback, SLO, security, and support evidence | Each level is a ceiling. It does not imply the next level. ## Where to work in the App [#where-to-work-in-the-app] Open **Application kits** to see available modules, registered immutable plans, the developer command flow, and the evidence join. Open **Release qualification** for the same-digest release boundary and **Modernization studio** to connect an accepted kit to a separate modernization release and measurable outcome. See [Databricks application-kit commands](/docs/cli/application-kits) for the complete CLI contract. # Pipeline and code modernization # Pipeline and code modernization [#pipeline-and-code-modernization] A warehouse migration is incomplete when its orchestration stays behind. Airlift treats pipeline graphs, scripts, parameters, schedules, retry behavior, checkpoints, external effects, and downstream consumers as migration objects with the same provenance and evidence discipline as tables and SQL code. ## End-to-end flow [#end-to-end-flow] | Stage | What you do | What Airlift produces or governs | | ---------------- | ----------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- | | Export | Export source metadata without credentials | Native source input or a normalized, content-digested manifest | | Import and route | Run `fa migration-ir import` for ADF, or supply a normalized manifest for another profile | Lossless IR, dependency order, dispositions, provenance, and remediation packs | | Generate | Run `fa migration-ir generate` for a native generator, or implement the declared targets | Concrete file bodies for ADF; explicit descriptor-only status for other profiles | | Verify bytes | Run `fa migration-ir validate` | Artifact-set digest plus per-file content and byte-length verification | | Implement | Send bounded cases to Harness; assign human cases; review replacement artifacts | Immutable artifact references and an auditable repair history | | Validate | Execute representative data and control-flow scenarios in Experiments | Independent verdicts for data, branch, retry, failure, and restart behavior | | Deploy | Register the requirement in Airlift; deploy the immutable release with Runway | Preview, release digest, promotion, reconciliation, and rollback reference | | Certify | Evaluate object-specific readiness | Signed certificate that identifies exactly what was tested | | Modernize | Create a separate native redesign release | Measured Lakeflow, serverless, cost, reliability, and operability improvements | ## Source-to-target mappings [#source-to-target-mappings] ### ADF and Synapse pipelines [#adf-and-synapse-pipelines] ADF is the first native export path. Airlift reads ARM or Git exports, resolves pipeline, dataset, trigger, and activity dependencies, and emits concrete Databricks Workflow, Python, BDD, configuration, and Asset Bundle files. One-input, one-output Copy activities produce a parameterized table materialization task. Mapping Data Flow, expression, condition, loop, and tumbling-window semantics enter bounded repair because source syntax alone cannot prove equivalent behavior. Web activities and other side effects require an engineer to design and approve the replacement. ### SSIS [#ssis] The current SSIS profile accepts a credential-free normalized manifest and classifies Data Flow Tasks, package execution, containers, SQL tasks, and Script Tasks. It produces target declarations and remediation packs, not DTSX parsing or file bodies. A native DTSX/ISPac importer and concrete generator must pass the same no-silent-drop contract before this profile is described as a compiler. ### Informatica PowerCenter [#informatica-powercenter] The current PowerCenter profile routes normalized mappings, source qualifiers, workflows, expressions, stored-procedure transformations, and external effects. Repository XML import and concrete Lakeflow/Workflow/PySpark generation remain separate adapter work. ### SAS [#sas] The current SAS profile routes normalized PROC SQL, DATA step, macro, schedule, X command, and host-side-effect nodes. It does not parse SAS source or claim semantic conversion. Native parsing, macro expansion, generated SQL/PySpark, and analytic/model equivalence remain required before SAS receives a compiler claim. ### DataStage, Talend, and ODI [#datastage-talend-and-odi] These are normalized-manifest routing profiles. They preserve known and unknown constructs, select deterministic, repairable, or human lanes, and state the intended Databricks target. They do not yet parse native exports or materialize those targets. ### dbt and Airflow [#dbt-and-airflow] These are normalized-manifest routing profiles. Native `manifest.json` and serialized-DAG importers, SQL/Python file preservation, and concrete Databricks job generation remain required. Hooks, Bash operators, and host-specific effects stay explicit human remediation. ## Behavior is the acceptance contract [#behavior-is-the-acceptance-contract] Syntax conversion is not completion. For each migrated unit, exercise: * representative input and output data; * null, decimal, timestamp, ordering, and duplicate behavior; * parameters, variables, branches, and loops; * schedule and trigger behavior; * retry limits and failure propagation; * checkpoint, restart, and duplicate-run behavior; * external side effects and apply-once constraints; and * runtime, throughput, and cost thresholds. Only independent admitted verdicts advance readiness. Compiler success and agent output are implementation evidence, not proof of parity. ## Migration release before modernization release [#migration-release-before-modernization-release] First preserve the source behavior needed for safe cutover. Then create a separate modernization item for native redesign: simplify orchestration, adopt serverless, consolidate pipelines, improve observability, or change the data product contract. This keeps the baseline certificate honest and makes modernization value measurable. Start with the [Migration IR command guide](/docs/cli/migration-ir), then use [Engineering remediation](/docs/migration/residue), [Validation](/docs/migration/validation), and the [Runway integration](/docs/integrations/runway). # Plan cutover waves # Plan cutover waves [#plan-cutover-waves] An accepted assessment can generate multiple migration scenarios before execution waves exist. `airlift.plan_generate` deterministically maps object types to Databricks target patterns, topologically groups dependency layers into candidate waves, identifies cycle blockers, and calculates effort/value from explicit inputs. ```bash fa plan generate --file plan.json --idempotency-key plan-parity-1 fa plan compare --engagement-id eng_01ARZ3NDEKTSV4RRFFQ69G5FAV --json fa plan select pln_01ARZ3NDEKTSV4RRFFQ69G5FAV --idempotency-key plan-select-1 fa plan freeze pln_01ARZ3NDEKTSV4RRFFQ69G5FAV --idempotency-key plan-freeze-1 ``` In the Databricks App, **Plan → Target blueprint and wave planner** exposes the same target services, mappings, prerequisites, estimates, selection, and freeze evidence. A wave is the unit of rehearsal, approval, cutover, and rollback. Group objects by dependency, business domain, data synchronization boundary, operational window, and rollback coupling—not merely by object count. ## Wave design [#wave-design] Each wave needs an owner, planned window, object assignments, validation expectations, approval policy, rollback procedure, and observation window. Keep a deliberately hard pilot wave: easy objects alone do not test the factory. ## Freeze digest [#freeze-digest] Before the final rehearsal, freeze object scope and the applicable profile, readiness, artifact, deployment, and source/target snapshot identities. A changed dependency or digest invalidates the old decision rather than being edited into it. ## Separation of duties [#separation-of-duties] The converter or agent cannot approve its own wave. Natural-person approvers must be distinct as configured by policy. Tower may track tasks, but task completion never advances Airlift state without the governed Airlift action. # Qualify a generated release # Qualify a generated release [#qualify-a-generated-release] Generation answers “what files did the compiler produce?” Release qualification answers “is this exact candidate ready for the requested proof level?” Airlift keeps those questions separate so a syntactically valid bundle cannot be mistaken for a tested migration. The release qualification path composes three owners: | System | Owns | Evidence Airlift consumes | | ------------------ | -------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | | Airlift | candidate lineage, runtime requirements, residue disposition, migration decision | registered artifact, deployment requirement, validation scope, derived qualification report | | Fabric Runway | staging, preview, deploy, gate, promotion, reconciliation, rollback | deployment ID, staged artifact digest, terminal state, observation digest | | Fabric Experiments | BDD, parity, security, performance, and cost execution | request, run, manifest, per-object verdict, artifact digest, watermarks | Airlift may block on Runway or Experiments. It never changes their release or test state. ## 1. Generate and verify the candidate [#1-generate-and-verify-the-candidate] ```bash fa migration-ir generate \ --file migration-ir.json \ --out-dir generated fa migration-ir validate \ --file generated/artifact-set.json \ --root generated ``` `validate` recomputes the artifact-set digest and verifies every materialized path, body, byte length, and SHA-256 digest. It rejects missing files, modified files, duplicate paths, and path traversal. ## 2. Bind runtime configuration [#2-bind-runtime-configuration] Generated code contains no credentials. Create a binding-evidence file that maps every required binding name to an immutable reference: ```json title="release-bindings.json" { "source_identifier": { "system": "databricks", "type": "unity_catalog_table", "id": "catalog.schema.source_table", "digest": "" }, "target_identifier": { "system": "databricks", "type": "unity_catalog_table", "id": "catalog.schema.target_table", "digest": "" }, "experiments_profile": { "system": "experiments", "type": "validation_profile", "id": "migration-pipeline-v1", "digest": "" } } ``` The file contains identifiers and digests, not tokens, passwords, connection strings, or secret values. A required binding without evidence blocks qualification. ## 3. Dispose every unresolved source node [#3-dispose-every-unresolved-source-node] The generated artifact set lists `unresolvedSourceIds`. Each one must remain visible until it has a reviewed disposition: ```json title="residue-resolutions.json" [ { "sourceId": "pipeline:Orders/activity:NotifyExternalSystem", "resolution": "remediated", "evidenceRef": { "system": "airlift", "type": "residue_review", "id": "", "digest": "" } } ] ``` Allowed decisions are `remediated`, `accepted_difference`, and `excluded`. Qualification rejects a missing disposition and a resolution for a node outside the candidate. ## 4. Prove the hermetic candidate [#4-prove-the-hermetic-candidate] ```bash fa migration-ir qualify \ --file generated/artifact-set.json \ --root generated \ --bindings release-bindings.json \ --resolutions residue-resolutions.json \ --proof hermetic_proven \ --output qualification.json ``` The command reruns byte verification and invokes `databricks bundle validate` in the materialized directory. Use `--databricks-cli ` when CI pins the Databricks CLI at a non-default path. A non-zero bundle validation, missing executable, missing binding, or unreviewed residue returns a non-zero `fa` exit code. Hermetic proof means the candidate is deterministic and locally executable against its synthetic contract. It does **not** mean the bundle was deployed, its business behavior is equivalent, or a client approved it. ## 5. Register the immutable candidate [#5-register-the-immutable-candidate] Store `artifact-set.json` in an admitted immutable store, then register its reference: ```bash fa migration-ir register \ --file generated/artifact-set.json \ --engagement-id "$ENGAGEMENT_ID" \ --estate-id "$ESTATE_ID" \ --artifact-id "$IMMUTABLE_ARTIFACT_REF" \ --idempotency-key "adf-release-${SOURCE_REVISION}" ``` This is the governed Airlift artifact. The ledger stores its reference and digest, not the generated file bodies. ## 6. Deploy with Runway [#6-deploy-with-runway] Runway stages and deploys the materialized directory: ```bash fr deploy --dir generated ``` Capture the Runway deployment ID and the staged Runway artifact digest. Add that immutable foreign reference to the Airlift requirement: ```json title="deployment-requirement.json" { "engagementId": "", "estateId": "", "operation": "deploy", "environment": "dev", "artifactIds": [""], "requiredState": "succeeded", "runwayRequestRef": { "system": "runway", "type": "deployment", "id": "", "digest": "" } } ``` ```bash fa deployment require \ --file deployment-requirement.json \ --idempotency-key "adf-release-${SOURCE_REVISION}-dev" ``` The Airlift worker reads Runway’s authenticated projection API. It rejects a missing deployment, organization mismatch, environment mismatch, staged-artifact digest drift, or non-terminal deployment. Only after those checks does it record and reconcile the Runway observation through governed Airlift actions. Configure the observer with: ```bash AIRLIFT_DEPLOYMENT_ADAPTER=runway AIRLIFT_RUNWAY_API_URL=https:// AIRLIFT_RUNWAY_API_TOKEN= ``` Supply the token through the deployment secret system; never put it in a bundle source, action payload, artifact set, or evidence document. ## 7. Execute behavioral validation [#7-execute-behavioral-validation] Create an Airlift validation request for the migrated objects and required readiness tracks: ```bash fa validation run \ --file validation-request.json \ --idempotency-key "adf-release-${SOURCE_REVISION}-validation" ``` The worker compiles object-specific checks and delegates them to Experiments. Workspace qualification requires: * one completed Experiments execution with no failed objects; * a passing Experiments run for every requested object; * every run bound to the candidate artifact-set digest; * immutable request, run, evidence-manifest, and per-object evidence references; and * source watermark and target snapshot identities. See [Fabric Experiments integration](/docs/integrations/experiments) for the executable case-module and evidence-store boundary. ## 8. Evaluate workspace proof [#8-evaluate-workspace-proof] Export the governed Airlift artifact, deployment requirement, validation execution, and validation runs, plus the Databricks workspace references asserted by the two service integrations. Put those rows in the `workspaceEvidence` contract consumed by `fa migration-ir qualify`. The contract has two versions. Validation runs recorded against an anchored conversion attempt carry a `conversionId`, and a document containing them must declare `"schemaVersion": 2` at its top level. Documents written before that field existed stay valid unchanged: an untagged document is read as version 1, whose runs carry no `conversionId`. Mixing the two — an attempt-bound run inside an untagged document — is refused rather than silently accepted, so a version-1 reader never receives version-2 content: ```bash fa migration-ir qualify \ --file generated/artifact-set.json \ --root generated \ --bindings release-bindings.json \ --resolutions residue-resolutions.json \ --proof workspace_proven \ --workspace-evidence workspace-evidence.json \ --output workspace-qualification.json ``` The command is a read-only decision tool. It cannot import a caller-authored success into the Airlift ledger. It verifies that: 1. artifact, deployment, and validation share one tenant, engagement, and estate; 2. the Airlift artifact and validation runs use the candidate digest; 3. the deployment is Runway-owned, terminal, and reconciled as matched; 4. Experiments classified every object exactly once with no failure; and 5. deployment and validation name the same Databricks workspace. If any check fails, `maximumProofLevel` remains `hermetic_proven` or `contract_only` and the report lists exact blockers. Workspace proof is still below client proof and production certification; those require representative client acceptance and current operational, rollback, signing, and cutover evidence. ## App workflow [#app-workflow] Open **Release qualification** in the Airlift Databricks App. The screen joins registered ADF artifact sets to their Airlift deployment requirements and Experiments executions. It shows the candidate digest, missing evidence, and deep links to Deployment requirements and Validation laboratory. Release actions remain in Runway, and test execution remains in Experiments. # Engineering remediation If a case is waiting for an immutable artifact, use the [artifact registration guide](/docs/migration/artifacts) for the exact kind, reference, digest, media type, and producer fields. # Engineering remediation [#engineering-remediation] A residue case turns a failed or incomplete conversion into managed engineering work. It records why automation stopped, which skills are needed, who owns the next step, any optional delivery forecast, which artifact contains the fix, and which independent validation run supports closure. This is how Airlift actively helps a migration: automation handles repeatable objects; specialists receive a prioritized, contextual queue for the work that requires judgment; engineering leadership can see remaining effort and commercial treatment without pretending every object was automated. ## Airlift Migration Assistant [#airlift-migration-assistant] On a focused blocker, select **Ask Migration Assistant**. The right-hand drawer starts with the exact residue state already recorded by Airlift, so a developer can ask: * What should I do next? * Why did automation stop? * What evidence clears this blocker? * How should I diagnose or validate this construct? The first answer always comes from the governed projection. It identifies the affected object, current residue state, accountable role, next valid control, and evidence needed to unlock the following step. This works even when no model provider is configured. When a model is configured, a finite Harness agent may explain the failure and suggest bounded diagnostic checks. The preferred provider is Unity AI Gateway or Databricks Model Serving through the App service principal. Airlift sends only a bounded, object-scoped context envelope and redacts common credential-shaped values. Never put credentials in residue text; keep secrets in Databricks secret resources. An optional Databricks Genie Agent can answer curated migration-analytics questions or return diagnostic SQL through user OBO. Airlift displays that SQL for review and never executes it automatically. Neither deterministic guidance nor the Migration Assistant can approve evidence, change readiness, waive policy, deploy, certify, or cut over. Use the highlighted Airlift control to register artifacts and decisions through the governed action pipeline. A model failure falls back to the ledger-derived answer; it never weakens the gate. ### Configure the Migration Assistant [#configure-the-migration-assistant] 1. Open **Advanced tools → Migration Assistant** and inspect the three readiness cards. 2. Prefer a Unity AI Gateway model service or Model Serving endpoint and grant the App service principal access. 3. Bind `AIRLIFT_GENIE_PROVIDER`, `AIRLIFT_GENIE_MODEL`, and the inference mode during deployment. For an external provider, bind `AIRLIFT_GENIE_API_KEY` from a Databricks secret resource; do not enter it in Airlift. 4. Ask a known blocker question and verify the response labels its provider. 5. Remove model access and verify the same question safely falls back to **Airlift ledger**. See the [Airlift Migration Assistant developer guide](/docs/integrations/airlift-genie) for exact bundle commands, external-provider settings, permissions, and failure behavior. ## State machine [#state-machine] | State | Meaning | Governed next action | | ----------- | -------------------------------------------------------------- | -------------------------------------- | | `open` | Issue is classified and needs an owner | assign, optionally estimate, or cancel | | `estimated` | Optional effort and skill forecast is recorded | assign, re-estimate, or cancel | | `assigned` | A person/team or work reference owns delivery | resolve or cancel | | `resolved` | A replacement artifact and validation reference were submitted | independent approve or reject | | `reviewed` | Independent reviewer accepted the evidence | terminal | | `cancelled` | Scope changed with an audit reason | terminal | A rejected review returns the case to `assigned` and clears the prior resolution fields. The assignee must submit a new artifact/evidence pair. The resolver cannot review the same case. Agent and system principals cannot assign commercial work or review closure. The state transitions are the Platform actions `airlift.residue_create`, `airlift.residue_estimate`, `airlift.residue_assign`, `airlift.residue_resolve`, `airlift.residue_review`, and `airlift.residue_cancel`. ## Conversion-hazard cases [#conversion-hazard-cases] The `conversion_hazard` category (residue contract v2) tracks converter output that was reported as converted but proven invalid or silently lossy by the qualified hazard scan — see [conversion hazards](/docs/migration/conversion#conversion-hazards-when-converted-is-not-safe) for the hazard codes. Each case carries a required typed `origin` (`conversionId`, `assessmentDigest`, `findingCodes`) binding it to **one exact conversion attempt and one exact scan result**; deduplication is origin-exact, so a new conversion attempt or re-scan opens a new case and an older reviewed case never unlocks newer output. The trusted worker opens these cases automatically; people assign, resolve (with a repaired `target_code` artifact for the same object), and review them. While the exact-origin case is not independently reviewed, `airlift.conversion_hazard_gate.v1` blocks parity certification, migration-certificate minting, and cutover for the object. A certification request denied by the gate stays denied under its original idempotency key: after the approved review, submit the certification again with a **fresh idempotency key** — replaying the old key returns the recorded denial, by design. ## 1. Create and classify [#1-create-and-classify] Create `residue-create.json`: ```json { "engagementId": "eng_01J00000000000000000000000", "objectId": "obj_01J00000000000000000000002", "category": "unsupported_construct", "lane": "human", "summary": "Rewrite dynamic SQL for Databricks SQL", "detail": "The deterministic pass cannot prove identifier construction semantics.", "requiredSkills": ["T-SQL", "Databricks SQL"], "commercialAttribution": "managed_service" } ``` ```bash fa residue create --file residue-create.json --idempotency-key object-2-residue-1 fa residue list --engagement-id eng_01J00000000000000000000000 ``` Categories are `unsupported_construct`, `compile_failure`, `validation_failure`, `missing_semantics`, `security_design`, `consumer_change`, `client_decision`, and `conversion_hazard` (worker-created with a required exact origin — see [conversion-hazard cases](#conversion-hazard-cases)). Use the `agent` lane only when the input fits the bounded repair contract. Use `human` when semantic judgment, client choice, security design, or cross-system work is required. Commercial attribution is explicit: `included`, `change_request`, `client_owned`, or `managed_service`. This field does not invoice a client; it gives delivery and commercial systems an auditable source for downstream reporting. ## 2. Assign an owner [#2-assign-an-owner] Assignment is the first required remediation action. A delivery lead can assign an open case directly; developers do not need to estimate project effort before beginning work. ```json { "residueId": "res_01J00000000000000000000000", "assignedToRef": { "system": "delivery_directory", "type": "team", "id": "sql-modernization" }, "workRef": { "system": "work_manager", "type": "work_item", "id": "MIG-142" } } ``` ```bash fa residue assign --file residue-assign.json --idempotency-key residue-1-assignment-v1 ``` The work manager owns task coordination. Airlift stores only the foreign reference. A task being marked complete never advances the residue state; closure requires the Airlift resolution and review actions below. ## Optional: record a delivery forecast [#optional-record-a-delivery-forecast] Effort is planning metadata owned by a delivery lead. It supports staffing, commercial reporting, and estimate-versus-actual analysis, but it does not clear a technical gate. Record it before assignment when the program needs it: ```json { "residueId": "res_01J00000000000000000000000", "estimateMinutes": 240, "requiredSkills": ["T-SQL", "Databricks SQL"] } ``` ```bash fa residue estimate --file residue-estimate.json --idempotency-key residue-1-estimate-v1 ``` Re-estimation while `estimated` creates another governed event. Dashboards report the latest projection and retain the audit history for variance analysis. ## 3. Resolve with artifact and validation evidence [#3-resolve-with-artifact-and-validation-evidence] First register the repaired artifact with `fa artifact register`. Then create `residue-resolve.json`: ```json { "residueId": "res_01J00000000000000000000000", "resolvedArtifactId": "art_01J00000000000000000000000", "resolutionEvidenceRef": { "system": "experiments", "type": "validation_run", "id": "repair-validation-42", "digest": "VALIDATION_EVIDENCE_SHA256" }, "actualMinutes": 210, "resolutionNotes": "Replaced identifier construction with an explicit mapping and passed compile and business scenarios." } ``` ```bash fa residue resolve --file residue-resolve.json --idempotency-key residue-1-resolution-v1 ``` Airlift verifies that the artifact belongs to the same engagement and object. The validation reference is provenance, not a caller-authored parity verdict; certificate admission still uses the configured Experiments evidence registry and readiness actions. ## 4. Review independently [#4-review-independently] ```json { "residueId": "res_01J00000000000000000000000", "decision": "approved", "reviewNotes": "Artifact lineage and required validation scenarios reviewed." } ``` ```bash fa residue review --file residue-review.json --idempotency-key residue-1-review-v1 ``` Use `rejected` to return the case to the assignee. The API denies self-review even when a principal otherwise holds the review permission. ### Exercise review in a development App [#exercise-review-in-a-development-app] An explicitly configured development App may allow the resolver to approve the same case so one developer can exercise the complete workflow: ```bash AIRLIFT_DEPLOYMENT_ENV=dev DATABRICKS_APP_NAME=-dev AIRLIFT_ALLOW_DEVELOPMENT_SELF_REVIEW=1 ``` All three conditions are required. If the flag appears under a non-dev deployment name or an App name that does not end in `-dev`, Airlift fails startup. The action still requires a natural-person workspace identity and `airlift:residue:review`; it does not bypass authorization. Airlift records `reviewMode: development_self_review` and labels the override in the App. Only an admitted system principal may use it to mint an explicitly `development` migration certificate, and production cutover always rejects that assurance mode. Staging and production continue to require a distinct reviewer. ## Cancel when scope changes [#cancel-when-scope-changes] ```bash fa residue cancel --file residue-cancel.json --idempotency-key residue-1-cancel ``` The migration remains blocked after cancellation unless another governed decision covers the object. Cancellation requires a reason and explicit confirmation in the App and is available only before resolution. Excluding the migration object is a separate governed decision; cancelling a residue does not silently exclude or certify the object. ## App workflow and leadership metrics [#app-workflow-and-leadership-metrics] Choose **Open remediation case** from an engagement status page to open **Remediation** already filtered to that engagement and scrolled to the exact case. Each case shows the complete resolution path even when later controls are unavailable: 1. assign a remediation owner; 2. register the repaired artifact; 3. attach Experiments validation evidence; 4. submit for independent review; and 5. record the independent review. The current step is identified explicitly. Later steps remain visible as locked instead of disappearing. For example, an assigned case without an artifact displays **A repaired artifact is required** and links to the artifact registration form with the engagement and object preselected. The focused case explains the affected object, unresolved behavior, risk of bypassing the issue, next owner, and evidence required to continue. The App renders one primary action for the current state. Optional delivery forecasting is collapsed. Cancellation is hidden in a red danger zone and remains disabled until the operator supplies a reason and confirms that cancellation does not resolve the migration blocker. Open the organization-wide **Remediation** entry when you want to see work across all engagements. It shows human and agent lanes, active case count, estimated hours, commercial treatment, skills, external work references, and evidence status. Delivery leads assign and may forecast; engineers register repair evidence; a separately authorized approver records review. Useful operating measures include active residue by category, estimated versus actual minutes, aging by state, first-pass conversion rate, repair validation pass rate, and review rejection rate. Never report an unmeasured time-savings percentage: use recorded benchmark observations and delivery actuals. # Migration status # Read migration status [#read-migration-status] Migration status is a read-only projection of records already admitted through the governed action pipeline. It is not a manually edited percentage and it cannot advance an object or wave. Open **Migration inbox** for the portfolio-level work queue, select an engagement, and choose **Migration status** from the engagement tabs. Airlift shows the source-to-target route and these phases: 1. Discover 2. Plan 3. Convert 4. Move data 5. Deploy 6. Validate 7. Certify 8. Cut over ## Completion semantics [#completion-semantics] The percentage counts only phases with a `complete` verdict. Partial object counts are visible within a phase but do not inflate the overall percentage. `not_applicable` phases are excluded from the denominator. Each phase returns: * `verdict`: `not_started`, `in_progress`, `blocked`, `failed`, `complete`, or `not_applicable`; * completed and total work counts; * immutable evidence references already admitted to the ledger; * active blockers; * one concrete next action. Every phase card also links to its owning workspace. A blocked conversion with open residue links to the exact remediation case rather than the organization-wide queue. The blocker panel at the top provides the same **Resolve blocker** action, so the operator does not have to infer which screen owns the next step. Validation failure blocks certification. Missing certificates block cutover. A Runway deployment observation remains a Runway reference; Airlift does not duplicate release state. The same ownership rule applies to Experiments validation evidence. ## CLI and API [#cli-and-api] ```bash fa engagement status fa engagement status --json ``` The JSON form is suitable for CI, customer status exports, and operational dashboards. The API resource is `engagement_status` and requires `engagementId`; organization and actor are derived from authenticated ingress. Example automation check: ```bash fa engagement status "$ENGAGEMENT_ID" --json > migration-status.json jq -e '.blockers | length == 0' migration-status.json ``` A non-empty blocker list is actionable project state, not an error in the status query. Use the phase `nextAction` to decide which owning workflow should run next. The engagement overview surfaces this value as **Next recommended action**. The UI action is a navigation aid only: it does not mutate progress, approve evidence, or bypass the governed action pipeline. The **Artifacts** tab exposes the immutable inputs and outputs behind those verdicts. Open an artifact to copy its SHA-256 digest or provider reference and to see the validation and release requirements attached to that artifact. The **Run ledger** tab answers which assessment, conversion, movement, deployment, and independent validation executions produced the visible state. Press ⌘K/Ctrl K anywhere in the App to find these views without leaving the active engagement context. ## Source-aware navigation [#source-aware-navigation] The App lists only source systems attached to active engagements. Adding SQL Server does not expose Synapse, Snowflake, or other unrelated workspaces. Add another source through the engagement’s governed source action; after it is attached, its source workspace and the combined status route appear automatically. # Build a resumable data transfer # Build a resumable data transfer [#build-a-resumable-data-transfer] Data transfer and reconciliation are one governed lifecycle in Airlift: moving bytes is not complete until independently inspectable evidence satisfies the planned policy. Airlift turns data movement into an executable, observable migration stage. It does not copy bytes itself. Your transfer runner—Lakeflow, a source-native unload/load job, or another certified implementation—moves data. Airlift binds that runner to accepted migration scope, starts a durable Temporal workflow, records every restart checkpoint through Fabric Platform, and refuses to complete when counts, lag, rejects, or evidence violate policy. This separation gives a developer two useful guarantees: 1. retrying a worker does not create a second logical transfer; and 2. a successful copy job is not treated as proof of reconciled data. ## Lifecycle [#lifecycle] `completed` is system-derived. A runner submits immutable reconciliation evidence; Airlift checks the latest row counts, rejects, lag, restart reference, and configured threshold before projecting the terminal state. ## Prerequisites [#prerequisites] Before planning a transfer, create these governed records: * an active or frozen engagement with an accepted assessment; * in-scope data objects; * a verified source binding with `snapshot_read` or `change_feed_read`; * a verified target binding with `target_write`; and * an immutable `transfer_specification` artifact. Connection records contain opaque secret or Databricks connection references. Do not put credentials in a JSON file or action payload. ## 1. Register a transfer specification [#1-register-a-transfer-specification] ```json title="transfer-spec-artifact.json" { "engagementId": "eng_01J...", "estateId": "est_01J...", "kind": "transfer_specification", "name": "Orders snapshot and incremental catch-up", "artifactRef": { "system": "artifact_store", "type": "transfer_specification", "id": "orders-transfer-v3", "digest": "<64-character-sha256>" }, "digest": "<64-character-sha256>", "mediaType": "application/json", "toolVersion": "migration-assets@3.0.0" } ``` ```bash fa artifact register \ --file transfer-spec-artifact.json \ --idempotency-key orders-transfer-spec-v3 ``` The artifact body stays in your artifact store. Airlift records its reference, digest, media type, producer generation, and scope. ## 2. Plan the transfer [#2-plan-the-transfer] ```json title="transfer-plan.json" { "engagementId": "eng_01J...", "estateId": "est_01J...", "waveId": "wav_01J...", "mode": "incremental", "sourceBindingId": "bnd_01J...", "targetBindingId": "bnd_01J...", "specificationArtifactId": "art_01J...", "profileId": "airlift.snapshot_incremental_lakeflow.v1", "toolVersion": "client-transfer-runner@1.4.2", "objectIds": ["obj_01J...", "obj_01K..."], "maximumLagSeconds": 300 } ``` ```bash fa transfer plan \ --file transfer-plan.json \ --idempotency-key orders-transfer-plan-v1 ``` The first executable profile is `airlift.snapshot_incremental_lakeflow.v1`. The schema also represents snapshot, CDC, and streaming transfers, but each source/profile pair needs an admitted runner and live certification before a client engagement uses it. ## 3. Authorize and run [#3-authorize-and-run] ```bash fa transfer run xfr_01J... \ --idempotency-key orders-transfer-run-v1 ``` This command authorizes canonical `running` state through `airlift.transfer_start`. In a deployed Airlift composition, the worker starts or attaches to `transferWorkflowV1` with workflow ID `airlift-v1-xfr-`. Duplicate starts attach to the same Temporal execution. The runner receives opaque connection references and stable per-object/per-step idempotency keys. It returns snapshot, catch-up, restart, lag, and reconciliation references; it never writes Airlift projections directly. ## 4. Inspect and control the run [#4-inspect-and-control-the-run] ```bash fa transfer status xfr_01J... --json fa transfer pause xfr_01J... \ --reason "Pause before source maintenance" \ --idempotency-key orders-pause-maintenance fa transfer resume xfr_01J... \ --idempotency-key orders-resume-maintenance ``` Pause and resume are natural-person controls. A payload-free Temporal signal may wake the workflow after the governed action changes state, but it cannot carry identity or authority. A checkpoint contains: ```ts type TransferCheckpoint = { sequence: number; phase: 'snapshot' | 'catch_up' | 'streaming' | 'freeze'; checkpointRef: ForeignReference; checkpointDigest: string; sourceWatermark: string; targetWatermark?: string; rowsRead: number; rowsWritten: number; bytesRead: number; bytesWritten: number; rejects: number; lagSeconds?: number; restartRef: ForeignReference; observedAt: string; }; ``` Only an admitted runner can record checkpoints. Sequences must increase exactly, and the workflow resumes at the first object without a durable checkpoint. ## 5. Reconcile [#5-reconcile] ```bash fa transfer reconcile xfr_01J... \ --idempotency-key orders-reconcile-start # Normally submitted by the admitted runner, not a person: fa transfer reconcile-record \ --file reconciliation-result.json \ --idempotency-key orders-reconcile-result-v1 ``` For a passing result, Airlift requires at least one checkpoint, equal source/target row counts, zero rejects, and lag at or below `maximumLagSeconds`. The runner's `passed` claim cannot override these checks. ## Failure and uncertainty [#failure-and-uncertainty] | Condition | Airlift behavior | | ------------------------------------------ | --------------------------------------------------------------------------- | | activity retry before an external effect | reuses the same step idempotency key | | worker restart | Temporal replays history and resumes from the projected checkpoint sequence | | operator pause | finishes an in-flight checkpoint, then waits and re-reads canonical state | | row mismatch, rejects, or excessive lag | reconciliation projects `failed` | | cancellation or ambiguous external outcome | projects `uncertain`; never infers success | | unsupported profile | fails before runner execution | Use the **Data transfer** screen in the Databricks App to inspect active runs, checkpoint counts, lag thresholds, row counts, restart references, runner generations, and reconciliation state. # Build and run validation suites # Build and run validation suites [#build-and-run-validation-suites] Airlift's validation laboratory is executable orchestration, not a checklist. For each converted object it combines the assigned object profile, artifact digest, requested readiness tracks, and migration scope into a deterministic suite specification. The worker sends that specification to Fabric Experiments, admits the returned immutable evidence, updates readiness, and creates discrepancies for failed required checks. Use the laboratory after conversion, transfer, and target deployment have produced the artifacts and snapshots you want to judge. ## 1. Find eligible objects [#1-find-eligible-objects] ```bash fa inventory list --estate-id "$ESTATE_ID" --json \ | jq '.rows[] | select(.state == "converted" or .state == "certified")' fa profiles --json ``` Every selected object needs a versioned validation-profile assignment. The profile says which of the nine readiness tracks are required; it does not assert that any track has passed. ## 2. Request a validation execution [#2-request-a-validation-execution] Create `validation-request.json`: ```json { "engagementId": "eng_01J00000000000000000000000", "estateId": "est_01J00000000000000000000000", "waveId": "wav_01J00000000000000000000000", "objectIds": [ "obj_01J00000000000000000000000", "obj_01J00000000000000000000001" ], "suiteVersion": "client-regression-suite@3", "requestedTracks": [ "code", "data_movement", "functional_parity", "non_functional" ], "reason": "Validate the Wave 2 candidate before certificate evaluation." } ``` ```bash fa validation run \ --file validation-request.json \ --idempotency-key wave-2-validation-v3 fa validation list --engagement-id "$ENGAGEMENT_ID" fa validation status "$VALIDATION_EXECUTION_ID" --json ``` The command creates Airlift-owned scope and request state. In Temporal mode the Airlift worker starts `validationWorkflowV1`, which delegates each generated suite to the configured Experiments runner. Repeating the same logical start attaches to the same workflow ID. ## 3. Implement the Experiments runner [#3-implement-the-experiments-runner] The worker expects three callbacks through `ExperimentsValidationAdapter`: ```bash npm install @fabricorg/airlift@0.16.0 @fabricorg/airlift-worker@0.14.0 ``` ```ts import { ExperimentsValidationAdapter } from '@fabricorg/airlift-worker/validation'; const adapter = new ExperimentsValidationAdapter({ begin: async (request) => { // Create or attach to one Experiments request using request.idempotencyKey. return { providerRequestRef }; }, run: async ({ suite, idempotencyKey }) => { // Execute suite.requiredChecks against the source watermark and target snapshot. // Persist the complete LiveEvidence document before returning its immutable ref. return { providerRunRef, evidenceRef, evidence, sourceWatermark, targetSnapshot }; }, complete: async ({ results, idempotencyKey }) => { // Persist a manifest containing every object result and return its digest. return { providerRunRef, evidenceManifestRef, evidenceManifestDigest }; }, }); ``` Also provide `validationEvidencePublisher`. It must write the evidence body and registry entry before Airlift invokes `validation_run_record`. With no publisher, validation fails closed. The built-in stub is for hermetic tests only and identifies its output as synthetic. Common generated checks cover compilation, dynamic SQL and cast behavior, row counts, checksums, null/decimal/time-zone/collation semantics, deletes and restart behavior, business queries, errors, performance, cost, security, concurrency, deployment drift, acceptance, and rollback readiness. Your runner maps those stable check IDs to concrete source and Databricks assertions. ## 4. Inspect results and readiness [#4-inspect-results-and-readiness] ```bash fa validation runs --object-id "$OBJECT_ID" --json fa validation readiness --object-id "$OBJECT_ID" --json fa discrepancy list --object-id "$OBJECT_ID" ``` An Experiments run can pass or fail. Airlift stores the provider run reference, evidence reference and SHA-256 digest, artifact digest, source watermark, target snapshot, tool version, verdict, and admitted producer. A passing run is still only evidence for the tracks explicitly linked to it. ## 5. Triage and remediate a failure [#5-triage-and-remediate-a-failure] ```bash fa discrepancy show "$DISCREPANCY_ID" --json fa discrepancy triage \ --file discrepancy-triage.json \ --idempotency-key "$DISCREPANCY_ID-triage-v1" fa discrepancy resolve \ --file discrepancy-resolution.json \ --idempotency-key "$DISCREPANCY_ID-resolution-v1" ``` The bounded Harness triage agent may propose category, severity, disposition, and a rationale. Its only admitted mutation is `airlift.discrepancy_triage`. It cannot accept a difference, submit evidence, waive a requirement, mint a certificate, or change release state. Low and medium differences may be accepted by an authorized natural person who did not triage the same discrepancy. High and critical differences are non-acceptable and must be remediated. Resolution does not close the discrepancy: `discrepancy_verify` requires a separately admitted, passing Experiments run for the same object and matching provider evidence identity. ## 6. Evaluate certificate readiness [#6-evaluate-certificate-readiness] ```bash fa certificate list --object-id "$OBJECT_ID" fa certificate mint \ --file certificate-mint.json \ --idempotency-key "$OBJECT_ID-certificate-v1" ``` Certificate minting is normally performed by the admitted worker after it recalculates the profile and readiness digests. The action fails if required evidence is pending, failed, stale, or not covered by an approved and unexpired waiver. Business acceptance remains a separate human decision. The resulting envelope is signed and names exactly the profile, artifacts, snapshots, provider evidence, policy revision, and tool versions it covers. Use the **Validation laboratory** screen for executions and discrepancy actions. Use the **Assurance center** for the readiness matrix, certificates, waivers, and evidence-pack exports. # Assurance center # Use the assurance center [#use-the-assurance-center] Open **Govern → Assurance center** in the Airlift Databricks App. The page is the operator projection over governed state; it does not allow callers to edit readiness cells. The readiness matrix shows every in-scope object across inventory, target design, code, data movement, deployment, functional parity, non-functional behavior, business acceptance, and cutover readiness. Each cell comes from the assigned validation profile and the latest admitted observation. `n/a` is profile applicability; `pending`, `passed`, `failed`, `stale`, and `waived` are readiness verdicts. Certificate cards show the signed envelope identity, object, profile generation, readiness digest, key ID, issue time, and active/stale/revoked state. Waiver cards remain visible with their requirement, status, compensating control, decision provenance, and expiry. A **Development evidence** badge means the factory journey ran with an admitted development relaxation. The certificate is signed and auditable, but it is not production authority. Inspect the certificate, validation run, and evidence digest exactly as you would for independent evidence; then replace it with an independently reviewed run before production cutover. See [Development assurance](/docs/operations/development-assurance). The page does not mint certificates or verify discrepancies on an operator's behalf. Those decisions require admitted validation evidence and the governed action permissions described in [Build and run validation suites](/docs/migration/validation). Export a wave evidence pack from this page or with: ```bash fa evidence export \ --file evidence-export.json \ --idempotency-key wave-2-evidence-v1 ``` The export is a content-digested audit artifact. It includes governed references and policy outcomes, not credentials, SQL bodies, client records, or mutable screenshots. # Conversion factory controls # Conversion factory controls [#conversion-factory-controls] The conversion factory is designed for two audiences. Developers need reproducible attempts and artifact lineage. Engineering leaders need an honest answer to five questions: what scope is running, what finished, what still needs people, what evidence supports quality, and whether a strategy should be promoted. ## Control boundaries [#control-boundaries] | State | Owner | Leadership implication | | ----------------------------- | ------------------------------------ | ----------------------------------------------------------------- | | Accepted migration scope | Airlift | Batch membership cannot drift from the accepted assessment | | SQL translation | Lakebridge | Converter generation is pinned on every batch and attempt | | Repair candidate | Harness agent or engineer | A candidate has no approval or certificate authority | | Validation run | Experiments | Quality evidence is independent from the actor that produced code | | Artifact body | Repository, volume, or release store | Airlift keeps a reference and digest, not a second code store | | Work coordination | Tower or another work manager | Work completion does not close migration truth | | Residue closure and readiness | Airlift | Evidence and independent review are required | These boundaries let teams replace a specialist tool without rewriting the migration ledger. They also prevent the common failure mode where a ticket status or converter exit code is presented as production readiness. ## Batch admission [#batch-admission] A batch is allowed only when the assessment has been accepted for the engagement and all objects belong to that accepted snapshot. Each batch pins method and tool generation. Every attempt must match those values. Use smaller batches for high-complexity stored code and larger batches for homogeneous low-complexity objects; the contract supports up to 1,000 objects but does not prescribe an operational size. ## Completion rules [#completion-rules] Counts are computed from the ledger. A batch cannot complete while an object is missing an attempt or has a running attempt. Converted objects need matching immutable artifacts. Failed objects need active residue. This makes throughput reports reproducible after a worker restart or full event replay. ## Human capacity [#human-capacity] Residue estimates are minutes plus required skills, not story points hidden in a delivery tool. Use the remediation projection to plan specialist capacity and join its `workRef` to your work manager for scheduling. Keep `commercialAttribution` visible so included, client-owned, change-request, and managed-service work do not collapse into one cost pool. ## Repair strategy promotion [#repair-strategy-promotion] Run versioned repair datasets through Experiments. Feed the resulting digest-bound summary into `evaluateRepairPromotion` with explicit thresholds for: * minimum sample size; * schema-valid output rate; * compile-pass rate; * parity-pass rate; * maximum unsafe-output rate; * minimum acceptable regression delta. Any failed threshold blocks promotion. Dataset and evaluation-run references stay in the decision record so a future model, prompt, or policy change can be compared against the same baseline. ## Reporting without overclaiming [#reporting-without-overclaiming] Never report an unmeasured time-savings percentage. Report observable measures: accepted objects, terminal attempts, artifact-backed conversions, open residue, estimated and actual remediation minutes, independent validation passes, and reviewed closures. A time-reduction claim requires paired or historical benchmark evidence; the number is not inferred from automation coverage. ## Operational screens [#operational-screens] The **Conversion** App screen shows batch scope/outcomes, attempt generations, artifact digests, and validation references. The **Remediation** screen shows human/agent lanes, effort, skills, commercial treatment, assignment, artifact/evidence submission, and independent review. Both use the same tenant-scoped governed mutations as the CLI and remote API. # Cutover control room # Cutover control room [#cutover-control-room] Open **Cutover control** in the Airlift Databricks App after a wave has been frozen. The screen is a projection of governed state, not a second workflow or monitoring system. Each wave card shows: * the frozen scope digest and object count; * runbook version, timed steps, owners, and rollback links; * rehearsal and parallel-run verdicts; * the exact certified effector generation and expiry; * readiness blockers and open incidents; * hypercare and source-disposition state. If the frozen wave contains a development certificate, the card shows **Development evidence** and readiness remains blocked even after a successful rehearsal. This is the expected development endpoint. Independent certificates, an admitted deployment digest, a certified effector, and the required operational window must replace the development proof before production. See [Development assurance](/docs/operations/development-assurance). The **Start cutover** action appears only when the evidence projection is ready and the wave is eligible. Submitting it starts or attaches to the same durable Temporal workflow used by `fa cutover start`. The workflow checks policy again, so stale browser state cannot bypass a gate. Use **Open incident** whenever operational risk appears. The action immediately blocks production readiness. Resolve the incident through a governed command with evidence; closing a work item in an external system does not advance Airlift state. After a completed wave and independent cutover-verification observation, **Start hypercare** pins the minimum observation deadline and acceptance criteria digest. Radar or an external monitor continues to own raw operational truth. Airlift records only the admitted reference and uses its verdict in migration policy. For bulk configuration, CI, and evidence ingestion, use the [cutover CLI](/docs/cli/cutover). # Industrialize migration delivery # Industrialize migration delivery [#industrialize-migration-delivery] An accelerator becomes a delivery system when another trained team can reproduce its outputs, controls, and evidence without relying on the original builders. Airlift industrializes the method in six layers. ## 1. Qualify before estimating [#1-qualify-before-estimating] Record exact source variant, authority, network shape, representative-data approval, scope surfaces, exclusions, recovery objectives, and decision owners. Run authenticated preflights for source metadata, target access, artifact storage, validation, release reconciliation, and evidence signing. Re-baseline delivery after the accepted inventory; do not sell an object count guessed from discovery calls. ## 2. Bound every provider [#2-bound-every-provider] For each source adapter and execution provider, define concurrency, rate, quota, cost, timeout, retryable errors, checkpoints, reconciliation, and circuit breaking. A non-idempotent external effect gets one attempt. On timeout, observe external state and reconcile; do not infer failure and retry. ## 3. Generate one consistent evidence chain [#3-generate-one-consistent-evidence-chain] Qualification, assessment, blueprint, waves, human residue, certificates, runbooks, and value reports derive from product contracts. Each output records immutable refs and digests. The delivery team can add narrative, but it cannot rewrite source scope, provider verdicts, readiness, or certificate claims. ## 4. Rehearse operations [#4-rehearse-operations] Before the project depends on the installation, rehearse: * backup and restore; * evidence-signer rotation; * connector and credential-reference rotation; * evidence retention and export; * disaster recovery; * application and package upgrade/rollback; and * workspace/private-link/offline dependency paths. Radar observations and Tower work remain foreign references. Airlift may block on them; it does not copy their specialty truth. ## 5. Prove the scale you claim [#5-prove-the-scale-you-claim] Repository scale probes cover contract compilation at 10,000 and 50,000 objects. That is not a claim about source API throughput, Lakebase event hydration, validation runtime, or production cutover. Certify those boundaries separately with the project network, provider quotas, dataset, event fan-out, and workspace topology. ## 6. Separate engagement value from external claims [#6-separate-engagement-value-from-external-claims] Engagement reports can describe an admitted project scope. Aggregate external language needs a versioned cohort, comparable manual and assisted scopes, minimum sample and engagement counts, confidence, methods, release generation, exclusions, limitations, methodology review, and immutable evidence. Use [delivery commands](/docs/cli/delivery) in CI and [measure engagement value](/docs/operations/value-measurement) for governed observations and reports. # Development assurance # Exercise the factory in development [#exercise-the-factory-in-development] Development assurance lets an engineering team prove that Airlift can carry a migration through its governed actions before independent client reviewers, production effectors, and every sibling service are available. It is useful for product development and demos. It is not a weaker production certificate. An object or wave marked **Development evidence** has passed the configured development journey, but Airlift still refuses production cutover. The assurance marker is stored in the signed certificate envelope and the frozen cutover record; hiding a banner or changing an App route cannot remove it. ## What the journey exercises [#what-the-journey-exercises] | Stage | Development proof recorded by Airlift | What remains external | | --------------- | -------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------- | | convert | immutable target artifact digest and governed conversion outcome | correctness beyond the admitted fixture | | validate | provider run identity, evidence reference, evidence digest, snapshots, and verdict | independent client validation for production | | certify | object-specific readiness profile and signed `development` certificate | production separation of duties | | move data | source/target bindings, checkpoint, restart identity, row counts, rejects, lag, and reconciliation | client-scale throughput and change-feed behavior | | prepare cutover | frozen scope digest, versioned runbook, and rehearsal evidence | certified effector, parallel run, approvals, and a real change window | The expected endpoint is **Rehearsed · Development evidence · Not production ready**. If the UI says `Ready` after a development certificate, treat that as a defect. The engagement's **Development assurance** badge and **Independent validation** count use only passing governed validation runs recorded by an admitted validation principal. A subject-bound run must still resolve to the exact artifact registered to that engagement, with the same digest. Legacy runs recorded before subject binding count only when their artifact digest matches an artifact registered to the engagement. An artifact labeled as Experiments evidence does not raise the badge, and completing a validation execution records factory activity but does not by itself prove an independent verdict. ## Run the repository certification journey [#run-the-repository-certification-journey] Authenticate the Databricks CLI as the developer who should appear in the governed action ledger, then run the repository command: ```bash databricks current-user me pnpm certify:synapse:development ``` When you use a named Databricks CLI profile: ```bash AIRLIFT_DATABRICKS_PROFILE= pnpm certify:synapse:development ``` The command derives the authenticated operator, obtains a short-lived Lakebase credential, and reads the configured authorization, evidence-registry, and signing values from the Databricks secret scope. Secret values stay inside the command process and are never printed. A configured operator override that differs from the authenticated user is rejected rather than silently creating misleading audit history. ## Inspect the result with `fa` [#inspect-the-result-with-fa] Configure the normal authenticated remote CLI connection, then use governed identifiers from the engagement page: ```bash fa engagement status fa engagement status --json > migration-status.json fa validation list --engagement-id fa validation status fa validation runs --object-id fa validation readiness --object-id fa certificate list --object-id fa transfer list --engagement-id fa transfer status fa cutover status ``` These commands read the same governed projections as the App. They do not infer success from files on a developer laptop. Save JSON output as a CI artifact when you need a machine-readable audit snapshot. ## Find the records in the App [#find-the-records-in-the-app] 1. Open **Engagements** and select the active engagement. 2. Use **Migration status** to inspect all eight delivery phases and their blockers. 3. Open **Artifacts** for immutable plan, target-code, test, evidence, and transfer-specification references. 4. Open **Run ledger** for assessment, conversion, transfer, validation, and deployment execution records. 5. Open **Assurance center** to inspect readiness cells and signed certificate envelopes. 6. Open **Cutover control** to inspect frozen scope, runbook, rehearsal, and remaining production blockers. The UI intentionally shows references and SHA-256 digests rather than credential values, source records, or mutable file bodies. Follow a provider reference to the admitted artifact store when you need the underlying report. ## Development configuration boundary [#development-configuration-boundary] Repository operators can enable the development path only when every boundary is present: ```dotenv AIRLIFT_DEPLOYMENT_ENV=dev DATABRICKS_APP_NAME= AIRLIFT_ALLOW_DEVELOPMENT_SELF_REVIEW=1 AIRLIFT_DEVELOPMENT_OPERATOR_PRINCIPAL= AIRLIFT_DEVELOPMENT_WORKER_PRINCIPAL= ``` The admitted worker still records validation evidence and mints certificates. A browser form or natural-person CLI caller cannot mint development certificates directly. The Synapse certification runner operates only against the schema installed by deployment; it never performs schema creation, table alteration, or index maintenance with a developer identity. Authorization, policy, evidence verification, signing, and idempotency remain enforced. Never configure these controls in staging or production. Remove development certificates and rerun independent validation before preparing a production wave. ## Why production remains blocked [#why-production-remains-blocked] Development mode may relax reviewer separation only. It does not waive tenant isolation, evidence identity, signing, immutable digests, transfer reconciliation, idempotency, uncertain non-idempotent effects, or the production cutover gate. A development wave must still report the missing independent evidence, certified effector, operational window, and any unbound Runway release as explicit blockers. See [Assurance center](/docs/operations/assurance-center), [Cutover control room](/docs/operations/cutover-control-room), and [Roles and separation](/docs/operations/roles-and-separation) for the production path. # Evidence ledger and exports # Evidence ledger and exports [#evidence-ledger-and-exports] The Platform Host invocation/event ledger is the audit spine. Airlift projections are replayable views; they do not replace the source events. Large reports and provider outputs live in an artifact/evidence store and enter the ledger by immutable reference and digest. Inspect and admit those references with `fa evidence inspect` and `fa evidence admit`. Admission registers artifacts only: it cannot manufacture an Experiments verdict, a Runway release result, a Radar observation, or a cutover decision. See [live evidence commands](/docs/cli/evidence) for the schema, App locations, and automation outcomes. ## Evidence pack [#evidence-pack] `airlift.evidence_export` assembles the object or wave trail: governed events addressed to the subject plus related object/wave events, policy revision, and current projection. The export records its own key and SHA-256 report digest. That digest proves content identity, not signer identity. ## Migration certificate [#migration-certificate] The migration certificate is an Ed25519-signed envelope derived from admitted readiness evidence. Verify it independently with: ```bash fa certificate verify migration-certificate.json --keys public-keys.json ``` ## Evidence retention [#evidence-retention] Retain immutable references for the application build, deployment, source snapshot, provider runs, certificates, approvals, and observation windows used for each migration decision. Screenshots, mutable branch links, and dashboard state without a bounded observation window are supporting context only. # Operate Airlift # Operate Airlift [#operate-airlift] Operating Airlift means keeping its claims reproducible over a long migration, not only keeping a web process alive. ## Daily operator view [#daily-operator-view] Monitor assessments and conversions, residue and human-lane queues, readiness blockers, expiring waivers, stale certificates, wave scope drift, transfer lag, Temporal workflow state, adapter failures, and evidence-store availability. The funnel is useful for executive progress; the readiness matrix explains production state. ## Production invariants [#production-invariants] * identity and organization come from authenticated server context; * every meaningful mutation is one of the 66 Platform actions; * retries reuse the same logical idempotency key; * agents cannot gain decision authority; * external effects are checkpointed and reconciled; * evidence references are immutable and content-digested; * signed claims can be verified outside the service; * unresolved readiness remains visible rather than being converted into a success claim. # Migration-pack certification # Migration-pack certification [#migration-pack-certification] Run one recurring suite across the SQL Server, Snowflake, Redshift, Oracle, Teradata, and Hadoop packs: ```bash pnpm certify:migration-packs ``` The runner uses credential-free fixtures and local adapters. It writes a report and an evaluator input for each source under `reports/`. These artifacts prove repeatable product behavior for the bound build and fixture; they are not client acceptance or production evidence. ## Required stages [#required-stages] | Stage | Required behavior | | ------------------------- | ----------------------------------------------------------------------------------------------- | | manifest | validate source, variant, inventory, dependencies, and hard-case labels | | native inventory | retain SQL, procedural, orchestration, utility, policy, and consumer objects | | dependency compilation | produce a stable order and fail closed on missing nodes or cycles | | routing | classify every object as deterministic, bounded-agent repair, or human remediation | | CLI and governed API | preserve schemas, exit behavior, tenant scope, identity, artifact digest, and idempotency | | App workbench | expose all six packs, required hard cases, commands, and proof boundaries | | transfer | prove snapshot, catch-up, checkpoint, restart, late data, deletion, and reconciliation behavior | | validation and deployment | require Experiments evidence and Runway-owned immutable deployment inputs | | cutover | retain approval, denial, expiry, replay, uncertain-result, compensation, and rollback controls | | scale | compile a 10,000-object estate for every source without scope or target-map loss | ## Source-specific corpus [#source-specific-corpus] | Source | Examples that must remain covered | | ------------- | -------------------------------------------------------------------------------------------------------------- | | Microsoft SQL | collation, identity, temporary tables, dynamic SQL, SQL Agent, SSIS, SSRS, linked servers | | Snowflake | VARIANT, timezone, scripting, tasks and streams, stages and pipes, Snowpark, sharing | | Redshift | distribution and sort behavior, SUPER, Spectrum, COPY/UNLOAD, WLM, S3 restart manifests | | Oracle | NUMBER and DATE/null semantics, PL/SQL packages, exceptions, database links, scheduler, SCN catch-up | | Teradata | SET/MULTISET, primary index, QUALIFY, BTEQ, load utilities, volatile tables, utility restart | | Hadoop | Hive and Impala, metastore, partitions and files, Oozie, Pig, Sqoop, Spark/MapReduce, UDF/SerDe, Ranger/Sentry | Removing a required case blocks hermetic certification even when every command exits successfully. ## Proof boundary [#proof-boundary] `hermetic_proven` is the account-independent ceiling. It covers the compiler, routing, transfer simulation, integration contracts, governed paths, UI, scale, and failure behavior for the bound fixtures. `workspace_proven` additionally requires immutable evidence from real workspace inventory, target materialization, Experiments validation, Runway promotion, and a cutover rehearsal. An App deployment alone does not satisfy workspace proof. The evaluator structurally rejects `client_proven` and `production_certified`. Those levels require representative source behavior, client acceptance, current rollback evidence, production authority, and a certified effector. Every admitted stage records a reference, digest, build reference, dataset reference, tool version, completion time, and assertions. Store reports in an immutable CI artifact store; do not derive client-readiness or time-savings claims from fixture evidence. # Observability and operational evidence # Observability and operational evidence [#observability-and-operational-evidence] Operational signals should correlate organization, estate, object, wave, invocation, workflow, adapter run, and evidence reference. Redact secret-shaped fields before durable logging. ## What to alert on [#what-to-alert-on] * repeated assessment/conversion adapter failure; * projection cursor lag or reconciliation divergence; * evidence registry or signing-key unavailability; * validation runs rejected for provenance or digest mismatch; * readiness blockers that exceed the wave plan; * waivers approaching expiry; * stale certificates in a planned wave; * transfer lag outside the accepted RPO; * workflows waiting beyond their approval window; * uncertain cutover effects awaiting reconciliation. Radar can own cross-estate operational observations. Airlift may require a Radar-green observation and record its immutable reference/digest; it does not copy the Radar monitor model or infer health from task completion. # Failure and recovery # Failure and recovery [#failure-and-recovery] | Failure | Recovery rule | | -------------------------------- | --------------------------------------------------------------------------------- | | Duplicate action delivery | Same key and parameters collapse; conflicting reuse is rejected | | Projection process restart | Rehydrate from the tenant event cursor/checkpoint and reconcile concurrent writes | | Lakebridge job failure | Record terminal adapter evidence; retry only under its classified policy | | Bounded repair failure | Route to visible human residue; never broaden or loop indefinitely | | Transfer interruption | Resume from a durable watermark/checkpoint and reconcile duplicates/counts | | Temporal worker restart | Replay deterministic workflow history; activities retain stable action keys | | Approval wait expiry | Fail closed under policy; a late signal does not manufacture approval | | Cutover verification uncertainty | Remain executing and reconcile; do not auto-retry the external effect | Test historical workflow replay before deploying incompatible workflow code. Query errors from authentication or Temporal outages propagate; only a typed workflow-not-found condition becomes absence. # Roles and separation of duties # Roles and separation of duties [#roles-and-separation-of-duties] Natural-person membership is organization-, principal-type-, and role-bound. Airlift defines five small roles: | Role | Typical authority | | --------- | ---------------------------------------------------------------------------------------------------------- | | Viewer | read projections and evidence | | Operator | estates, assessments, objects, waves, conversions, profile assignment, waiver requests, cutover submission | | Approver | wave approval, business acceptance, waiver decisions | | Validator | parity, validation-run, and readiness records | | Admin | full tenant administration, including certificate and policy operations | Trusted workers have a narrow transition set. Harness agents have an even smaller lane: assessment/object recording and conversion start/record only. Platform installers act only on the `airlift-platform` organization registry. Validation principals are admitted per organization and optionally per provider. Service accounts may hold the separate `automation` role. Its machine-checked permissions cover bounded operational assessment, inventory, planning, transfer, validation, discrepancy, observation, and evidence work—not approval, waiver, certificate, policy, membership, or cutover execution. See [organization membership](/docs/reference/organization-membership/) for the exact list. Submission authorization is repeated immediately before execution. Approval authorization is independent. An agent, converter, or submitter cannot manufacture or inherit a natural person's approval from a workflow signal, form field, or CLI flag. # Scale and performance # Scale and performance [#scale-and-performance] Large estates can contain tens of thousands of objects and generate high event volume during assessment import, readiness evaluation, and wave execution. Size the deployment with representative source inventory, event, and artifact workloads rather than object counts alone. ## Required benchmark shapes [#required-benchmark-shapes] * 10k and 50k objects across multiple estates and waves; * assessment import and dependency projection time; * steady-state incremental hydration latency; * concurrent writer plus projection restart/recovery; * wave approval/cutover fan-out event volume; * evidence export size and artifact-store latency; * readiness evaluation and stale-certificate scans. Record the application build, store configuration, dataset generator or source snapshot, concurrency, run duration, and latency percentiles with each result. Set an accepted operating envelope for the project, alert before it is exceeded, and repeat the benchmark after storage, projection, policy, or evidence-provider changes. # Account-independent Synapse certification # Account-independent Synapse certification [#account-independent-synapse-certification] Use this pack when engineers need confidence in the complete Airlift composition but do not have a live Synapse tenant. It proves deterministic product behavior against a versioned representative corpus and can incorporate live non-production Databricks evidence. It does not simulate away client networking, permissions, source semantics, or production authority. ## What the pack executes [#what-the-pack-executes] | Stage | Executed assertion | | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------- | | manifest validation | dedicated, serverless, or mixed schema; unique IDs; complete dependencies; explicit exclusions and dead assets | | inventory and dependency compilation | stable counts, digest, topological order, distributions, partitions, paths, owners, and consumers | | ADF IR compilation | supported activity mappings, parameters, retries, dependencies, triggers, and credential-free linked-service requirements | | CLI contract | inspect, plan, register, and certification commands preserve schemas, digests, identity boundaries, and exit behavior | | governed API | the generated plan enters the active engagement through `airlift.artifact_register` and tenant-scoped projections | | authenticated remote API | request identity and organization come from the admitted principal; payloads cannot select either | | App workbench | desktop and mobile journeys expose every execution phase and the proof boundary | | transfer and restart | snapshot, catch-up, watermarks, retries, restart reference, deletes, late arrivals, and reconciliation are repeatable | | validation contract | the bundle requires independent Experiments suites and returns discrepancies to Airlift | | deployment contract | the bundle requires a Runway-owned immutable release and observed deployment digest | | cutover rehearsal | Temporal approval, expiry, denial, duplicate start, restart/replay, uncertain outcome, compensation, and rollback paths | | scale | a 10,000-object Synapse estate compiles without losing objects or target mappings | Run it from a development checkout: ```bash pnpm certify:synapse ``` For the focused Temporal provider journey, run: ```bash pnpm test:e2e:synapse:mock ``` That command executes the parent and child workflows under Temporal's time-skipping test server and verifies success, provider failure, cross-engagement denial, production rejection, and deployment reconciliation. ## Publish the isolated browser lane [#publish-the-isolated-browser-lane] The browser journey is published separately from production. Its configuration is fixed to `sandbox.airlift.fabric.pro`, `AIRLIFT_RUNTIME_TIER=sandbox`, the `airlift-sandbox-v1` task queue, shared PostgreSQL, dark worker activation, and `AIRLIFT_SYNTHETIC_JOURNEY_MODE=hermetic`. Run the repository's deployment migration before starting the worker. It creates the private `fabric_airlift.validation_evidence` table used by the hermetic Experiments adapter. Evidence is immutable and content-addressed, and the same PostgreSQL-backed object publishes and verifies it. These rows remain private sandbox evidence and never establish workspace, client, or production proof. From a reviewed, clean commit already pushed to `origin/main`: ```bash pnpm cloudflare:sandbox:preflight pnpm cloudflare:sandbox:deploy ``` The release preflight rebuilds the console and Temporal worker artifacts, verifies every artifact byte against its manifest, requires both manifests to name the exact pushed commit, binds a SHA-256 release ID, and rejects production routes or changed sandbox variables. Immediately before Wrangler runs, it repeats the clean-tree, pushed-commit, artifact-manifest, template-digest, release-ID, and generated-byte checks. The production command remains `pnpm cloudflare:deploy` and its template cannot activate a synthetic provider. The worker remains dark after publication. Activate it only through the existing sandbox worker activation path after confirming the configured PostgreSQL database, Temporal namespace, Cloudflare Access audience, and governed authorization map all belong to the sandbox. Do not infer or copy production resources. The run-ledger start control remains disabled until one unambiguous governed Synapse candidate exists. Candidate preparation is explicit operator work; the application does not seed an organization, engagement, source binding, transfer, deployment, or object scope on startup. This prevents a publication from fabricating tenancy or migration state. Start identity is derived on the server from that exact scope plus its durable run generation. Concurrent browser submissions therefore attach to one Temporal execution; a later generation follows the terminal PostgreSQL row for the same binding, transfer, deployment, object, and cutover scope. Unrelated rows over the same objects cannot advance it, and generation is bounded to 32 before operator reconciliation is required. Cancellation records a terminal row inside a non-cancellable Temporal activity; after hard termination, the next deterministic Temporal generation hydrates the authoritative tenant projection and first reconciles its selected stale running predecessor rows. If the prior Temporal execution is no longer retained or the sandbox namespace was replaced, only Temporal's typed workflow-not-found result may advance that durable running row; describe timeouts and other transport failures do not mutate the ledger. No browser nonce can select workflow identity. The parent workflow deliberately pauses at `awaiting_validation_request`. On that ledger row, an authorized natural person selects **Request validation and continue**. The rendered row supplies its workflow ID, and the server revalidates that exact workflow's engagement, running phase, and frozen validation contract from the durable journey row before it submits the exact idempotent `airlift.validation_execution_request`, and sends only a payload-free Temporal wake. The workflow then discovers the governed request from projections. A signal cannot supply or override validation scope, suite, tracks, identity, or evidence. Successful execution writes: * `reports/f8c-synapse-evidence.json` — input accepted by the public evaluator; * `reports/f8c-synapse-certification.json` — evaluation plus command-result digests. Reports are generated evidence and should be retained in an immutable CI artifact store, not committed as permanent proof. ## Proof levels [#proof-levels] | Result | What must be present | What it does not prove | | ---------------------- | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- | | `contract_only` | some required stages are missing or invalid | executable end-to-end behavior | | `hermetic_proven` | every account-independent stage passed against the same build and dataset | live workspace integration or client behavior | | `workspace_proven` | hermetic proof plus live App API, target materialization, Experiments, Runway, and Temporal evidence | client source behavior or production authority | | `client_proven` | unavailable from this evaluator | requires a separate client-bound dataset and acceptance process | | `production_certified` | unavailable from this evaluator | requires current production operations, rollback, signing, and effector certification | The request schema rejects the final two levels. This is a structural policy, not a documentation warning. ## Evidence integrity [#evidence-integrity] Every stage carries a reference, SHA-256 digest, build reference, dataset reference, tool version, completion time, and asserted behavior. The evaluator rejects: * a missing required stage; * workspace stages represented by hermetic evidence; * a build or dataset mismatch; * evidence completed after the observation time; * an attempted client or production request. The report has separate evidence and report digests. Publishing the report does not promote a source-capability claim; use the governed capability registry and human review for that decision. ## Engineering-leadership interpretation [#engineering-leadership-interpretation] `hermetic_proven` means the product composition and failure behavior are repeatable without purchasing or borrowing a source account. It is the gate for expanding the same contract to another source pack. `workspace_proven` is the stronger pre-client gate. Do not assign it merely because the App is running. Target assets, validation, release promotion, and durable rehearsal must all return immutable live references. An App deployment alone is insufficient for workspace proof. Neither result supports a time-savings percentage. Measure manual baseline effort, automation runtime, human remediation, and accepted outcomes during an actual engagement before publishing a value claim. # Measure engagement value # Measure engagement value [#measure-engagement-value] Airlift measures migration and modernization outcomes; it does not assume a fixed time reduction. The unit of evidence is a benchmark observation for one engagement, estate, activity, method, and optional object. For every observation record four numbers separately: * `manualBaselineMinutes`: hands-on time for the comparable manual process; * `assistedHandsOnMinutes`: developer or operator time with Airlift; * `automatedRuntimeMinutes`: unattended machine runtime; * `reworkMinutes`: additional hands-on correction effort. This prevents a long automated job from being presented as developer effort and prevents rework from disappearing inside a headline percentage. ## Record observations [#record-observations] ```json title="benchmark.json" { "engagementId": "eng_...", "estateId": "est_...", "objectId": "obj_...", "activity": "modernization", "method": "time_study", "manualBaselineMinutes": 120, "assistedHandsOnMinutes": 45, "automatedRuntimeMinutes": 12, "reworkMinutes": 8, "confidence": "high", "dimensions": { "sourceSystem": "synapse", "objectType": "table", "complexity": "medium" }, "evidenceRef": { "system": "airlift", "type": "time_study", "id": "study-42", "digest": "" }, "evidenceDigest": "", "observedAt": "2025-06-15T12:00:00Z" } ``` ```bash fa value record --file benchmark.json --idempotency-key "benchmark-study-42" fa value observations --engagement-id "$ENGAGEMENT_ID" --json ``` Supported methods are `historical_actual`, `time_study`, `expert_estimate`, and `paired_execution`. A system-recorded paired execution must reference Experiments. ## Mint a summary [#mint-a-summary] Select the exact observation IDs included in the calculation: ```json title="summary.json" { "engagementId": "eng_...", "benchmarkObservationIds": ["bmk_...", "bmk_...", "bmk_..."], "reason": "Summarize the accepted engagement time studies." } ``` ```bash fa value summarize --file summary.json --idempotency-key "value-summary-current-scope" fa value summaries --engagement-id "$ENGAGEMENT_ID" --json ``` Airlift computes hands-on minutes saved, hands-on reduction rate, acceleration multiple, sample count, methods, source systems, and confidence. A new benchmark observation makes every active summary for that engagement stale. Mint a new summary instead of silently changing a previously published result. ## Publish safely [#publish-safely] Client reports require at least three observations and medium or high computed confidence. Lower-confidence summaries can be published only for an account-team audience and retain their limitations. ```bash fa value publish --file report.json --idempotency-key "value-report-client-1" fa value reports --engagement-id "$ENGAGEMENT_ID" --json ``` Every generated claim identifies its engagement scope and evidence digest. Say “the recorded scope reduced hands-on effort by 58%,” never “Airlift always reduces migrations by 60%.” Before using an aggregate claim outside one engagement, run `fa delivery claim-check` against a versioned cohort packet. The external gate requires at least three engagements, ten comparable observations, reviewed methodology, medium/high confidence, exclusions, limitations, and immutable evidence. See [industrialize migration delivery](/docs/operations/delivery-industrialization). # Waivers, staleness, and revocation # Waivers, staleness, and revocation [#waivers-staleness-and-revocation] A waiver targets one profile requirement and records its owner, reason, compensating control, evidence, approver, and expiry. Agents cannot request or approve waivers. ## Non-waivable controls [#non-waivable-controls] Tenant isolation, authenticated identity, required separation of duties, inventory, functional parity, cutover readiness, and an uncertain non-idempotent cutover outcome do not become acceptable through a waiver. Policy may make additional tracks non-waivable by object class or environment. ## Staleness [#staleness] A certificate becomes stale or revoked when policy-defined material changes: artifact or deployment digest, source watermark, target snapshot, profile/policy revision, dependency readiness, evidence validity, tool generation, waiver expiry, deployment drift, or a critical observation breach. Historical certificates stay immutable and inspectable. A stale or revoked certificate cannot satisfy the cutover gate; the object returns to a state where new evidence can be admitted and a new certificate minted. # Governed action catalog # Governed action catalog [#governed-action-catalog] Every action is namespaced `airlift`, versioned, schema-validated, authorized, policy evaluated, idempotent, event-emitting, and tenant-scoped unless the platform organization registry is the documented exception. | Area | Actions | | ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Organization registry | `org_provision`, `org_retire` | | Organization membership | `organization_membership_set` | | Engagements | `engagement_create`, `engagement_update`, `engagement_activate`, `engagement_freeze` | | Connection bindings | `connection_binding_register`, `connection_binding_verify`, `connection_binding_retire`, `connection_diagnostic_record` | | Deployment access | `access_preflight_record` | | Evaluator readiness | `evaluator_candidate_freeze`, `evaluator_rehearsal_record` | | Unity Catalog evidence | `uc_projection_run_record`, `uc_target_reconciliation_record` | | Estates | `estate_register`, `estate_update`, `estate_retire` | | Assessment | `assessment_start`, `assessment_record`, `assessment_accept`, `assessment_export` | | Dependencies | `dependency_graph_start`, `dependency_batch_record`, `dependency_graph_accept` | | Planning | `plan_generate`, `plan_select`, `plan_freeze` | | Objects | `object_register`, `object_exclude`, `object_rework` | | Waves | `wave_plan`, `wave_assign`, `wave_approve` | | Conversion batches | `conversion_batch_create`, `conversion_batch_start`, `conversion_batch_complete` | | Conversion attempts | `conversion_start`, `conversion_record` | | Artifacts | `artifact_register` | | Remediation | `residue_create`, `residue_estimate`, `residue_assign`, `residue_resolve`, `residue_review`, `residue_cancel` | | Data transfer | `transfer_plan`, `transfer_start`, `transfer_checkpoint_record`, `transfer_pause`, `transfer_resume`, `transfer_reconciliation_start`, `transfer_reconciliation_record`, `transfer_failure_record`, `transfer_cancel` | | Deployment requirements | `deployment_request`, `deployment_reference_bind`, `deployment_observe`, `deployment_reconcile` | | Validation executions | `validation_execution_request`, `validation_execution_start`, `validation_execution_complete`, `validation_execution_fail`, `validation_execution_cancel` | | Discrepancies | `discrepancy_create`, `discrepancy_triage`, `discrepancy_accept`, `discrepancy_resolve`, `discrepancy_verify` | | Parity | `parity_certify` | | Readiness | `validation_profile_assign`, `validation_run_record`, `readiness_record`, `business_accept` | | Waivers | `waiver_request`, `waiver_approve` | | Migration certificate | `migration_certificate_mint`, `migration_certificate_invalidate` | | Cutover control | `wave_freeze`, `cutover_runbook_configure`, `cutover_rehearsal_record`, `operational_evidence_record`, `effector_certification_record`, `cutover_incident_open`, `cutover_incident_resolve` | | Cutover execution | `cutover_execute`, `cutover_record`, `cutover_rollback` | | Hypercare | `hypercare_start`, `hypercare_complete`, `decommission_attest` | | Modernization | `modernization_recommend`, `modernization_disposition`, `modernization_plan`, `modernization_start`, `modernization_evidence_record`, `modernization_promote`, `modernization_rework` | | Measured value | `benchmark_record`, `value_summary_mint`, `value_report_publish` | | Source capabilities | `capability_propose`, `capability_evidence_record`, `capability_promote`, `capability_expire`, `capability_revoke`, `capability_reconcile` | | Governance/evidence | `policy_configure`, `evidence_export` | `organization_membership_set` governs the projected organization membership registry: it is tenant-scoped like every other action here — the platform tenant exception is documented for the organization registry alone — and it requires `airlift:membership:manage`. See [Organization membership](/docs/reference/organization-membership). The full IDs are `airlift.`. Action parameters never accept authoritative organization or natural-person identity; they come from invocation context. Use `fa actions` for permissions, policies, versions, and emitted events in the installed package. # Generated capability catalog # Generated capability catalog [#generated-capability-catalog] This page is generated from the versioned Airlift SDK contracts during every documentation build. Do not edit it manually. It describes identifiers and declared developer routing; it does not claim that a provider is installed or certified in your organization. Query the tenant capability registry for evidence-derived coverage. ## Capability identifiers [#capability-identifiers] | Identifier | Developer meaning | | ---------------------------- | --------------------------------------------------------------------------------- | | `assessment` | Metadata collection, profiling, complexity analysis, and inventory normalization. | | `dependency_lineage` | Object, pipeline, scheduler, semantic, and consumer dependencies. | | `sql_conversion` | Deterministic SQL conversion with explicit unsupported constructs. | | `procedural_conversion` | Procedures, functions, packages, macros, and procedural scripts. | | `etl_conversion` | Orchestration, transformation, parameters, dependencies, retries, and triggers. | | `notebook_script_conversion` | Notebooks, Python or shell scripts, and job definitions. | | `bi_semantic_transition` | Semantic models, reports, downstream consumers, and query contracts. | | `data_snapshot` | Restartable baseline data movement with immutable manifests. | | `incremental_cdc` | Change capture, watermarks, schema drift, lag, and catch-up. | | `validation_reconciliation` | Independent schema, data, query, and non-functional evidence. | | `target_generation` | Unity Catalog, Delta, SQL, job, pipeline, and bundle artifacts. | | `deployment_promotion` | Immutable deployment requirements and promotion references. | | `cutover_rollback` | Rehearsed endpoint change, verification, rollback, and recovery. | | `modernization` | Separately measured Databricks-native improvement after baseline parity. | ## Proof levels [#proof-levels] 1. `contract_only` 2. `hermetic_proven` 3. `workspace_proven` 4. `client_proven` 5. `production_certified` Supported preflight targets are `assessable`, `executable`, `certifiable`, `cutover_certified`. ## Delivery modes [#delivery-modes] * `unavailable` * `not_applicable` * `external` * `human` * `automated` * `mixed` An `external`, `human`, or `mixed` route still requires current evidence for the pinned provider generation. `not_applicable` is a governed disposition, not an omitted test. ## Automation dispositions [#automation-dispositions] * `deterministic` * `agent_repairable` * `human_remediation` * `retain_or_federate` * `replace_native` * `retire` * `excluded` * `unsupported` ## Independent required-construct denominator [#independent-required-construct-denominator] Contract version: `1`. Only variants in this table can advance beyond `cataloged`. Every required construct must join to exactly one current, source-version-matched registry-v2 cell at the target proof level. Missing contracts, missing cells, duplicates, stale evidence, version mismatches, and unavailable execution routes fail closed. Declared validation profiles: `airlift.database_object.baseline`, `airlift.data_asset.baseline`, `airlift.orchestration.baseline`, `airlift.notebook_script.baseline`, `airlift.spark.baseline`, `airlift.kql.baseline`, `airlift.security.baseline`, `airlift.networking.baseline`, `airlift.consumer.baseline`, `airlift.operational_dependency.baseline`, `airlift.data_movement.baseline`. | Source | Variant | Source version | Required constructs | | ------------ | ---------------------------- | ------------------------------------ | ------------------- | | `sql_server` | `azure_sql` | `azure_sql_current` | 81 | | `sql_server` | `azure_sql_managed_instance` | `azure_sql_managed_instance_current` | 81 | | `sql_server` | `rds_sql_server` | `rds_sql_server_2022` | 81 | | `sql_server` | `sql_server` | `sql_server_2022` | 81 | | `synapse` | `synapse_dedicated_sql` | `synapse_dedicated_sql_current` | 102 | | `synapse` | `synapse_mixed` | `synapse_mixed_current` | 102 | | `synapse` | `synapse_serverless_sql` | `synapse_serverless_sql_current` | 102 | ## Construct-scoped cells (registry v2) [#construct-scoped-cells-registry-v2] Registry-v2 cells add `constructScope` with `construct`, `artifactKind`, `targetPattern`, `automationDisposition`, optional `sourceVersion`, and optional `documentationReference`. The program term `known_constraints` maps to the replay-safe `limitations` field; `required_validation_profile` maps to `validationProfileIds`. A construct cell proves only the named source construct. One cell never raises the whole source variant's derived support level or clears engagement preflight by itself; advancement requires the complete independent denominator and every mandatory lifecycle capability. ## Repository construct routing [#repository-construct-routing] This repository-owned catalog is the normalization and routing contract consumed by assessment, conversion, artifact admission, the API, the App, and `fa source constructs`. It reports implementation status only. It does not report provider installation, execution, validation, or proof. | Source | Construct | Artifact kind | Disposition | Target pattern | Implementation | | --------- | ------------------------------ | ---------------------- | ------------------- | ------------------------------- | ------------------ | | `synapse` | `adf.blobeventstrigger` | `target_code` | `human_remediation` | `manual_runbook` | `routing_only` | | `synapse` | `adf.blobeventstrigger` | `test` | `human_remediation` | `manual_runbook` | `routing_only` | | `synapse` | `adf.copy` | `target_code` | `deterministic` | `lakeflow_declarative_pipeline` | `native_generator` | | `synapse` | `adf.copy` | `test` | `deterministic` | `lakeflow_declarative_pipeline` | `native_generator` | | `synapse` | `adf.databricksnotebook` | `target_code` | `agent_repairable` | `notebook` | `descriptor_only` | | `synapse` | `adf.databricksnotebook` | `test` | `agent_repairable` | `notebook` | `descriptor_only` | | `synapse` | `adf.dataset` | `target_configuration` | `deterministic` | `bundle_resource` | `native_generator` | | `synapse` | `adf.dataset` | `test` | `deterministic` | `bundle_resource` | `native_generator` | | `synapse` | `adf.executepipeline` | `target_code` | `deterministic` | `databricks_workflow` | `native_generator` | | `synapse` | `adf.executepipeline` | `test` | `deterministic` | `databricks_workflow` | `native_generator` | | `synapse` | `adf.foreach` | `target_code` | `agent_repairable` | `databricks_workflow` | `routing_only` | | `synapse` | `adf.foreach` | `test` | `agent_repairable` | `databricks_workflow` | `routing_only` | | `synapse` | `adf.ifcondition` | `target_code` | `agent_repairable` | `databricks_workflow` | `routing_only` | | `synapse` | `adf.ifcondition` | `test` | `agent_repairable` | `databricks_workflow` | `routing_only` | | `synapse` | `adf.mappingdataflow` | `target_code` | `agent_repairable` | `pyspark_module` | `routing_only` | | `synapse` | `adf.mappingdataflow` | `test` | `agent_repairable` | `pyspark_module` | `routing_only` | | `synapse` | `adf.notebook` | `target_code` | `agent_repairable` | `notebook` | `descriptor_only` | | `synapse` | `adf.notebook` | `test` | `agent_repairable` | `notebook` | `descriptor_only` | | `synapse` | `adf.parameter` | `target_code` | `deterministic` | `bundle_resource` | `native_generator` | | `synapse` | `adf.parameter` | `test` | `deterministic` | `bundle_resource` | `native_generator` | | `synapse` | `adf.pipeline` | `target_code` | `deterministic` | `databricks_workflow` | `native_generator` | | `synapse` | `adf.pipeline` | `test` | `deterministic` | `databricks_workflow` | `native_generator` | | `synapse` | `adf.scheduletrigger` | `target_code` | `deterministic` | `bundle_resource` | `native_generator` | | `synapse` | `adf.scheduletrigger` | `test` | `deterministic` | `bundle_resource` | `native_generator` | | `synapse` | `adf.script` | `target_code` | `agent_repairable` | `databricks_sql` | `routing_only` | | `synapse` | `adf.script` | `test` | `agent_repairable` | `databricks_sql` | `routing_only` | | `synapse` | `adf.sqlserverstoredprocedure` | `target_code` | `agent_repairable` | `databricks_sql` | `descriptor_only` | | `synapse` | `adf.sqlserverstoredprocedure` | `test` | `agent_repairable` | `databricks_sql` | `descriptor_only` | | `synapse` | `adf.tumblingwindowtrigger` | `target_code` | `agent_repairable` | `bundle_resource` | `routing_only` | | `synapse` | `adf.tumblingwindowtrigger` | `test` | `agent_repairable` | `bundle_resource` | `routing_only` | | `synapse` | `adf.until` | `target_code` | `agent_repairable` | `databricks_workflow` | `routing_only` | | `synapse` | `adf.until` | `test` | `agent_repairable` | `databricks_workflow` | `routing_only` | | `synapse` | `adf.variable` | `target_code` | `deterministic` | `bundle_resource` | `native_generator` | | `synapse` | `adf.variable` | `test` | `deterministic` | `bundle_resource` | `native_generator` | | `synapse` | `adf.webactivity` | `target_code` | `human_remediation` | `manual_runbook` | `routing_only` | | `synapse` | `adf.webactivity` | `test` | `human_remediation` | `manual_runbook` | `routing_only` | | `synapse` | `synapse.consumer` | `runbook` | `deterministic` | `consumer_repointing_runbook` | `descriptor_only` | | `synapse` | `synapse.copy_statement` | `target_code` | `deterministic` | `lakeflow_pipeline` | `descriptor_only` | | `synapse` | `synapse.external_table` | `target_code` | `deterministic` | `unity_catalog_external_table` | `descriptor_only` | | `synapse` | `synapse.function` | `target_code` | `deterministic` | `databricks_sql_function` | `descriptor_only` | | `synapse` | `synapse.linked_service` | `target_configuration` | `deterministic` | `connection_binding` | `descriptor_only` | | `synapse` | `synapse.notebook` | `target_code` | `deterministic` | `databricks_notebook` | `descriptor_only` | | `synapse` | `synapse.permission` | `target_configuration` | `deterministic` | `unity_catalog_grant` | `descriptor_only` | | `synapse` | `synapse.pipeline` | `target_configuration` | `deterministic` | `lakeflow_job` | `descriptor_only` | | `synapse` | `synapse.polybase_object` | `target_code` | `deterministic` | `lakeflow_pipeline` | `descriptor_only` | | `synapse` | `synapse.schema` | `target_configuration` | `deterministic` | `unity_catalog_schema` | `descriptor_only` | | `synapse` | `synapse.stored_procedure` | `target_code` | `deterministic` | `databricks_sql_or_workflow` | `descriptor_only` | | `synapse` | `synapse.table` | `target_code` | `deterministic` | `delta_table` | `descriptor_only` | | `synapse` | `synapse.trigger` | `target_configuration` | `deterministic` | `lakeflow_job_trigger` | `descriptor_only` | | `synapse` | `synapse.view` | `target_code` | `deterministic` | `databricks_sql_view` | `descriptor_only` | | `synapse` | `synapse.workload_group` | `target_configuration` | `deterministic` | `sql_warehouse_policy` | `descriptor_only` | ## Databricks target-integration states [#databricks-target-integration-states] These construct-independent, source-independent integrations are declared identifiers. Declaring one does not configure, connect, observe, or prove anything. They are not `MigrationCapability` members and never aggregate into a source variant's derived support level. | Integration | Developer meaning | Owner of the underlying truth | | ----------------- | --------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- | | `uc_governance` | Unity Catalog target objects, permissions, grants, classifications, and policies for migrated assets. | Unity Catalog owns target objects, permissions, grants, classifications, and native lineage state | | `uc_lineage` | Unity Catalog native and runtime lineage reconciled from accepted source dependencies to target objects and downstream consumers. | Unity Catalog owns native and runtime lineage state | | `ai_gateway` | Unity AI Gateway and Model Serving model routes, policies, request tags, and usage observations for bounded migration assistance. | Databricks AI Gateway / Model Serving owns model-route observations and usage state | | `ai_bi_reporting` | Databricks AI/BI dashboards that present governed Airlift migration projections to evaluators and operators. | Databricks AI/BI owns dashboard resources and presentation | | `genie_analytics` | Databricks Genie analytics over curated, read-only Airlift migration views; distinct from the bounded Airlift assistant. | Databricks Genie owns analytics-space execution | Every Airlift surface renders exactly one observed state per integration: | State | Meaning | Can carry proof? | | ---------------- | ---------------------------------------------------------------------------------------------------- | ----------------------------------------- | | `not_configured` | no admitted configuration for this integration | no | | `not_observed` | configured, but no admitted workspace observation (or observation evidence below workspace strength) | no | | `stale` | the last admitted observation is outside its freshness window | no | | `blocked` | a named blocker prevents observation | no | | `observed` | a current admitted observation exists | yes — derived only from admitted evidence | Configuration, an environment variable, a catalog declaration, a screenshot, dashboard state, model output, and caller-declared metadata never raise proof and never render as `observed`. Dashboards, Genie, model output, screenshots, and provider configuration have no mutation, approval, waiver, certificate, deployment, cutover, or proof authority. ## Source and variant identifiers [#source-and-variant-identifiers] Static implementation routing describes shipped developer workflow breadth; it is not workspace or client proof. Only the authenticated capability matrix can render an organization's current, evidence-derived state. | Source id | Label | Implementation routing (non-evidence) | Variant ids | | ------------------- | -------------------------------- | ------------------------------------- | ----------------------------------------------------------------------------------------------- | | `athena` | Amazon Athena | `cataloged` | `athena` | | `kinesis` | Amazon Kinesis | `cataloged` | `kinesis_data_streams`, `kinesis_firehose` | | `redshift` | Amazon Redshift | `executable` | `redshift_serverless`, `redshift_provisioned`, `redshift_provisioned_multi_az` | | `kafka` | Apache Kafka | `cataloged` | `apache_kafka`, `confluent_cloud`, `amazon_msk` | | `event_hubs` | Azure Event Hubs | `cataloged` | `azure_event_hubs` | | `synapse` | Azure Synapse | `executable` | `synapse_dedicated_sql`, `synapse_serverless_sql`, `synapse_mixed` | | `databricks` | Databricks native estate | `executable` | `databricks_unity_catalog`, `databricks_workspace_consolidation`, `databricks_cost_performance` | | `etl_modernization` | ETL and orchestration platforms | `cataloged` | `informatica`, `ssis`, `azure_data_factory`, `aws_glue`, `talend`, `matillion`, `oracle_odi` | | `bigquery` | Google BigQuery | `assessable` | `bigquery` | | `greenplum` | Greenplum | `cataloged` | `greenplum` | | `hadoop` | Hadoop | `executable` | `cloudera`, `hortonworks`, `apache_hadoop` | | `db2` | IBM Db2 and mainframe data | `cataloged` | `db2_luw`, `db2_zos`, `db2_i` | | `dynamics_365` | Microsoft Dynamics 365 | `cataloged` | `dynamics_365_dataverse`, `dynamics_365_finance_operations`, `dynamics_365_sales_service` | | `mysql` | MySQL and Aurora MySQL | `cataloged` | `mysql`, `aurora_mysql`, `rds_mysql`, `mariadb` | | `netezza` | Netezza | `assessable` | `netezza_performance_server` | | `oracle` | Oracle | `executable` | `oracle_database`, `oracle_exadata`, `oracle_autonomous` | | `postgresql` | PostgreSQL and Aurora PostgreSQL | `cataloged` | `postgresql`, `aurora_postgresql`, `rds_postgresql` | | `presto` | Presto | `cataloged` | `presto` | | `salesforce` | Salesforce | `cataloged` | `salesforce_sales_service`, `salesforce_marketing_cloud` | | `sap` | SAP | `cataloged` | `sap_bdc`, `sap_s4hana`, `sap_ecc`, `sap_bw4hana`, `sap_hana`, `sap_datasphere` | | `servicenow` | ServiceNow | `cataloged` | `servicenow_platform` | | `snowflake` | Snowflake | `executable` | `snowflake` | | `sql_server` | SQL Server | `executable` | `sql_server`, `azure_sql`, `azure_sql_managed_instance`, `rds_sql_server` | | `teradata` | Teradata | `executable` | `teradata_vantage`, `teradata_appliance` | | `trino` | Trino | `cataloged` | `trino`, `starburst` | | `vertica` | Vertica | `cataloged` | `vertica` | | `workday` | Workday | `cataloged` | `workday_hcm`, `workday_reports` | ## Runtime checks [#runtime-checks] ```bash fa source doctor synapse --variant synapse_dedicated_sql fa source constructs synapse --variant synapse_dedicated_sql fa source limitations synapse --variant synapse_dedicated_sql fa source certify synapse --variant synapse_dedicated_sql --level certifiable fa engagement preflight eng_ ``` These commands query the authenticated organization registry. `source certify` is a fail-closed readiness check; it does not mint a migration certificate or promote a provider claim. # Configuration reference # Configuration reference [#configuration-reference] ## Store and identity [#store-and-identity] * `AIRLIFT_STORE=postgres` * Lakebase App binding: `DATABRICKS_LAKEBASE_ENDPOINT` or `ENDPOINT_NAME`, with injected `PGHOST`, `PGDATABASE`, and `PGUSER` * Direct Postgres: `AIRLIFT_DATABASE_URL` or the `AIRLIFT_PG*` variables * `AIRLIFT_AUTHORIZATION_JSON` — bootstrap membership snapshot only; after first governed adoption the [organization membership registry](/docs/reference/organization-membership) replaces env rows, tenant admins manage members in the **Team access** App page, and env changes do not grant access * `AIRLIFT_TRUST_DATABRICKS_APP_HEADERS=1` and `DATABRICKS_APP_NAME` * `AIRLIFT_PSEUDONYM_KEY` — dedicated App secret used to derive stable participant, deployment-principal, and workspace refs without storing raw identities in assurance evidence * `AIRLIFT_PROJECTION_REFRESH_MS` ## Airlift Migration Assistant [#airlift-migration-assistant] * `AIRLIFT_GENIE_PROVIDER` — `auto`, `deterministic`, `unity-ai-gateway`, `model-serving`, `openai-compatible`, `anthropic`, or `azure-openai` * `AIRLIFT_GENIE_MODEL` — approved Unity Catalog model service, Model Serving endpoint, or external model name * `DATABRICKS_INFERENCE_MODE` — `auto`, `ai-gateway`, or `serving-endpoints` * `AIRLIFT_GENIE_BASE_URL` — external-provider or Azure OpenAI URL * `AIRLIFT_GENIE_DEPLOYMENT` and optional `AIRLIFT_GENIE_API_VERSION` — Azure OpenAI * `AIRLIFT_GENIE_API_KEY` — external-provider key from a Databricks App secret resource; never a literal configuration value * `AIRLIFT_GENIE_AGENT_ID` — optional curated Genie Agent for migration analytics The preferred production path is Unity AI Gateway or Model Serving through the Databricks App service principal. A missing model, permission, secret, timeout, invalid response, or budget failure falls back to deterministic Airlift guidance and cannot weaken a gate. See [Airlift Migration Assistant](/docs/integrations/airlift-genie) and [Engineering remediation](/docs/migration/residue#airlift-migration-assistant). ## Development-only workflow exercise [#development-only-workflow-exercise] * `AIRLIFT_DEPLOYMENT_ENV=dev` * `AIRLIFT_ALLOW_DEVELOPMENT_SELF_REVIEW=1` * `DATABRICKS_APP_NAME` ending in `-dev` * `AIRLIFT_DEVELOPMENT_OPERATOR_PRINCIPAL` * `AIRLIFT_DEVELOPMENT_WORKER_PRINCIPAL` Together these allow one authenticated developer to resolve and review the same residue while exercising a dev App. The event is marked `development_self_review`. An admitted system principal may then mint an explicitly `development` certificate, which the production cutover policy always rejects. The flag fails startup outside the complete dev boundary and must not be configured in staging or production. The Synapse certification command uses the schema installed by deployment and never requests schema DDL under a developer identity. See [Development assurance](/docs/operations/development-assurance) for the complete boundary and inspection commands. ## Evidence and certificates [#evidence-and-certificates] * `AIRLIFT_EVIDENCE_REGISTRY_JSON` * `AIRLIFT_EVIDENCE_SIGNING_KEY_ID` * `AIRLIFT_EVIDENCE_SIGNING_PRIVATE_KEY_PEM` * `AIRLIFT_EVIDENCE_VERIFY_KEYS_JSON` Each `AIRLIFT_EVIDENCE_REGISTRY_JSON` entry requires `organizationId`, `producerPrincipal`, `provider`, `providerRunRef`, `evidenceRef`, and `evidenceDigest`. It may also carry `artifactDigest`, `verdict`, and `completedAt`. Any declared field is **bound**: it must match the submitted run exactly, so a manifest naming one artifact and verdict cannot verify a different result smuggled in under the same evidence identity. Two of those fields can additionally be **attested** — reported as established by the provider rather than merely recorded next to it. Only `verdict` and `completedAt` qualify, because a provider's run document reports them. `artifactDigest` never does: provider run documents generally do not name the artifact a run exercised, so Airlift binds that field without presenting it as provider proof and lets the conversion-hazard gate corroborate it at certification against the artifact the certificate is effective for. How attestation is established depends on the verifier. A provider-backed verifier — such as the worker's Volume-backed evidence store — re-reads the provider's evidence body, checks its digest, and re-derives the verdict and completion time from it, reporting a field as attested only when its derived value equals what was submitted. The two fields are judged independently: evidence whose recorded outcome contradicts the submitted verdict, but whose completion time matches, attests `completedAt` and not `verdict` — which still fails a governed conversion, because that requires both. A body that is not a provider document at all derives nothing and so attests neither. This static manifest cannot do that: it never sees an evidence body, so it establishes nothing and reports no attested fields at all. That is deliberate. It is also the verifier Airlift falls back to whenever no provider-backed one is injected, and an operator writing outcome fields into configuration is not provider proof — treating it as such would defeat the gate. (Manifests that once set `providerAttested` are rejected outright rather than silently downgraded, so the change is visible.) The practical consequence: **a governed Teradata/Lakebridge conversion fails closed under the manifest fallback.** Its validation runs are refused with a denial naming this variable and the service to supply, until a provider-backed verifier is injected as the `airliftEvidenceVerifier` runtime service — for Experiments, the worker's Volume-backed evidence store, which reads the published evidence document and derives the outcome from it. Manifest entries remain fully usable for conversions outside a governed hazard profile. ## Lakebridge [#lakebridge] * `AIRLIFT_CONVERTER=lakebridge` * `DATABRICKS_HOST`, `DATABRICKS_CLIENT_ID`, `DATABRICKS_CLIENT_SECRET` * `AIRLIFT_LAKEBRIDGE_ANALYZER_JOB_ID` * `AIRLIFT_LAKEBRIDGE_CONVERTER_JOB_ID` * `AIRLIFT_LAKEBRIDGE_VOLUME_ROOT` * optional certified `AIRLIFT_LAKEBRIDGE_VERSION` ## Temporal worker [#temporal-worker] * `AIRLIFT_TEMPORAL_MODE=temporal` * `TEMPORAL_ADDRESS`, `TEMPORAL_NAMESPACE`, TLS/API-key variables or their supported Harness `FABRIC_TEMPORAL_*` equivalents * `AIRLIFT_TASK_QUEUE` * `AIRLIFT_APPROVAL_TIMEOUT_MINUTES` * `AIRLIFT_SWEEP_CRON`, `AIRLIFT_SWEEP_MAX_ATTEMPTS` ## Runway deployment observation [#runway-deployment-observation] * `AIRLIFT_DEPLOYMENT_ADAPTER=runway` * `AIRLIFT_RUNWAY_API_URL` * `AIRLIFT_RUNWAY_APP_URL` (optional console deep link) * `AIRLIFT_RUNWAY_API_TOKEN` from a deployment secret This adapter observes a deployment created by `fr`; it does not deploy, promote, reconcile, or roll back a release. Before observation, connect the Airlift requirement to a `runway/deployment` reference containing the Runway deployment ID and staged artifact digest. There is no environment-only switch that certifies a cutover effector. It is explicitly injected after client certification. Never store secret values in source, docs, action payloads, workflow inputs, or evidence. # Errors and recovery reference # Errors and recovery reference [#errors-and-recovery-reference] | Class | Meaning | Operator response | | ---------------------- | -------------------------------------------------------------------- | ------------------------------------------------------------------- | | Authorization denial | principal lacks tenant/type/action authority | correct membership/admission; do not retry as another claimed actor | | `ComplianceBlocked` | policy or governed gate denied execution | resolve the named evidence/state blocker | | Schema/state rejection | malformed input or invalid transition | correct the command or current object state | | Idempotency conflict | same key reused with different parameters | stop; issue a new logical command only if intent changed | | Adapter failure | external analyzer/converter/effect did not complete | follow retry classification and reconciliation evidence | | Evidence rejection | producer, run, reference, digest, watermark, or version not admitted | correct the authoritative evidence record | | Stale certificate | material dependency changed or waiver expired | revalidate and mint a new certificate | | Workflow not found | typed Temporal absence | confirm workflow ID/start status | | Temporal/auth outage | operational failure, not absence | restore service/auth; error must propagate | | Uncertain cutover | external effect cannot be proven success/failure | remain executing; reconcile from checkpoint and external state | Never turn an operational outage into “not found,” or a failed validation into a warning merely to advance the wave. # Events and identifiers # Events and identifiers [#events-and-identifiers] | Entity | Shape | | --------------------- | ----------------------------------- | | Organization | lowercase slug, up to 63 characters | | Estate | `est_` | | Migration object | `obj_` | | Wave | `wav_` | | Conversion | `cnv_` | | Assessment | `asm_` | | Parity certificate | `crt_` | | Validation run | `val_` | | Waiver | `wvr_` | | Migration certificate | `mct_` | | Dependency graph | `dpg_` | | Migration plan | `pln_` | | Immutable artifact | `art_` | | Conversion batch | `cbh_` | | Residue case | `res_` | Domain subjects include `Organization`, `Estate`, `MigrationObject`, `Wave`, `Engagement`, `ConnectionBinding`, `DependencyGraph`, `MigrationPlan`, `Artifact`, `ConversionBatch`, and `Residue`. Events use past-tense Airlift names such as `AirliftAssessmentRecorded`, `AirliftConversionRecorded`, `AirliftReadinessRecorded`, `AirliftMigrationCertificateMinted`, and `AirliftCutoverRecorded`. Discovery and planning also emit `AirliftAssessmentAccepted`, `AirliftDependencyGraphStarted`, `AirliftDependencyBatchRecorded`, `AirliftDependencyGraphAccepted`, `AirliftMigrationPlanGenerated`, `AirliftMigrationPlanSelected`, and `AirliftMigrationPlanFrozen`. The conversion factory emits `AirliftArtifactRegistered`, `AirliftConversionBatchCreated`, `AirliftConversionBatchStarted`, `AirliftConversionBatchCompleted`, and the `AirliftResidue*` lifecycle events. Source access emits `AirliftConnectionBindingRegistered`, `AirliftConnectionBindingVerified`, `AirliftConnectionBindingRetired`, and `AirliftConnectionDiagnosticRecorded` (an admitted connectivity diagnostic bound to a connection binding; its digest is derived by the handler, never supplied by the caller). Databricks access emits `AirliftAccessPreflightRecorded`. V1 and V2 `deployment` records describe the App service principal; V2 `participant` records bind one pseudonymous user and workspace for evaluator readiness. Participant records never satisfy deployment reachability, and deployment records never satisfy participant readiness. The handler derives every digest and overall state; access observations never raise target integration state to `observed`. Evaluation readiness emits `AirliftEvaluationCandidateFrozen` and `AirliftEvaluatorRehearsalRecorded` (an admitted candidate freeze and rehearsal; both digests are handler-derived, and `evaluation_ready` is derived only — never stored on either event). Unity Catalog evidence emits `AirliftUcProjectionRunRecorded` and `AirliftUcTargetReconciliationRecorded` (an admitted analytics projection run and a per-artifact target reconciliation; both digests are handler-derived, freshness derives from the cursor watermark, and a current evidence state promotes nothing). Platform also emits `ComplianceBlocked`, `AdapterInvocationStarted`, `AdapterInvocationSucceeded`, and `AdapterInvocationFailed`. Authorization can reject a principal before domain/policy execution; an authorization denial therefore does not promise a `ComplianceBlocked` event. Every envelope carries the organization as `tenantId`, plus invocation/correlation identity. Projections derive tenant scope from the envelope, not an event payload field. # Airlift reference # Airlift reference [#airlift-reference] Use this section when implementing an adapter, reviewing a policy, diagnosing a deployment, or reconciling generated evidence. Repository exports and contract tests remain authoritative for exact schemas. * [Action catalog](/docs/reference/action-catalog) * [Events and identifiers](/docs/reference/events-and-identifiers) * [Organization membership](/docs/reference/organization-membership) * [Validation profiles](/docs/reference/validation-profiles) * [Source registry API](/docs/reference/source-registry) * [Source-system developer guides](/docs/sources) * [CLI command reference](/docs/cli/command-reference) * [Configuration](/docs/reference/configuration) * [Errors and recovery](/docs/reference/errors) * [Security model](/docs/reference/security) Run `fa actions --json`, `fa profiles --json`, and `fa sources --json` against the installed release to compare its machine-readable contracts with these pages. Use `createSourceMigrationPlan()` when an application needs the same source plan without invoking a child process. # Organization membership # Organization membership [#organization-membership] Airlift roles (`viewer`, `operator`, `approver`, `validator`, `admin`, and the narrow `automation` role for service accounts) are granted through a **governed organization membership registry**: a projected admin registry built from the durable event stream, one projection per organization. Install-time configuration (`AIRLIFT_AUTHORIZATION_JSON`) seeds the registry exactly once. At startup the console adopts the configured rows for each active organization into the governed stream through the platform-installer system principal, using a digest-bound idempotency key so a restart replays one logical command. From the moment any governed row exists for an organization, the environment directory is **never consulted again** for that organization — not even as a fallback. If the governed reader exists and returns no rows, every principal is denied (fail closed), and that organization is marked unavailable when it has no bootstrap members to adopt; other correctly initialized organizations remain healthy. After bootstrap, membership changes are governed mutations, not configuration edits. ## The action contract [#the-action-contract] `airlift.organization_membership_set` uses a revisioned version-2 request and applies one atomic batch of changes: ```json { "submissionId": "team-access-7c9f6d", "changes": [ { "principal": "teammate@example.com", "principalType": "natural_person", "role": "operator", "status": "active", "expectedRevision": "absent" }, { "principal": "departed-admin@example.com", "principalType": "natural_person", "role": "admin", "status": "revoked", "expectedRevision": "event:01JEXAMPLE" }, { "principal": "onboarding-runner", "principalType": "service_account", "role": "automation", "status": "active", "expectedRevision": "absent" } ] } ``` * **Tenant-free payload.** The strict schema accepts `submissionId` plus `changes` (1–100 entries). Actor and organization are never payload fields; the authenticated API derives the organization from the caller's single resolved membership. * **Typed principals.** A `natural_person` holds `viewer`, `operator`, `approver`, `validator`, or `admin`. A `service_account` holds only `automation`. That role covers bounded operational recording and execution across assessment, inventory, planning, transfer, validation, discrepancy, observation, and evidence lanes; it never carries approval, waiver, certificate, policy, membership, or cutover-execution authority. `status` is `active` or `revoked`. * **Permission.** The action requires `airlift:membership:manage`, granted to the `admin` role only. The `automation` role is excluded, and no agent, worker, or evidence producer is admitted. The platform installer is admitted only for the startup bootstrap path. * **Last-admin protection.** The handler projects the resulting registry and fails the whole batch when it would leave zero active natural-person admins. Grants and revocations in one batch are evaluated against the projected result, never per row. The API returns `last_admin_conflict` with HTTP 409. * **Optimistic concurrency.** Every row carries `expectedRevision` (`absent` for a new principal or the projected `event:...` revision for an update). A stale row rejects the whole batch as `membership_conflict` with HTTP 409; no partial event is appended. * **Governed revocation.** A revoked row stops authorizing as soon as the projection updates, and Platform Host re-authorizes at execution time, so a durable invocation cannot retain a revoked grant. * **Idempotency.** `submissionId`, actor, tenant, canonical ordered batch, and expected revisions bind one logical attempt. An exact transport retry collapses to the original invocation; a new user gesture uses a new submission ID. ### Canonical automation permissions [#canonical-automation-permissions] This block is machine-checked against `AIRLIFT_ROLE_PERMISSIONS.automation`: {/* AIRLIFT_AUTOMATION_PERMISSIONS:START */} ```json [ "airlift:read", "airlift:assessment:record", "airlift:assessment:start", "airlift:assessment:export", "airlift:object:register", "airlift:object:exclude", "airlift:dependency:record", "airlift:plan:generate", "airlift:engagement:create", "airlift:engagement:update", "airlift:engagement:activate", "airlift:engagement:freeze", "airlift:connection:register", "airlift:connection:verify", "airlift:connection:retire", "airlift:artifact:register", "airlift:residue:create", "airlift:residue:estimate", "airlift:residue:resolve", "airlift:transfer:start", "airlift:transfer:checkpoint", "airlift:transfer:reconcile", "airlift:transfer:failure", "airlift:deployment:observe", "airlift:deployment:reconcile", "airlift:validation:execute", "airlift:discrepancy:create", "airlift:discrepancy:triage", "airlift:discrepancy:verify", "airlift:cutover:rehearsal", "airlift:cutover:observe", "airlift:cutover:incident", "airlift:modernization:recommend", "airlift:modernization:evidence", "airlift:value:record", "airlift:connection:diagnose", "airlift:access:observe", "airlift:evaluator:observe" ] ``` {/* AIRLIFT_AUTOMATION_PERMISSIONS:END */} ## CLI [#cli] ```bash fa organization membership set \ --file membership-changes.json \ --idempotency-key membership-grant-2026-08 fa organization membership set \ --file membership-changes.json \ --idempotency-key membership-grant-2026-08 \ --json ``` The command is a transport client for the action above: it forwards the file as the action `params` unchanged and lets the server schema reject anything outside the governed shape. Use a stable, non-secret idempotency prefix; the CLI binds it to the canonical request digest. Exit codes follow the shared remote contract (`0` applied, `2` usage or invalid request, `3` unauthenticated or forbidden, `5` blocked, `6` unavailable). There is deliberately no `fa organization membership list`: the authenticated read surface has no membership resource, and the CLI never opens a second read path. Read the projected registry in the App instead. ## Team access page (App) [#team-access-page-app] The **Team access** page of the Airlift App is the admin surface for this registry. It renders the projected memberships for the current organization — principal, principal type, role, and an active/revoked status pill — sorted by principal, with an explicit empty state when no governed rows have been adopted yet. Who can manage members, and what a denied state means: * **Natural-person tenant admins only.** The page requires `airlift:membership:manage` before listing any principal or role, and the governed action independently re-checks it at execution time. Ordinary members receive no tenant roster. A service account — including one holding the `automation` role — is denied with "Only a signed-in tenant administrator can manage team access." Databricks workspace identity can deny access to the App, but it never grants an Airlift organization role. * **One change per submission.** The form takes the exact Databricks UC principal or service-principal ID, a principal type, a role, and a status (`active` or `revoked`). Choosing *Service account* pins the role to `automation`; people hold `viewer`, `operator`, `approver`, `validator`, or `admin`. Principal strings with whitespace, control characters, or a `secret:`/`token:`/`credential:` prefix are rejected before the action is invoked — credentials are never membership input. * **Governed mutation, never a store write.** The submission invokes `airlift.organization_membership_set` with a content-digest idempotency key, so an exact resubmission collapses to the original invocation. A governed denial (including last-admin protection) is returned verbatim as the form message; nothing is applied partially. * **Recovery.** If a batch fails because it would leave zero active natural-person admins, grant `admin` to another person first and revoke the departing admin in a second submission. For any other denial, correct the named input and resubmit — the digest-bound key makes the retry safe. See [When something fails](/docs/getting-started/when-something-fails/) for the shared blocked-state model. After bootstrap, this page and `fa organization membership set` are the only membership surfaces: install-time configuration (`AIRLIFT_AUTHORIZATION_JSON`) is never consulted again once any governed row exists for the organization, not even as a fallback. ## Failure recovery [#failure-recovery] * **Zero-active-admin prevention.** A batch that would demote or revoke the last active admin fails whole. Grant `admin` to another natural person first, then revoke the departing admin in a second mutation with a fresh idempotency prefix. * **Bootstrap failure.** A missing or divergent bootstrap directory marks only that organization unavailable (`bootstrap_conflict` for divergent input); healthy tenants continue serving. Correct the rows and restart under the same digest-bound attempt. * **Revocation propagation.** Protected reads refresh the registry and tenant at request time; mutations refresh once for admission and again immediately before effect. Cross-replica revocation therefore denies the next request without waiting for polling. ## Honest support boundary [#honest-support-boundary] Organization membership governs Airlift application roles only. It never grants or revokes Databricks workspace entitlements, App `CAN_USE` permission, Unity Catalog privileges, or Runway/Experiments authority — those remain in their owning systems. Membership rows are install-time seed input until adoption; after adoption, the projected registry is the only grant source and this page, the action catalog, and the repository contract tests are the authoritative contract. # Releases and versioning # Releases and versioning [#releases-and-versioning] Airlift publishes independently versioned npm packages. Install the CLI when you need the `fa` command and install the SDK when you are building an application integration. | Package | Purpose | Current version | | --------------------------------------- | ---------------------------------------------------------------------------------------- | --------------- | | `@fabricorg/airlift` | Typed migration contracts, schemas, projections, and runtime composition | `0.17.0` | | `@fabricorg/airlift-cli` | The authenticated `fa` developer and automation command | `0.18.4` | | `@fabricorg/airlift-adapter-adf` | Credential-safe ADF/Synapse export import and target-artifact generation used by the CLI | `0.3.4` | | `@fabricorg/airlift-adapter-lakebridge` | Lakebridge workspace-job adapter seam | `0.2.0` | | `@fabricorg/airlift-worker` | Temporal worker and validation/cutover composition entry points | `0.15.0` | ## Version independence [#version-independence] Package versions do not move in lockstep. Each CLI release pins the exact SDK and adapter generations it supports. Inspect a particular release before changing an automated environment: ```bash npm view @fabricorg/airlift-cli@0.18.4 dependencies --json npm ls @fabricorg/airlift @fabricorg/airlift-adapter-adf ``` Do not infer compatibility from matching major or minor numbers. ## Reproducible installation [#reproducible-installation] Use exact versions for CI and unattended execution: ```bash npm install --save-exact @fabricorg/airlift@0.17.0 npm install --save-dev --save-exact @fabricorg/airlift-cli@0.18.4 npx --yes --package @fabricorg/airlift-cli@0.18.4 fa version ``` Before upgrading, read the command reference for removed or renamed commands and run your engagement automation against a non-production organization. ## Verify package provenance [#verify-package-provenance] npm exposes the source and release commit metadata recorded during publication: ```bash npm view @fabricorg/airlift-cli@0.18.4 repository gitHead --json ``` For a new release, require both fields and retain the resolved package integrity from the lockfile with your build evidence. A missing source or commit reference should block a regulated release until the package publisher supplies traceable metadata. # Security model # Security model [#security-model] ## Identity and tenancy [#identity-and-tenancy] The console resolves Databricks Apps forwarded identity only when the explicit trust flag and App name are present and the workspace identity boundary is authenticated. Actor and organization never come from a form, workflow signal, action payload, or CLI flag. ## Authority [#authority] Platform Host evaluates entitlement, submission authorization, execution-time reauthorization, policies, state transitions, and approval authorization. Trusted workers, agents, installers, and evidence producers have separate narrow admission sets. Agents never receive certification, acceptance, waiver, approval, policy, cutover, or rollback permissions. ## Secrets and evidence [#secrets-and-evidence] Secrets remain in Databricks or deployment secret stores. Durable records carry opaque references and redact common token, key, password, and secret field shapes. Certificate private keys are secret-backed; offline verification uses public keys. ## External effects [#external-effects] Adapters define timeout, retry class, provider idempotency, checkpoint, redaction, and reconciliation. A non-idempotent cutover effect is attempted once and independently verified. Unknown outcomes fail closed. Production readiness additionally requires spoof, OBO, cross-tenant, agent-bypass, separation-of-duties, signature-tamper, replay, cancellation, restart, and source-and-target integration negative tests. # Source registry API # Source registry API [#source-registry-api] Import the static, versioned registry from `@fabricorg/airlift`. It contains no credentials, network clients, or client state. ## Main exports [#main-exports] | Export | Purpose | | ------------------------------------------- | ----------------------------------------------------- | | `SOURCE_SYSTEM_PROFILES` | canonical profile record keyed by source ID | | `SOURCE_MIGRATION_AREAS` | ten mandatory migration areas | | `SOURCE_SUPPORT_LEVELS` | ordered evidence claim vocabulary | | `resolveSourceSystemProfile(value)` | resolve a canonical ID or alias | | `resolveSourceVariant(profile, value)` | validate a variant belongs to its source | | `createSourceMigrationPlan(value, options)` | compile the current archetype-aware plan | | `sourcePackCertificationManifestSchema` | validate immutable live-run input | | `evaluateSourcePackCertification(input)` | fail closed on missing evidence for a requested level | ## Resolve and plan [#resolve-and-plan] ```ts import { createSourceMigrationPlan, resolveSourceSystemProfile, resolveSourceVariant, } from '@fabricorg/airlift'; const profile = resolveSourceSystemProfile('confluent'); const variant = resolveSourceVariant(profile, 'confluent_cloud'); const plan = createSourceMigrationPlan(profile.id, { variant: variant.id }); profile.archetype; // streaming profile.implementationRoutingLevel; // cataloged; static implementation routing only profile.adapter.movementOptions; plan.schemaVersion; // 3 plan.steps[2]?.id; // parallel_run ``` Unknown sources and mismatched variants throw. Do not substitute a generic profile. ## Plan-v3 shape [#plan-v3-shape] Every plan includes the canonical source and variant, archetype, non-evidentiary implementation routing, program track, ten common areas, adapter contract, steps, external evidence gates, and source-specific surfaces. Implementation routing is never achieved proof. Every step includes: * `goal`; * `areas`; * `specialistCommands`; * exact governed `airliftActions`; * `outputs`; * `exitCriteria`. The compiler is deterministic and performs no I/O. ## Evaluate a certification manifest [#evaluate-a-certification-manifest] ```ts const result = evaluateSourcePackCertification({ schemaVersion: 1, sourceSystem: 'sap', sourceVariant: 'sap_s4hana', requestedLevel: 'executable', buildRef: 'builds/airlift-release', datasetRef: 'datasets/sap-representative', inventoryRun, transferRun, validationRun, }); if (!result.eligible) throw new Error(`Missing: ${result.missing.join(', ')}`); ``` Each run carries an immutable reference, SHA-256 digest, tool version, and completion time. Passing this check is evidence for review; it does not update the registry or mint a migration certificate. # Validation profile reference # Validation profile reference [#validation-profile-reference] Built-in version-1 profiles cover `table`, `view`, `stored_procedure`, `function`, `etl_job`, and `report`. Each contains exactly one requirement for all nine readiness tracks and marks it required or not applicable. ## Tracks and verdicts [#tracks-and-verdicts] Tracks: `inventory`, `target_design`, `code`, `data_movement`, `deployment`, `functional_parity`, `non_functional`, `business_acceptance`, and `cutover_readiness`. Verdicts: `pending`, `passed`, `failed`, `stale`, and `waived`. `not_applicable` is profile applicability, not a verdict. ## Validation providers [#validation-providers] `experiments`, `lakebridge_reconcile`, `harness`, `runway`, `radar`, and `external`. Business acceptance is produced by a separate natural-person decision. ## Parity depth [#parity-depth] Parity evidence increases from `row_count` to `aggregates`, `sampled_rows`, and `full_checksum`. A stronger label requires its supporting fields. Parity depth never claims deployment, security, operability, business acceptance, or cutover readiness. Run `fa profiles --json` for the canonical digest and exact required tracks of each installed profile. # Athena to Databricks # Athena to Databricks [#athena-to-databricks] ```bash fa source inspect athena fa source plan athena ``` Athena is a query-engine migration. Inventory Glue Catalog objects, S3 layouts and file formats, partitions, views, queries, workgroups, scheduled queries, federated connectors, custom SerDes, access rules, and consumers. For each catalog, Airlift records an explicit route: govern an existing external location, copy data, convert it to Delta, federate temporarily, translate SQL, or assign a human-owned lane. It does not assume every table should be moved. The transfer or registration adapter returns manifests, partition state, incremental boundaries, restart, and reconciliation evidence. Validation covers Presto SQL semantics, partition discovery, file-format behavior, functions, access rules, business queries, and accepted performance thresholds. A produced query or successfully registered table is not a certificate. Airlift mints from independent evidence, freezes the consumer wave, and records the verifiable switch or rollback. Modernization targets include Unity Catalog external locations, Delta materialization, liquid clustering, Lakeflow, and Databricks SQL. The pack is `cataloged`. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe athena fa source recipe athena --variant athena --json > .airlift/athena-recipe.json ``` The App is engagement-aware. Amazon Athena appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect athena --json > .airlift/athena-profile.json fa source plan athena --variant athena --json > .airlift/athena-capability-plan.json ``` Expected artifacts: * .airlift/athena-profile.json * .airlift/athena-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file athena-estate.json --idempotency-key athena-estate-v1 fa connection register --file athena-connection.json --idempotency-key athena-connection-v1 fa engagement update --file athena-scope.json --idempotency-key athena-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file athena-assessment-start.json --idempotency-key athena-assessment-start-v1 fa assessment status --json fa assessment record --file athena-assessment-record.json --idempotency-key athena-assessment-record-v1 fa assessment accept --file athena-assessment-accept.json --idempotency-key athena-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/athena` after replacing the placeholder ID with the governed engagement ID. ### 3. Implement the admitted source adapter [#3-implement-the-admitted-source-adapter] > No source-specific executable compiler exists in this release. Continue with assessment, governed work, and an admitted adapter; Airlift does not invent executable output. Expected artifacts: * Capability plan and adapter requirements only Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. > This source is cataloged or assessable but has no source-specific executable compiler in the installed Airlift generation. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file athena-migration-plan.json --idempotency-key athena-plan-v1 fa conversion batch create --file athena-batch.json --idempotency-key athena-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file athena-transfer.json --idempotency-key athena-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file athena-validation.json --idempotency-key athena-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file athena-evidence-export.json --idempotency-key athena-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to Amazon Athena; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # BigQuery to Databricks # BigQuery to Databricks [#bigquery-to-databricks] The BigQuery pack inventories datasets, tables, nested schemas, views, routines, scheduled queries, BigQuery ML, remote functions, authorized views, and downstream BI. It combines specialist conversion and transfer tools with Airlift's governed ledger and independent evidence model. ```bash fa source inspect bigquery fa source plan bigquery fa source plan bigquery --json > .airlift/bigquery-plan.json ``` ## Migration route [#migration-route] | Stage | Implementation | Airlift value | | --------- | ----------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------- | | Assess | Export metadata and source artifacts; run the admitted analyzer route | Accepted inventory, dependencies, exclusions, immutable report refs | | Convert | Translate supported GoogleSQL and disposition scripting, ML, remote functions, and authorized-view behavior | Versioned attempts, artifact digests, warnings, residue ownership | | Transfer | Use restartable storage exports or an admitted connector with timestamp/change catch-up | Manifests, watermarks, restart, lag, reconciliation | | Validate | Test nested/repeated fields, numeric and timestamp semantics, partitions, access, and business queries | Independent readiness evidence | | Cut over | Freeze the wave and change consumers with checkpoint/apply-once/verify | Approvals and verifiable outcome or rollback | | Modernize | Adopt Delta, liquid clustering, Lakeflow, Unity Catalog, and serverless SQL | Separate, independently validated release | The pack is `assessable`: it describes an admitted assessment route, but that does not certify your transfer, validation, scale, or cutover. Produce a live manifest and run: ```bash fa source certification-check bigquery-certification.json ``` {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe bigquery fa source recipe bigquery --variant bigquery --json > .airlift/bigquery-recipe.json ``` The App is engagement-aware. Google BigQuery appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect bigquery --json > .airlift/bigquery-profile.json fa source plan bigquery --variant bigquery --json > .airlift/bigquery-capability-plan.json ``` Expected artifacts: * .airlift/bigquery-profile.json * .airlift/bigquery-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file bigquery-estate.json --idempotency-key bigquery-estate-v1 fa connection register --file bigquery-connection.json --idempotency-key bigquery-connection-v1 fa engagement update --file bigquery-scope.json --idempotency-key bigquery-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file bigquery-assessment-start.json --idempotency-key bigquery-assessment-start-v1 fa assessment status --json fa assessment record --file bigquery-assessment-record.json --idempotency-key bigquery-assessment-record-v1 fa assessment accept --file bigquery-assessment-accept.json --idempotency-key bigquery-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/bigquery` after replacing the placeholder ID with the governed engagement ID. ### 3. Implement the admitted source adapter [#3-implement-the-admitted-source-adapter] > No source-specific executable compiler exists in this release. Continue with assessment, governed work, and an admitted adapter; Airlift does not invent executable output. Expected artifacts: * Capability plan and adapter requirements only Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. > This source is cataloged or assessable but has no source-specific executable compiler in the installed Airlift generation. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file bigquery-migration-plan.json --idempotency-key bigquery-plan-v1 fa conversion batch create --file bigquery-batch.json --idempotency-key bigquery-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file bigquery-transfer.json --idempotency-key bigquery-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file bigquery-validation.json --idempotency-key bigquery-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file bigquery-evidence-export.json --idempotency-key bigquery-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to Google BigQuery; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # Capability registry # Capability registry [#capability-registry] A source pack tells your code which migration workflow to assemble. The capability registry tells operators what the installed provider generation has actually proved. Use both: * the source profile is deterministic developer guidance; * the capability matrix is tenant-scoped governed evidence; * object readiness and migration certificates prove a specific client workload; * cutover policy decides whether a wave can move. These layers intentionally do not imply each other. ## Matrix dimensions [#matrix-dimensions] Every registered source variant can carry independent entries for: | Capability | What the entry describes | | ---------------------------- | ----------------------------------------------------------------------- | | `assessment` | metadata collection, profiling, complexity, and inventory normalization | | `dependency_lineage` | object, pipeline, scheduler, and consumer dependencies | | `sql_conversion` | deterministic SQL conversion route and its boundaries | | `procedural_conversion` | procedures, functions, packages, macros, and scripts | | `etl_conversion` | orchestration and transformation pipeline transition | | `notebook_script_conversion` | notebooks, shell/Python scripts, and job definitions | | `bi_semantic_transition` | semantic models, reports, consumers, and query contracts | | `data_snapshot` | restartable baseline movement and manifests | | `incremental_cdc` | change capture, watermarks, lag, and catch-up | | `validation_reconciliation` | schema, data, query, and non-functional evidence | | `target_generation` | Unity Catalog, Delta, SQL, jobs, and pipeline artifacts | | `deployment_promotion` | deployment requirement and Runway release references | | `cutover_rollback` | rehearsed endpoint change, verify, rollback, and recovery | | `modernization` | separately measured Databricks-native improvement | Delivery mode is `automated`, `mixed`, `human`, `external`, `not_applicable`, or `unavailable`. A human or mixed entry must name the skills required. This makes specialist remediation a designed lane that can be assigned and estimated, not an exception hidden inside an automation percentage. ## Proof ladder [#proof-ladder] | Level | Required evidence | | ---------------------- | -------------------------------------------------------------------- | | `contract_only` | schema and contract test for the provider seam | | `hermetic_proven` | repeatable end-to-end run against a versioned fixture | | `workspace_proven` | immutable Databricks workspace run and build reference | | `client_proven` | representative client dataset run under the accepted profile | | `production_certified` | current production certification evidence for the bounded capability | An entry stores the provider version, run and build references, SHA-256 digest, optional dataset reference, completion time, and validity window. Airlift does not copy provider artifacts into its event log. ## Register and prove a capability [#register-and-prove-a-capability] Registry v1 entries describe a whole source variant capability. Registry v2 adds a construct-scoped cell without changing v1 replay. For example, add this block to a v2 proposal: ```json { "schemaVersion": 2, "constructScope": { "construct": "merge_statement", "artifactKind": "source", "targetPattern": "Databricks SQL MERGE INTO", "automationDisposition": "deterministic", "sourceVersion": "Synapse SQL 2025", "documentationReference": "/docs/sources/synapse" } } ``` The disposition is one of `deterministic`, `agent_repairable`, `human_remediation`, `retain_or_federate`, `replace_native`, `retire`, `excluded`, or `unsupported`. Construct proof is deliberately non-aggregating: it cannot raise the whole variant's support level or clear engagement preflight by itself. Create `capability-proposal.json`: ```json { "schemaVersion": 1, "sourceSystem": "synapse", "sourceVariant": "synapse_dedicated_sql", "capability": "assessment", "availability": "automated", "providerId": "lakebridge.analyzer", "providerVersion": "0.7.0", "targetProofLevel": "workspace_proven", "limitations": ["ADF orchestration requires a separate export"], "requiredHumanSkills": [], "validationProfileIds": ["synapse-assessment-v1"], "expiresAt": "2031-08-03T00:00:00Z", "reason": "Register the provider boundary for independent review." } ``` ```bash fa capability propose \ --file capability-proposal.json \ --idempotency-key synapse-assessment-proposal-v1 ``` The result is `proposed`; it is not active. An admitted automation principal records an immutable test result: ```json { "capabilityEntryId": "cap_", "evidence": { "evidenceId": "cev_synapse_assessment_workspace_01", "kind": "workspace_run", "ref": "artifact-store://airlift/evidence/synapse-assessment.json", "digest": "<64 lowercase hex characters>", "buildRef": "ci://fabric-airlift/build/123", "datasetRef": "dataset://synapse-representative/v1", "runRef": "databricks://jobs/assessment/runs/456", "providerVersion": "0.7.0", "completedAt": "2030-08-03T18:00:00Z", "validUntil": "2031-08-03T00:00:00Z" } } ``` ```bash fa capability evidence \ --file capability-evidence.json \ --idempotency-key synapse-assessment-workspace-proof-v1 ``` A different human reviewer can then promote no higher than the strongest admitted evidence: ```bash fa capability promote \ --file capability-promotion.json \ --idempotency-key synapse-assessment-promotion-v1 ``` Use `expire`, `revoke`, and `reconcile` for time expiry, an explicit human withdrawal, and provider-version observation respectively. History remains in the event and audit ledger. ## Read the installed matrix [#read-the-installed-matrix] ```bash fa capability list --source synapse --variant synapse_dedicated_sql fa capability list --source synapse --construct merge_statement --artifact-kind source fa capability matrix --source synapse --variant synapse_dedicated_sql fa capability matrix --source synapse --variant synapse_dedicated_sql --json fa source doctor synapse --variant synapse_dedicated_sql fa source limitations synapse --variant synapse_dedicated_sql fa source certify synapse --variant synapse_dedicated_sql --level certifiable fa engagement preflight eng_ ``` The matrix prints `implementationRoutingLevel`, `derivedSupportLevel`, lifecycle cells, and the independent required-construct denominator. Build automation and deployment gate only against the evidence-derived level or exact cells. Implementation routing selects an intended workflow; it never represents achieved proof. `source doctor` explains every missing, proposed, expired, unavailable, or under-proved requirement and prints the next action. `source limitations` lists the currently governed constraints and specialist skills. `source certify` is a fail-closed capability-readiness check; despite its concise name, it does not mint a migration certificate or activate a provider claim. `engagement preflight` derives its target from the engagement services, resolves every scoped estate to its source variant, and evaluates the same requirements. It returns a non-zero exit code while any estate is blocked. Human and mixed delivery modes can satisfy a requirement when they have sufficient evidence, but remain visible as staffed lanes. In TypeScript: ```ts import { buildSourceCapabilityMatrixRow, resolveSourceSystemProfile, type SourceCapabilityRow, } from '@fabricorg/airlift'; const profile = resolveSourceSystemProfile('synapse'); const entries: SourceCapabilityRow[] = await loadTenantCapabilityRows(); const matrix = buildSourceCapabilityMatrixRow( profile, 'synapse_dedicated_sql', entries, ); ``` The [generated capability catalog](/docs/reference/capability-catalog) lists the identifiers exported by the installed SDK. It is regenerated during every docs build so source variants, delivery modes, proof levels, and CLI documentation cannot silently drift from code. # Certify a source profile # Certify a source profile [#certify-a-source-profile] The typed profile says which implementation path Airlift supports. The capability registry separately proves each provider route for a source variant. Client certification then proves that path against a particular source version, workload corpus, Databricks target, and acceptance policy. Do not update a profile's implementation routing metadata as a substitute for registry evidence. Use [the capability registry](/docs/sources/capability-registry) to register the provider generation, delivery mode, limitations, proof, and expiry. ## Developer acceptance contract [#developer-acceptance-contract] | Layer | Implement and retain | Failure behavior | | -------------- | --------------------------------------------------------------------------------------------------- | ---------------------------------------- | | Assessment | profiler/analyzer run, source version, accepted inventory, dependencies, exclusions, report digests | estate remains unassessed | | Conversion | representative corpus, pinned transpiler, artifact digests, warnings/errors, residue taxonomy | object moves to rework or human lane | | Transfer | snapshot, incremental catch-up, watermark, restart, lag, counts, cleanup | data readiness remains pending or failed | | Validation | schema, row/data, business query and object-type scenarios from an admitted producer | object cannot be certified | | Non-functional | workload, threshold, measured result, security checks, observation window | non-functional track remains open | | Cutover | endpoint checkpoint, apply-once, verify, rollback/compensation rehearsal | automated cutover remains disabled | ## Add a new source pack [#add-a-new-source-pack] 1. Add the ID to `sourceSystemSchema` in `packages/airlift/src/schemas.ts`. 2. Add a complete `SourceSystemProfile` with archetype, variants, adapter contract, and an honest initial support level. 3. Map applicable analyzer, profiler, converter, reconcile, managed-ingestion, API, pipeline, query-engine, or streaming boundaries; leave unavailable routes explicit. 4. Populate workload surfaces, transfer strategy, validation checks, residue, modernization targets, and credential boundary. 5. Add aliases only when they resolve unambiguously. 6. Extend `packages/airlift/test/sources.test.ts`; every generated action ID must exist in the installed governed action catalog. 7. Add a developer guide under `apps/docs/content/docs/sources/`. 8. Add the pack to `source-packs-e2e.test.ts` through the canonical registry and run the hermetic suite. 9. Run representative live routes and check their immutable certification manifest before changing a support level. Use these commands during development: ```bash fa sources --json fa source inspect --json fa source plan --variant --json fa source certification-check source-certification.json fa capability matrix --source --variant pnpm test:e2e:source pnpm test:e2e:hermetic ``` The generated plan is deterministic and testable. Client acceptance remains governed state and must enter through Platform actions. # Connection diagnostics # Connection diagnostics [#connection-diagnostics] Connection bindings start as configuration. A binding is **usable for assessment** only when Airlift has an admitted, fresh, capability-covering connectivity diagnostic for it. Configuration, environment variables, lifecycle status (`pending`/`verified`), screenshots, and caller-declared metadata never raise connectivity state. ## One shared observed-state vocabulary [#one-shared-observed-state-vocabulary] Connectivity renders through the same five-token observed-state model as the Databricks target integrations: | State | Meaning | | ---------------- | ---------------------------------------------------------------------------------------- | | `not_configured` | The binding was retired; it is no longer a configured surface. | | `not_observed` | No diagnostic recorded for the current binding revision. | | `stale` | Every recorded diagnostic predates the current revision or the 24-hour freshness window. | | `blocked` | The newest matching diagnostic contains a denied or failed required probe. | | `observed` | The newest matching diagnostic passed every required probe inside the freshness window. | Only `observed` marks a binding usable for assessment. The freshness window is a module constant (`CONNECTION_DIAGNOSTIC_FRESHNESS_HOURS = 24`), never organization-configurable: widening it by configuration would raise health by configuration, which the observed-state law forbids. ## Probes and capability coverage [#probes-and-capability-coverage] Required coverage derives from the binding's own declared capabilities, never from what a submitter chose to run. Every diagnostic requires the three connectivity probes — `endpoint_reachable`, `authentication`, `tls_verified` — plus one permission probe per declared capability: | Probe | Class | Evidences | | -------------------------- | ------------ | -------------------- | | `endpoint_reachable` | connectivity | every binding | | `authentication` | connectivity | every binding | | `tls_verified` | connectivity | every binding | | `catalog_list` | permission | `inventory_read` | | `object_metadata_read` | permission | `metadata_read` | | `ddl_definition_read` | permission | `metadata_read` | | `row_sample_read` | permission | `snapshot_read` | | `change_feed_read` | permission | `change_feed_read` | | `target_namespace_write` | permission | `target_write` | | `artifact_write` | permission | `artifact_write` | | `target_deploy_probe` | permission | `target_deploy` | | `validation_execute_probe` | permission | `validation_execute` | | `cutover_effect_probe` | permission | `cutover_effect` | A diagnostic that omits a required probe, marks a required probe `not_applicable`, or submits a probe for an undeclared capability is rejected. Coverage is conjunctive, never best-effort. ## Recording a diagnostic [#recording-a-diagnostic] Diagnostics enter Airlift only through `airlift.connection_diagnostic_record`, invoked by an **admitted system principal**. The admission policy fails closed when: * the actor is not a system principal; * no `airliftConnectionDiagnosticVerifier` service is configured; * the verifier cannot confirm the immutable evidence reference and digest. The handler derives `diagnosticDigest` and `outcome` — a caller can never supply either — and checks the `bindingRevisionDigest` against the binding's current configuration, so a diagnostic recorded against a superseded credential or capability set can never verify. Probe output is secret-safe: any field containing a credential reference or userinfo is rejected. ## Verifying a binding [#verifying-a-binding] `fa connection verify` requires the digest of a recorded, revision-matched, fully passing diagnostic: ```bash fa connection test # observed connectivity report; exit 0 iff observed fa connection diagnose \ --file probes.json \ --idempotency-key # admitted system principal only fa connection verify \ --digest \ --idempotency-key ``` A caller-chosen digest is not evidence: verification rejects any digest that is not the recorded diagnostic digest, and it rejects when the freshest diagnostic is blocked or stale. ## Blockers [#blockers] Every non-observed state renders a five-part blocker in the App and `fa connection test`: what happened, why it matters, who acts, the required evidence, and one primary action. The engagement status `discover` phase surfaces the same underlying condition while no source binding is observed — it carries the what/why summary and the primary action, and the App renders the full five-part detail. ## Honest support boundary [#honest-support-boundary] Fixtures and development self-review never admit connectivity. `observed` requires an admitted probe run against the real source. The development-assurance journey injects a permissive development verifier behind the same explicit escape hatch that already governs development self-review, and the resulting state never authorizes production cutover. **Production deployments currently fail closed.** No production runtime wires an admitted `airliftConnectionDiagnosticVerifier` yet: until a source-specific probe provider is implemented and admitted, governed deployments cannot record connectivity diagnostics, and every binding stays `not_observed`. That is the deliberate, honest default — a permissive production verifier would admit arbitrary evidence. The probe-provider seam is named in the A0 remaining exit and tracked as a dated staged-compliance exception. # Databricks-native modernization # Databricks-native modernization [#databricks-native-modernization] Airlift is also useful after a customer has adopted Databricks. A native engagement records the existing estate as the baseline and governs a separate modernization release with independent evidence, immutable promotion and rollback, and measured value. ```bash fa source inspect databricks fa source plan databricks --variant databricks_unity_catalog fa databricks-native inspect --file native-manifest.json fa databricks-native plan --file native-manifest.json --json > native-plan.json ``` ## What Airlift inventories [#what-airlift-inventories] * workspaces, metastores, Hive Metastore objects, and Unity Catalog assets; * jobs, pipelines, notebooks, warehouses, clusters, and policies; * permissions, shares, lineage, and external paths; * dashboards, models, endpoints, and downstream consumers; * cost, performance, reliability, and operational dependencies. ## What Airlift helps change [#what-airlift-helps-change] * Unity Catalog upgrades and workspace consolidation; * permission, identity, and path modernization; * serverless and Lakeflow adoption; * Delta layout and liquid clustering; * runtime and dependency upgrades; * dashboard and consumer transitions; and * cost and performance optimization. The workflow is baseline → design → implement → validate → release → measure. The baseline remains immutable. Each change gets a typed recommendation and evidence profile. A promoted release needs Runway identity, Experiments verdicts, operational evidence, acceptance, and a rollback path. External productivity claims require a reviewed Airlift value report. See [Databricks-native commands](/docs/cli/databricks-native) for the exact manifest and CLI contract. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe databricks fa source recipe databricks --variant databricks_unity_catalog --json > .airlift/databricks-recipe.json ``` The App is engagement-aware. Databricks native estate appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect databricks --json > .airlift/databricks-profile.json fa source plan databricks --variant databricks_unity_catalog --json > .airlift/databricks-capability-plan.json ``` Expected artifacts: * .airlift/databricks-profile.json * .airlift/databricks-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file databricks-estate.json --idempotency-key databricks-estate-v1 fa connection register --file databricks-connection.json --idempotency-key databricks-connection-v1 fa engagement update --file databricks-scope.json --idempotency-key databricks-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file databricks-assessment-start.json --idempotency-key databricks-assessment-start-v1 fa assessment status --json fa assessment record --file databricks-assessment-record.json --idempotency-key databricks-assessment-record-v1 fa assessment accept --file databricks-assessment-accept.json --idempotency-key databricks-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/databricks` after replacing the placeholder ID with the governed engagement ID. ### 3. Compile the Databricks-native modernization plan [#3-compile-the-databricks-native-modernization-plan] ```bash fa databricks-native inspect --file databricks-manifest.json fa databricks-native plan --file databricks-manifest.json --json > generated/databricks-plan.json ``` Expected artifacts: * Unity Catalog modernization plan * Operational application candidates * Measurement requirements Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file databricks-migration-plan.json --idempotency-key databricks-plan-v1 fa conversion batch create --file databricks-batch.json --idempotency-key databricks-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file databricks-transfer.json --idempotency-key databricks-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file databricks-validation.json --idempotency-key databricks-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file databricks-evidence-export.json --idempotency-key databricks-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to Databricks native estate; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # Db2 and mainframe data to Databricks # Db2 and mainframe data to Databricks [#db2-and-mainframe-data-to-databricks] ```bash fa source inspect db2 fa source plan db2 --variant db2_luw fa source plan db2 --variant db2_zos --json ``` The mainframe archetype keeps platform differences explicit. Variants are Db2 LUW, Db2 for z/OS, and Db2 for IBM i. Inventory tables, packages, SQL PL, utilities, load jobs, CDC or log capture, JCL and batch dependencies, encodings, and consumers. Airlift records a consistent unload or replication boundary, commit sequence, restart checkpoint, batch window, file manifest, and reconciliation evidence from the selected adapter. It routes z/OS utilities, JCL coupling, SQL PL packages, legacy encodings, and cross-platform transaction behavior into owned residue instead of hiding them in a SQL conversion percentage. Validation profiles should include EBCDIC and code-page behavior, decimal and date semantics, package behavior, commit ordering, batch totals, restart, and representative window performance. Certification and cutover use the same signed evidence, separation of duties, frozen wave, and rollback model as other Airlift sources. The public pack is `cataloged`; live platform access and certified extraction/cutover effectors are external evidence gates. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe db2 fa source recipe db2 --variant db2_luw --json > .airlift/db2-recipe.json ``` The App is engagement-aware. IBM Db2 and mainframe data appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect db2 --json > .airlift/db2-profile.json fa source plan db2 --variant db2_luw --json > .airlift/db2-capability-plan.json ``` Expected artifacts: * .airlift/db2-profile.json * .airlift/db2-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file db2-estate.json --idempotency-key db2-estate-v1 fa connection register --file db2-connection.json --idempotency-key db2-connection-v1 fa engagement update --file db2-scope.json --idempotency-key db2-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file db2-assessment-start.json --idempotency-key db2-assessment-start-v1 fa assessment status --json fa assessment record --file db2-assessment-record.json --idempotency-key db2-assessment-record-v1 fa assessment accept --file db2-assessment-accept.json --idempotency-key db2-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/db2` after replacing the placeholder ID with the governed engagement ID. ### 3. Implement the admitted source adapter [#3-implement-the-admitted-source-adapter] > No source-specific executable compiler exists in this release. Continue with assessment, governed work, and an admitted adapter; Airlift does not invent executable output. Expected artifacts: * Capability plan and adapter requirements only Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. > This source is cataloged or assessable but has no source-specific executable compiler in the installed Airlift generation. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file db2-migration-plan.json --idempotency-key db2-plan-v1 fa conversion batch create --file db2-batch.json --idempotency-key db2-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file db2-transfer.json --idempotency-key db2-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file db2-validation.json --idempotency-key db2-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file db2-evidence-export.json --idempotency-key db2-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to IBM Db2 and mainframe data; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # Build a migration project # Build a migration project [#build-a-migration-project] Use this workflow in a migration repository. Airlift supplies the source contract and assurance lifecycle; your integration supplies authenticated source, Databricks, transfer, validation, and client-effect adapters. ## 1. Pin the profile and plan [#1-pin-the-profile-and-plan] ```bash mkdir -p .airlift fa source inspect dynamics_365 --json > .airlift/source-profile.json fa source plan dynamics_365 \ --variant dynamics_365_finance_operations \ --json > .airlift/source-plan.json fa actions --json > .airlift/actions.json fa profiles --json > .airlift/validation-profiles.json ``` Commit these non-secret artifacts. They make source-pack and policy upgrades reviewable. Do not commit tokens, passwords, keys, or connection strings. ## 2. Implement the specialist boundaries [#2-implement-the-specialist-boundaries] Read `profile.adapter` and `plan.steps[].specialistCommands`. The plan names the required boundary; it does not invent a connector command. ```ts import { resolveSourceSystemProfile } from '@fabricorg/airlift'; const profile = resolveSourceSystemProfile('d365'); export const sourceAdapter = { async inventory(connectionRef: string) { // Resolve the opaque ref inside the authenticated connector boundary. // Return artifact refs, digests, source version, entities, and dependencies. }, async transfer(connectionRef: string, checkpoint?: string) { // Return snapshot, cursor/watermark, deletion, restart, lag, and count evidence. }, }; console.log(profile.adapter.inventoryInputs); console.log(profile.adapter.validationInputs); ``` For SQL sources, generated steps contain applicable Lakebridge commands. For ERP and SaaS systems, use the admitted managed connector or API/export adapter. For ETL systems, export pipeline definitions. For query engines, route each backing catalog. For streams, retain offset or sequence boundaries and checkpoint evidence. ## 3. Create governed estate state [#3-create-governed-estate-state] Actor and tenant come from the authenticated application boundary, never a CLI flag or request payload. ```ts import { AIRLIFT_ACTION_IDS } from '@fabricorg/airlift'; const estate = await runtime.invokeAction(AIRLIFT_ACTION_IDS.estateRegister, { ...authenticatedContext, idempotencyKey: 'finance-source-estate-v1', params: { name: 'Finance source', sourceSystem: 'dynamics_365', sourceVariant: 'dynamics_365_finance_operations', environment: 'prod', owner: 'finance-data', connectionRef: 'uc-connection://finance-source', }, }); ``` Start and record the assessment, then register normalized entities, tables, pipelines, reports, or other in-scope artifacts with `objectRegister`. Reuse stable idempotency keys for logical retries. ## 4. Produce target artifacts and residue [#4-produce-target-artifacts-and-residue] Use Lakebridge for supported SQL conversion, a typed target mapper for applications, a pipeline implementation adapter for ETL, or a stream bootstrap adapter for event systems. Record each attempt through `conversionStart` and `conversionRecord`. Unsupported constructs stay in deterministic, bounded-repair, or human-owned lanes. An artifact is not parity evidence. ## 5. Move and validate independently [#5-move-and-validate-independently] Movement returns source boundary, target snapshot, restart, lag, and reconciliation evidence. Validation runs separately through an admitted provider and tests the assigned object-type profile: schema or contract, data, business behavior, security, consumers, and non-functional thresholds. Record the immutable provider execution with `validationRunRecord`; link it to each requirement with `readinessRecord`. `businessAccept` is a separate governed decision by an admitted natural person. ## 6. Mint, freeze, and cut over [#6-mint-freeze-and-cut-over] Only the admitted system principal invokes `migrationCertificateMint`. Plan a wave, assign certified objects, freeze its evidence, collect authenticated approvals, and run automated cutover only through a client-certified checkpoint/apply-once/verify/rollback effector. ```bash fa certificate verify migration-certificate.json \ --keys trusted-public-keys.json ``` ## 7. Add scheduled source-pack regression [#7-add-scheduled-source-pack-regression] Hermetic source contracts run without credentials: ```bash pnpm test:e2e:source pnpm test:e2e:hermetic ``` Live adapter jobs should publish a source-certification manifest and run: ```bash AIRLIFT_LIVE_SOURCE_MANIFEST=source-certification.json pnpm test:e2e:live ``` Missing inventory, transfer, validation, scale, or cutover-effector evidence fails closed for the requested support level. # Dynamics 365 to Databricks # Dynamics 365 to Databricks [#dynamics-365-to-databricks] A Dynamics estate includes entity metadata, relationships, lookups, choices, change tracking, deletions, Power Platform dependencies, and business measures. Airlift scopes and certifies that connected estate while an admitted Dynamics 365 connector performs the actual extraction. ## Generate the route [#generate-the-route] ```bash fa source inspect d365 --json fa source plan dynamics_365 --variant dynamics_365_dataverse fa source plan dynamics_365 \ --variant dynamics_365_finance_operations \ --json > .airlift/d365-finance-plan.json fa application-pack plan --file dataverse-manifest.json --json > dataverse-plan.json ``` Variants are `dynamics_365_dataverse`, `dynamics_365_finance_operations`, and `dynamics_365_sales_service`. ## What Airlift adds [#what-airlift-adds] 1. Accepts an immutable entity, field, relationship, and consumer inventory. 2. Records target mappings and keeps plugins, virtual tables, polymorphic lookups, automations, and computed entities in explicit residue lanes. 3. Tracks snapshot, incremental cursor, deletion behavior, restart, and freshness from the connector run. 4. Requires independent key, relationship, choice, history, and business-measure validation rather than accepting connector completion as proof. 5. Mints signed certificates from admitted evidence and freezes the exact wave before consumer cutover. 6. Opens Lakeflow Connect, Unity Catalog, Delta, Databricks SQL, and AI/BI improvements as a separate modernization release. ```ts import { createSourceMigrationPlan } from '@fabricorg/airlift'; const plan = createSourceMigrationPlan('d365', { variant: 'dynamics_365_finance_operations', }); for (const step of plan.steps) { console.log(step.id, step.airliftActions, step.exitCriteria); } ``` Supply OAuth or service-principal credentials through a Databricks connection or secret scope. Airlift stores the opaque `connectionRef`, never the token. The Dataverse business-semantic compiler is executable; Finance and Operations and Sales/Customer Service remain cataloged profiles until their source-specific compilers ship. Use live evidence before claiming a higher support level. See [enterprise application-pack commands](/docs/cli/application-packs) for the credential-free manifest, semantic validation, and registration contracts. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe dynamics_365 fa source recipe dynamics_365 --variant dynamics_365_dataverse --json > .airlift/dynamics_365-recipe.json ``` The App is engagement-aware. Microsoft Dynamics 365 appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect dynamics_365 --json > .airlift/dynamics_365-profile.json fa source plan dynamics_365 --variant dynamics_365_dataverse --json > .airlift/dynamics_365-capability-plan.json ``` Expected artifacts: * .airlift/dynamics\_365-profile.json * .airlift/dynamics\_365-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file dynamics_365-estate.json --idempotency-key dynamics_365-estate-v1 fa connection register --file dynamics_365-connection.json --idempotency-key dynamics_365-connection-v1 fa engagement update --file dynamics_365-scope.json --idempotency-key dynamics_365-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file dynamics_365-assessment-start.json --idempotency-key dynamics_365-assessment-start-v1 fa assessment status --json fa assessment record --file dynamics_365-assessment-record.json --idempotency-key dynamics_365-assessment-record-v1 fa assessment accept --file dynamics_365-assessment-accept.json --idempotency-key dynamics_365-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/dynamics_365` after replacing the placeholder ID with the governed engagement ID. ### 3. Compile the enterprise application pack [#3-compile-the-enterprise-application-pack] ```bash fa application-pack inspect --file dynamics_365-manifest.json fa application-pack plan --file dynamics_365-manifest.json --json > generated/dynamics_365-plan.json ``` Expected artifacts: * Business-semantic target plan * Control totals * Security mappings * Consumer transitions Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file dynamics_365-migration-plan.json --idempotency-key dynamics_365-plan-v1 fa conversion batch create --file dynamics_365-batch.json --idempotency-key dynamics_365-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file dynamics_365-transfer.json --idempotency-key dynamics_365-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file dynamics_365-validation.json --idempotency-key dynamics_365-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file dynamics_365-evidence-export.json --idempotency-key dynamics_365-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to Microsoft Dynamics 365; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # ETL modernization # ETL modernization [#etl-modernization] This capability treats pipelines and analytics code as first-class migration objects. ADF and Synapse Pipelines have a native export importer and concrete Databricks file generator. SSIS, Informatica PowerCenter, SAS, IBM DataStage, Talend, Oracle Data Integrator, dbt, and Apache Airflow currently have explicit normalized-manifest routing profiles. Those profiles preserve scope and classify work, but do not yet parse native exports or emit deployable file bodies. ```bash fa source inspect etl_modernization fa source plan etl_modernization --variant informatica fa source plan etl_modernization --variant azure_data_factory --json fa migration-ir import --source adf-synapse --file adf-export/ \ --estate-name "Commerce ADF" --snapshot-at 2030-01-15T12:00:00Z \ --output migration-ir.json fa migration-ir generate --file migration-ir.json --out-dir generated fa migration-ir validate --file generated/artifact-set.json --root generated ``` ## What Airlift adds [#what-airlift-adds] | Phase | Developer work | Governed outcome | | ----------------- | --------------------------------------------------------------------------------------- | --------------------------------------------------------------------- | | Discover | Export pipelines, mappings, connections, parameters, triggers, lineage, and run history | Accepted inventory and dependency graph; credentials replaced by refs | | Map and implement | Map source semantics to Lakeflow Jobs, Declarative Pipelines, or Databricks Workflows | Versioned target artifacts and explicit custom-code residue | | Rehearse | Run production-like parameters, volumes, dependencies, retries, and checkpoints | Immutable run and reconciliation evidence | | Validate | Prove row/business results, schedules, failure behavior, and SLA thresholds | Independent readiness observations | | Cut over | Freeze schedule scope and switch orchestration through a certified effector | Authenticated approvals, checkpoint, verification or rollback | | Modernize | Optimize serverless compute, lineage, and pipeline design | Separate release and evidence profile | Connection identities are mapped to governed target references. Airlift never imports source passwords from exported pipeline files. Custom components, proprietary connectors, script tasks, implicit variables, and scheduler coupling remain visible in the residue ledger. The ADF native path has recurring credential-free import, generation, tamper-denial, and materialization coverage. The other eight routing profiles have deterministic fixture coverage for provenance, dependency ordering, dispositions, and no-silent-drop behavior. Neither test level substitutes for live workspace or representative client evidence. See [Migration IR commands](/docs/cli/migration-ir) for the manifest, routing, and registration contracts and [Pipeline and code modernization](/docs/migration/pipeline-modernization) for the complete developer workflow. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe etl_modernization fa source recipe etl_modernization --variant informatica --json > .airlift/etl_modernization-recipe.json ``` The App is engagement-aware. ETL and orchestration platforms appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect etl_modernization --json > .airlift/etl_modernization-profile.json fa source plan etl_modernization --variant informatica --json > .airlift/etl_modernization-capability-plan.json ``` Expected artifacts: * .airlift/etl\_modernization-profile.json * .airlift/etl\_modernization-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file etl_modernization-estate.json --idempotency-key etl_modernization-estate-v1 fa connection register --file etl_modernization-connection.json --idempotency-key etl_modernization-connection-v1 fa engagement update --file etl_modernization-scope.json --idempotency-key etl_modernization-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file etl_modernization-assessment-start.json --idempotency-key etl_modernization-assessment-start-v1 fa assessment status --json fa assessment record --file etl_modernization-assessment-record.json --idempotency-key etl_modernization-assessment-record-v1 fa assessment accept --file etl_modernization-assessment-accept.json --idempotency-key etl_modernization-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/etl_modernization` after replacing the placeholder ID with the governed engagement ID. ### 3. Compile the normalized informatica pipeline manifest [#3-compile-the-normalized-informatica-pipeline-manifest] ```bash fa migration-ir inspect --file informatica-migration-ir.json fa migration-ir compile --file informatica-migration-ir.json --json > generated/informatica-routing-plan.json ``` Expected artifacts: * Validated normalized migration IR * Deterministic routing plan * Source-preserved residue Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. > Only ADF/Synapse currently has a native export importer and executable file generator. Other ETL variants require an adapter-produced Migration IR manifest and remain descriptor-only. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file etl_modernization-migration-plan.json --idempotency-key etl_modernization-plan-v1 fa conversion batch create --file etl_modernization-batch.json --idempotency-key etl_modernization-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file etl_modernization-transfer.json --idempotency-key etl_modernization-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file etl_modernization-validation.json --idempotency-key etl_modernization-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file etl_modernization-evidence-export.json --idempotency-key etl_modernization-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to ETL and orchestration platforms; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # Azure Event Hubs to Databricks # Azure Event Hubs to Databricks [#azure-event-hubs-to-databricks] ```bash fa source inspect event_hubs fa source plan event_hubs --variant azure_event_hubs ``` Inventory event hubs, partitions, consumer groups, schemas, Capture destinations, retention, access policies, producers, and consumers. The movement adapter uses checkpointed reads or Capture-file ingestion and returns starting offsets or sequence numbers, enqueue-time boundaries, checkpoints, restarts, lag, and throughput. Airlift requires independent loss, duplicate, schema compatibility, partition ordering, business aggregate, and performance evidence. Protocol-specific producers, capture format assumptions, shared-access policy design, checkpoint coupling, and cross-hub ordering remain visible as residue. Once evidence passes, freeze the producer and consumer scope, collect authenticated approvals, switch through a certified checkpoint/apply-once/verify effector, and retain the rollback result. Structured Streaming, Lakeflow, Delta event tables, Unity Catalog, and streaming tables are separate modernization targets. The pack is `cataloged`; a configured Event Hubs connector is not by itself a certified migration route. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe event_hubs fa source recipe event_hubs --variant azure_event_hubs --json > .airlift/event_hubs-recipe.json ``` The App is engagement-aware. Azure Event Hubs appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect event_hubs --json > .airlift/event_hubs-profile.json fa source plan event_hubs --variant azure_event_hubs --json > .airlift/event_hubs-capability-plan.json ``` Expected artifacts: * .airlift/event\_hubs-profile.json * .airlift/event\_hubs-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file event_hubs-estate.json --idempotency-key event_hubs-estate-v1 fa connection register --file event_hubs-connection.json --idempotency-key event_hubs-connection-v1 fa engagement update --file event_hubs-scope.json --idempotency-key event_hubs-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file event_hubs-assessment-start.json --idempotency-key event_hubs-assessment-start-v1 fa assessment status --json fa assessment record --file event_hubs-assessment-record.json --idempotency-key event_hubs-assessment-record-v1 fa assessment accept --file event_hubs-assessment-accept.json --idempotency-key event_hubs-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/event_hubs` after replacing the placeholder ID with the governed engagement ID. ### 3. Implement the admitted source adapter [#3-implement-the-admitted-source-adapter] > No source-specific executable compiler exists in this release. Continue with assessment, governed work, and an admitted adapter; Airlift does not invent executable output. Expected artifacts: * Capability plan and adapter requirements only Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. > This source is cataloged or assessable but has no source-specific executable compiler in the installed Airlift generation. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file event_hubs-migration-plan.json --idempotency-key event_hubs-plan-v1 fa conversion batch create --file event_hubs-batch.json --idempotency-key event_hubs-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file event_hubs-transfer.json --idempotency-key event_hubs-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file event_hubs-validation.json --idempotency-key event_hubs-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file event_hubs-evidence-export.json --idempotency-key event_hubs-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to Azure Event Hubs; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # Greenplum to Databricks # Greenplum to Databricks [#greenplum-to-databricks] ```bash fa source inspect greenplum fa source plan greenplum --json > .airlift/greenplum-plan.json ``` Inventory SQL objects, PL functions, distribution policies, external tables, load utilities, resource controls, and consumers. Use a specialist conversion adapter for supported PostgreSQL-family SQL and keep PL extensions, `gpfdist`, resource queues, distribution clauses, and extension dependencies in explicit residue. The transfer adapter should emit parallel-unload manifests, source watermarks, restart checkpoints, catch-up runs, counts, and reconciliation. Independent validation covers distribution-sensitive results, arrays and JSON, PL behavior, external formats, business queries, and representative concurrency. Airlift adds versioned attempts and artifact digests, dependency-aware waves, signed certificates, authenticated approvals, and checkpoint/apply-once/verify/rollback cutover. Delta tables, liquid clustering, Lakeflow, Unity Catalog, and Databricks SQL belong in a separate modernization release. The pack is `cataloged` pending live adapter evidence. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe greenplum fa source recipe greenplum --variant greenplum --json > .airlift/greenplum-recipe.json ``` The App is engagement-aware. Greenplum appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect greenplum --json > .airlift/greenplum-profile.json fa source plan greenplum --variant greenplum --json > .airlift/greenplum-capability-plan.json ``` Expected artifacts: * .airlift/greenplum-profile.json * .airlift/greenplum-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file greenplum-estate.json --idempotency-key greenplum-estate-v1 fa connection register --file greenplum-connection.json --idempotency-key greenplum-connection-v1 fa engagement update --file greenplum-scope.json --idempotency-key greenplum-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file greenplum-assessment-start.json --idempotency-key greenplum-assessment-start-v1 fa assessment status --json fa assessment record --file greenplum-assessment-record.json --idempotency-key greenplum-assessment-record-v1 fa assessment accept --file greenplum-assessment-accept.json --idempotency-key greenplum-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/greenplum` after replacing the placeholder ID with the governed engagement ID. ### 3. Implement the admitted source adapter [#3-implement-the-admitted-source-adapter] > No source-specific executable compiler exists in this release. Continue with assessment, governed work, and an admitted adapter; Airlift does not invent executable output. Expected artifacts: * Capability plan and adapter requirements only Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. > This source is cataloged or assessable but has no source-specific executable compiler in the installed Airlift generation. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file greenplum-migration-plan.json --idempotency-key greenplum-plan-v1 fa conversion batch create --file greenplum-batch.json --idempotency-key greenplum-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file greenplum-transfer.json --idempotency-key greenplum-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file greenplum-validation.json --idempotency-key greenplum-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file greenplum-evidence-export.json --idempotency-key greenplum-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to Greenplum; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # Hadoop # Migrate Hadoop with Airlift [#migrate-hadoop-with-airlift] Hadoop is not one SQL dialect. Airlift models it as an estate containing Hive and Impala SQL, Oozie, Pig, Sqoop, Spark and MapReduce jobs, HDFS or object storage, Hive Metastore, Ranger/Sentry policy, schedules, and downstream consumers. Each construct gets its own conversion and validation disposition. ## What Airlift adds to a Hadoop migration [#what-airlift-adds-to-a-hadoop-migration] | Stage | Databricks tool or project adapter | Airlift responsibility | | --------- | ---------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | | discover | Lakebridge Analyzer runs per Hive, Impala, Oozie, Pig, and Sqoop surface plus filesystem/metastore exports | accept one connected inventory, dependencies, exclusions, source versions, and report digests | | plan | workload disposition and storage analysis inform target architecture | assign owners and dependency-aware waves; preserve copy, register, rewrite, retire, and human-work decisions | | convert | deterministic SQL where supported plus construct-specific rewrites | record source and target artifacts, tool/model/human provenance, warnings, and residue lanes without calling redesign parity | | move data | Unity Catalog registration, copy, Delta conversion, or source-specific incremental drivers | track manifests, watermarks, partition and format behavior, lag, restart checkpoints, counts, and reconciliation | | validate | admitted Experiments scenarios for tables, workflows, jobs, formats, security, and scale | admit independent workload-specific evidence because no universal Hadoop reconcile connector exists | | certify | Airlift evaluates each object's active readiness profile | mint signed certificates identifying artifacts, snapshots, evidence, and policy per workload class | | cut over | project metastore, scheduler, policy, job, data-path, and consumer effectors | freeze scope, enforce approvals, checkpoint, apply once, verify, and retain rollback evidence | | modernize | Unity Catalog, Delta, Lakeflow, supported runtimes, and serverless work | keep architecture redesign in a separately validated release | This is why Airlift treats Hadoop as a workload portfolio rather than a dialect: data, metastore, policy, orchestration, libraries, jobs, and consumers can require different routes but must still cut over as one dependency-aware estate. ## Compile the executable migration pack [#compile-the-executable-migration-pack] Export Hive and Impala definitions, metastore and filesystem inventories, Oozie, Pig and Sqoop projects, Spark and MapReduce jobs, UDF/SerDe metadata, Ranger or Sentry policy, schedules, and consumers. Select `cloudera`, `hortonworks`, or `apache_hadoop`, then run: ```bash fa migration-pack inspect --file hadoop-manifest.json fa migration-pack plan --file hadoop-manifest.json --json > hadoop-plan.json fa migration-pack register --file hadoop-plan.json \ --engagement-id --estate-id \ --artifact-id --idempotency-key ``` The compiler detects incomplete dependencies and cycles, chooses register, convert, bounded repair, or human-remediation routes, and emits a file-manifest/ingestion- watermark transfer contract with workload-specific validation and deployment requirements. See [migration-pack commands](/docs/cli/migration-packs) for the complete manifest schema. ## Inspect and generate a plan [#inspect-and-generate-a-plan] ```bash fa source inspect hadoop fa source plan hive fa source plan hadoop --json > .airlift/hadoop-plan.json ``` The plan intentionally reports that no single deterministic dialect covers the estate. ## Analyze by workload [#analyze-by-workload] Export repository and filesystem metadata, then run Analyzer for each surface in scope: ```bash databricks labs lakebridge analyze \ --source-directory ./source-export/hive \ --source-tech "Hive" \ --report-file ./artifacts/hive-analysis.xlsx \ --generate-json true databricks labs lakebridge analyze \ --source-directory ./source-export/oozie \ --source-tech "Oozie" \ --report-file ./artifacts/oozie-analysis.xlsx \ --generate-json true ``` Repeat for `Cloudera (Impala)`, `PIG`, and `SQOOP`. Register SQL, workflows, transfers, custom jobs, storage assets, policies, and consumers as linked migration objects. ## Route conversion by construct [#route-conversion-by-construct] | Source construct | Target path | | ---------------------------- | --------------------------------------------------------------------------------- | | Hive or Impala SQL | deterministic SQL conversion where supported, otherwise bounded repair/human lane | | Oozie workflows | Lakeflow Jobs or Databricks Workflows design | | Pig | DataFrame, SQL, or declarative pipeline rewrite | | Sqoop | Lakeflow Connect, Auto Loader, federation, or client transfer adapter | | Spark | runtime/library/API upgrade with behavior and scale tests | | MapReduce | Spark, SQL, or purpose-built redesign | | Custom SerDe/InputFormat/UDF | explicit compatibility implementation and test profile | Do not record a modernization rewrite as deterministic parity. Keep its source, target, prompt/model or human provenance, and validation profile explicit. ## Move and validate data [#move-and-validate-data] Classify data by location and format. Some objects can be registered through Unity Catalog; others require copy and Delta conversion. Prove partition semantics, file formats, late data, incremental ingestion, restart, duplicate handling, metastore mapping, access-policy denial, workflow ordering, UDF behavior, and representative scale. Lakebridge Reconcile does not provide a universal Hadoop connector. Use admitted Experiments scenarios and the Airlift transfer evidence contract. Tables cannot pass data readiness from counts alone when partition, format, or late-data behavior matters. ## Certify and cut over [#certify-and-cut-over] Assign validation profiles by workload class rather than certifying the estate from a single table report. Require evidence for metastore mappings, partition and format semantics, workflows, jobs, custom code, policies, performance, and downstream consumers. The project effectors should switch catalogs, schedules, data paths, policies, jobs, and connections through rehearsed waves with explicit rollback or compensation. ## Modernize after parity [#modernize-after-parity] Move governance to Unity Catalog, data to Delta where appropriate, orchestration to Lakeflow, and workloads to supported Databricks runtimes or serverless compute. Retain the baseline certificate and validate the modernization as a new release. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe hadoop fa source recipe hadoop --variant cloudera --json > .airlift/hadoop-recipe.json ``` The App is engagement-aware. Hadoop appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect hadoop --json > .airlift/hadoop-profile.json fa source plan hadoop --variant cloudera --json > .airlift/hadoop-capability-plan.json ``` Expected artifacts: * .airlift/hadoop-profile.json * .airlift/hadoop-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file hadoop-estate.json --idempotency-key hadoop-estate-v1 fa connection register --file hadoop-connection.json --idempotency-key hadoop-connection-v1 fa engagement update --file hadoop-scope.json --idempotency-key hadoop-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file hadoop-assessment-start.json --idempotency-key hadoop-assessment-start-v1 fa assessment status --json fa assessment record --file hadoop-assessment-record.json --idempotency-key hadoop-assessment-record-v1 fa assessment accept --file hadoop-assessment-accept.json --idempotency-key hadoop-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/hadoop` after replacing the placeholder ID with the governed engagement ID. ### 3. Compile the hadoop migration pack [#3-compile-the-hadoop-migration-pack] ```bash fa migration-pack inspect --file hadoop-manifest.json fa migration-pack plan --file hadoop-manifest.json --json > generated/hadoop-plan.json ``` Expected artifacts: * Dependency-aware migration pack * Transfer requirements * Validation requirements * Residue lanes Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file hadoop-migration-plan.json --idempotency-key hadoop-plan-v1 fa conversion batch create --file hadoop-batch.json --idempotency-key hadoop-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file hadoop-transfer.json --idempotency-key hadoop-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file hadoop-validation.json --idempotency-key hadoop-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file hadoop-evidence-export.json --idempotency-key hadoop-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to Hadoop; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # Source systems # Source systems [#source-systems] Airlift source packs turn a source name into a developer contract: variants, workload surfaces, specialist adapter boundaries, governed actions, required evidence, residue, and Databricks modernization targets. The pack never contains credentials and never equates catalog coverage with live certification. ```bash npm install --global @fabricorg/airlift-cli fa sources fa source inspect sap fa source plan sap --variant sap_s4hana fa source plan kafka --variant confluent_cloud --json ``` ## Warehouses and analytical platforms [#warehouses-and-analytical-platforms] | Source | Profile | Public level | Developer guide | | ------------------------ | ------------ | ------------ | -------------------------------------- | | Azure Synapse | `synapse` | executable | [Synapse](/docs/sources/synapse) | | SQL Server and Azure SQL | `sql_server` | executable | [SQL Server](/docs/sources/sql-server) | | Snowflake | `snowflake` | executable | [Snowflake](/docs/sources/snowflake) | | Amazon Redshift | `redshift` | executable | [Redshift](/docs/sources/redshift) | | Oracle | `oracle` | executable | [Oracle](/docs/sources/oracle) | | Teradata | `teradata` | executable | [Teradata](/docs/sources/teradata) | | Netezza | `netezza` | assessable | [Netezza](/docs/sources/netezza) | | Hadoop | `hadoop` | assessable | [Hadoop](/docs/sources/hadoop) | | Google BigQuery | `bigquery` | assessable | [BigQuery](/docs/sources/bigquery) | | Greenplum | `greenplum` | cataloged | [Greenplum](/docs/sources/greenplum) | | Vertica | `vertica` | cataloged | [Vertica](/docs/sources/vertica) | The seven Databricks Migration & Modernization specialization tracks represented here are: * Microsoft SQL Migration; * Snowflake Migration; * AWS Redshift Migration; * Azure Synapse Migration; * Oracle Migration; * Hadoop Migration; * Teradata Migration. ## ERP, SaaS, and operational systems [#erp-saas-and-operational-systems] | Source | Profile | Important variants | Guide | | ---------------------- | -------------- | ---------------------------------------------------- | ------------------------------------------ | | SAP | `sap` | BDC, S/4HANA, ECC, BW/4HANA, HANA, Datasphere | [SAP](/docs/sources/sap) | | Microsoft Dynamics 365 | `dynamics_365` | Dataverse, Finance and Operations, Sales and Service | [Dynamics 365](/docs/sources/dynamics-365) | | Salesforce | `salesforce` | Sales/Service Cloud, Marketing Cloud | [Salesforce](/docs/sources/salesforce) | | PostgreSQL | `postgresql` | PostgreSQL, Aurora, RDS | [PostgreSQL](/docs/sources/postgresql) | | MySQL | `mysql` | MySQL, Aurora, RDS, MariaDB | [MySQL](/docs/sources/mysql) | | Workday | `workday` | HCM, reports and extracts | [Workday](/docs/sources/workday) | | ServiceNow | `servicenow` | ServiceNow platform | [ServiceNow](/docs/sources/servicenow) | ## Already on Databricks [#already-on-databricks] | Source | Profile | Modernization variants | Guide | | ------------------------ | ------------ | -------------------------------------------------------- | ------------------------------------------------------------------ | | Databricks native estate | `databricks` | Unity Catalog, workspace consolidation, cost/performance | [Databricks-native modernization](/docs/sources/databricks-native) | ## Pipelines, query engines, and streams [#pipelines-query-engines-and-streams] | Source | Profile | Route | Guide | | ---------------------------------------------------- | ------------------- | --------------------------------------------------- | ---------------------------------------------------- | | Informatica, SSIS, ADF, Glue, Talend, Matillion, ODI | `etl_modernization` | pipeline export, mapping, implementation, rehearsal | [ETL modernization](/docs/sources/etl-modernization) | | IBM Db2 and mainframe data | `db2` | platform-aware unload or log capture | [Db2](/docs/sources/db2) | | Amazon Athena | `athena` | catalog-by-catalog federation or materialization | [Athena](/docs/sources/athena) | | Presto | `presto` | backing-source disposition | [Presto](/docs/sources/presto) | | Trino and Starburst | `trino` | backing-source disposition | [Trino](/docs/sources/trino) | | Kafka, Confluent, Amazon MSK | `kafka` | checkpointed parallel run | [Kafka](/docs/sources/kafka) | | Azure Event Hubs | `event_hubs` | offsets, checkpoints, loss/duplicate proof | [Event Hubs](/docs/sources/event-hubs) | | Amazon Kinesis | `kinesis` | sequence checkpoints and parallel run | [Kinesis](/docs/sources/kinesis) | ## Ten areas every source pack covers [#ten-areas-every-source-pack-covers] `SOURCE_MIGRATION_AREAS` requires every plan to address: * inventory and dependencies; * SQL and code conversion or disposition; * data movement; * pipelines and orchestration; * security and Unity Catalog governance; * functional and data validation; * performance and scale; * BI and downstream consumers; * cutover and rollback; * Databricks modernization. ## Use the registry from TypeScript [#use-the-registry-from-typescript] ```ts import { createSourceMigrationPlan, resolveSourceSystemProfile, } from '@fabricorg/airlift'; const profile = resolveSourceSystemProfile('d365'); const plan = createSourceMigrationPlan(profile.id, { variant: 'dynamics_365_finance_operations', }); console.log(profile.archetype); // saas console.log(profile.implementationRoutingLevel); // cataloged; routing breadth, not achieved proof console.log(plan.steps.map((step) => step.id)); ``` Continue with the [source-pack contract](/docs/sources/source-packs) and [developer workflow](/docs/sources/developer-workflow). # Kafka to Databricks # Kafka to Databricks [#kafka-to-databricks] ```bash fa source inspect kafka fa source plan kafka --variant apache_kafka fa source plan kafka --variant confluent_cloud --json fa source plan kafka --variant amazon_msk --json ``` A streaming migration is a continuity problem, not a table copy. Airlift inventories topics, partitions, schemas and compatibility rules, consumer groups, Kafka Connect, retention, ACLs, producers, and downstream consumers. ## Streaming lifecycle [#streaming-lifecycle] 1. Record the accepted topic, schema, consumer, retention, and access inventory. 2. Define target streaming tables, checkpoints, evolution, dead-letter handling, replay, and rollback policy. 3. Start Structured Streaming or Lakeflow from a recorded offset boundary and run in parallel or dual-publish where supported. 4. Admit evidence for offset continuity, event loss and duplicates, key ordering, schema compatibility, lag, throughput, and business aggregates. 5. Freeze the producer/consumer wave, collect authenticated approvals, switch through a certified effector, and verify or roll back. 6. Modernize Kafka Streams state, connectors, and serving products separately. Custom serializers, exactly-once assumptions, Kafka Streams state, custom connectors, and cross-topic transactions remain explicit residue. Credentials stay in scoped connection objects. The pack is `cataloged`. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe kafka fa source recipe kafka --variant apache_kafka --json > .airlift/kafka-recipe.json ``` The App is engagement-aware. Apache Kafka appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect kafka --json > .airlift/kafka-profile.json fa source plan kafka --variant apache_kafka --json > .airlift/kafka-capability-plan.json ``` Expected artifacts: * .airlift/kafka-profile.json * .airlift/kafka-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file kafka-estate.json --idempotency-key kafka-estate-v1 fa connection register --file kafka-connection.json --idempotency-key kafka-connection-v1 fa engagement update --file kafka-scope.json --idempotency-key kafka-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file kafka-assessment-start.json --idempotency-key kafka-assessment-start-v1 fa assessment status --json fa assessment record --file kafka-assessment-record.json --idempotency-key kafka-assessment-record-v1 fa assessment accept --file kafka-assessment-accept.json --idempotency-key kafka-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/kafka` after replacing the placeholder ID with the governed engagement ID. ### 3. Implement the admitted source adapter [#3-implement-the-admitted-source-adapter] > No source-specific executable compiler exists in this release. Continue with assessment, governed work, and an admitted adapter; Airlift does not invent executable output. Expected artifacts: * Capability plan and adapter requirements only Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. > This source is cataloged or assessable but has no source-specific executable compiler in the installed Airlift generation. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file kafka-migration-plan.json --idempotency-key kafka-plan-v1 fa conversion batch create --file kafka-batch.json --idempotency-key kafka-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file kafka-transfer.json --idempotency-key kafka-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file kafka-validation.json --idempotency-key kafka-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file kafka-evidence-export.json --idempotency-key kafka-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to Apache Kafka; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # Amazon Kinesis to Databricks # Amazon Kinesis to Databricks [#amazon-kinesis-to-databricks] ```bash fa source inspect kinesis fa source plan kinesis --variant kinesis_data_streams fa source plan kinesis --variant kinesis_firehose --json ``` Inventory streams, shards, schemas, consumer applications, Firehose destinations, retention, IAM, producers, and downstream consumers. Run checkpointed shard reads, enhanced fan-out, or delivery-file ingestion from a recorded sequence boundary and retain checkpoint, restart, lag, throughput, and reconciliation evidence. Independent validation covers sequence continuity, event loss and duplicates, partition-key ordering, schema compatibility, resharding, business aggregates, and accepted latency or throughput thresholds. Producer aggregation, consumer lease state, Firehose transforms, and cross-stream ordering remain explicit residue. Airlift freezes the exact streams and consumers, enforces separation of duties, and records a checkpointed switch, verification, or rollback. Structured Streaming, Lakeflow, Delta event tables, Unity Catalog, and streaming tables are modernized in a separate release. The pack is `cataloged` pending live adapter and scale evidence. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe kinesis fa source recipe kinesis --variant kinesis_data_streams --json > .airlift/kinesis-recipe.json ``` The App is engagement-aware. Amazon Kinesis appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect kinesis --json > .airlift/kinesis-profile.json fa source plan kinesis --variant kinesis_data_streams --json > .airlift/kinesis-capability-plan.json ``` Expected artifacts: * .airlift/kinesis-profile.json * .airlift/kinesis-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file kinesis-estate.json --idempotency-key kinesis-estate-v1 fa connection register --file kinesis-connection.json --idempotency-key kinesis-connection-v1 fa engagement update --file kinesis-scope.json --idempotency-key kinesis-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file kinesis-assessment-start.json --idempotency-key kinesis-assessment-start-v1 fa assessment status --json fa assessment record --file kinesis-assessment-record.json --idempotency-key kinesis-assessment-record-v1 fa assessment accept --file kinesis-assessment-accept.json --idempotency-key kinesis-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/kinesis` after replacing the placeholder ID with the governed engagement ID. ### 3. Implement the admitted source adapter [#3-implement-the-admitted-source-adapter] > No source-specific executable compiler exists in this release. Continue with assessment, governed work, and an admitted adapter; Airlift does not invent executable output. Expected artifacts: * Capability plan and adapter requirements only Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. > This source is cataloged or assessable but has no source-specific executable compiler in the installed Airlift generation. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file kinesis-migration-plan.json --idempotency-key kinesis-plan-v1 fa conversion batch create --file kinesis-batch.json --idempotency-key kinesis-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file kinesis-transfer.json --idempotency-key kinesis-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file kinesis-validation.json --idempotency-key kinesis-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file kinesis-evidence-export.json --idempotency-key kinesis-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to Amazon Kinesis; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # MySQL to Databricks # MySQL to Databricks [#mysql-to-databricks] ```bash fa source inspect mysql fa source plan mysql --variant mysql fa source plan mysql --variant aurora_mysql --json ``` Variants are MySQL, Amazon Aurora MySQL, Amazon RDS for MySQL, and MariaDB. Inventory tables, views, routines, triggers, event schedules, binlog change data, and downstream applications before planning waves. ## What Airlift adds [#what-airlift-adds] * A governed estate with the exact platform variant and opaque connection reference. * Explicit disposition of routines, scheduler behavior, and engine-specific features into deterministic, repairable, or human-owned lanes. * Snapshot and binlog/CDC catch-up evidence with retained GTID or file-position state, restart checkpoints, and reconciliation. * Independent checks for character sets, collations, zero dates, temporal types, unsigned numerics, and JSON semantics. * Signed certificates, frozen wave scope, authenticated approvals, and a verifiable consumer cutover or rollback. ## Migration route [#migration-route] 1. Accept metadata and dependency artifacts through `assessment_start` and `assessment_record`. 2. Register schema objects and consumers. Route routines, scheduler behavior, engine-specific features, cross-schema side effects, and transaction coupling to deterministic, repairable, or human-owned lanes. 3. Run a consistent snapshot and binlog/CDC or watermark catch-up. Retain GTID or file-position boundaries, restart checkpoints, deletes, lag, and reconciliation. 4. Validate character sets, collations, zero dates, temporal types, unsigned numerics, JSON, change ordering, business queries, and performance thresholds independently. 5. Mint certificates from admitted evidence; freeze and approve the consumer cutover; verify or roll back. 6. Modernize to Lakeflow Connect, Delta, Unity Catalog, Databricks SQL, and streaming change-data products in a separate release. The pack is `cataloged`. Generate and check a live source-certification manifest before claiming assessable, executable, or certifiable support. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe mysql fa source recipe mysql --variant mysql --json > .airlift/mysql-recipe.json ``` The App is engagement-aware. MySQL and Aurora MySQL appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect mysql --json > .airlift/mysql-profile.json fa source plan mysql --variant mysql --json > .airlift/mysql-capability-plan.json ``` Expected artifacts: * .airlift/mysql-profile.json * .airlift/mysql-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file mysql-estate.json --idempotency-key mysql-estate-v1 fa connection register --file mysql-connection.json --idempotency-key mysql-connection-v1 fa engagement update --file mysql-scope.json --idempotency-key mysql-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file mysql-assessment-start.json --idempotency-key mysql-assessment-start-v1 fa assessment status --json fa assessment record --file mysql-assessment-record.json --idempotency-key mysql-assessment-record-v1 fa assessment accept --file mysql-assessment-accept.json --idempotency-key mysql-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/mysql` after replacing the placeholder ID with the governed engagement ID. ### 3. Implement the admitted source adapter [#3-implement-the-admitted-source-adapter] > No source-specific executable compiler exists in this release. Continue with assessment, governed work, and an admitted adapter; Airlift does not invent executable output. Expected artifacts: * Capability plan and adapter requirements only Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. > This source is cataloged or assessable but has no source-specific executable compiler in the installed Airlift generation. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file mysql-migration-plan.json --idempotency-key mysql-plan-v1 fa conversion batch create --file mysql-batch.json --idempotency-key mysql-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file mysql-transfer.json --idempotency-key mysql-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file mysql-validation.json --idempotency-key mysql-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file mysql-evidence-export.json --idempotency-key mysql-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to MySQL and Aurora MySQL; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # Netezza # Migrate Netezza with Airlift [#migrate-netezza-with-airlift] Netezza is an additional Airlift source profile beyond the seven named M\&M program tracks. The profile covers SQL, NZPLSQL, external tables, `nzload`/`nz_migrate`, distribution keys, scheduler dependencies, extensions, and downstream BI. ## What Airlift adds to a Netezza migration [#what-airlift-adds-to-a-netezza-migration] | Stage | Databricks tool or project adapter | Airlift responsibility | | --------- | ---------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- | | discover | Lakebridge Analyzer plus a reviewed appliance/workload extraction | accept inventory, dependencies, exclusions, source version, extraction identity, and report digests | | plan | assessment and unload measurements inform target architecture | assign owners and dependency-aware waves; preserve NZPLSQL, extension, utility, and human-work decisions | | convert | BladeBridge converts supported Netezza SQL | record attempts, tool generations, artifact digests, warnings, and residue lanes | | move data | external-table or utility snapshot plus a source-specific delta strategy | track manifests, watermarks, lag, restart checkpoints, rejected rows, counts, and target snapshots | | validate | admitted Experiments schema, row, aggregate, sampled, and business scenarios | admit independent evidence because there is no universal Lakebridge Reconcile connector | | certify | Airlift evaluates the active readiness profile | mint a signed certificate identifying artifacts, snapshots, evidence, and policy | | cut over | project load, scheduler, application, and BI connection effectors | freeze scope, enforce approvals, checkpoint, apply once, verify, and retain rollback evidence | | modernize | Unity Catalog, Lakeflow, Delta, liquid clustering, and Databricks SQL work | keep appliance-specific redesign separate from baseline parity | This preserves the work that an SQL-only report misses: appliance utilities, extensions, physical organization, scheduler calls, data extraction, permissions, and consumers. ## Inspect and generate a plan [#inspect-and-generate-a-plan] ```bash fa source inspect netezza fa source plan nz --json > .airlift/netezza-plan.json ``` ## Analyze [#analyze] Lakebridge Analyzer supports Netezza exports. The current upstream matrix does not list a Netezza database profiler, so collect workload and appliance metrics with a reviewed client extraction and retain that artifact separately. ```bash databricks labs lakebridge analyze \ --source-directory ./source-export \ --source-tech "Netezza" \ --report-file ./artifacts/netezza-analysis.xlsx \ --generate-json true ``` Inventory NZPLSQL, UDF/UDA extensions, external table formats, load scripts, distribution/organization clauses, scheduler calls, grants, and consumers in addition to standard DDL. ## Convert [#convert] ```bash databricks labs lakebridge transpile \ --source-dialect netezza \ --input-source ./source-export/sql \ --output-folder ./artifacts/converted ``` BladeBridge is the preferred deterministic path. Route NZPLSQL, extensions, appliance utilities, cross-database references, and physical organization behavior to explicit residue. Airlift preserves these objects and owners in the ledger. ## Transfer and validate [#transfer-and-validate] Implement snapshot extraction with external tables or approved Netezza utilities, then add a source-specific delta strategy. Return file manifests, restart points, source watermarks, target snapshots, row counts, and rejected-row evidence. Lakebridge Reconcile does not currently list a Netezza connector. Run schema, row, aggregate, sampled, and business-query scenarios through an admitted Fabric Experiments producer. Record the immutable run and evidence digest before resolving readiness. ## Certify and cut over [#certify-and-cut-over] Assign profiles to SQL, NZPLSQL, utilities, extensions, jobs, and consumers, then admit source-specific validation runs before minting certificates. The project effector may switch loads, schedules, endpoints, and BI connections only after Airlift verifies the frozen wave and current approvals. Retain extraction completion, target observations, and rollback evidence because converted SQL alone cannot prove appliance retirement. ## Modernize after parity [#modernize-after-parity] Replace appliance utilities, map tables to Delta and liquid clustering from measured workloads, move jobs into Lakeflow, and apply Unity Catalog governance in a separate modernization release. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe netezza fa source recipe netezza --variant netezza_performance_server --json > .airlift/netezza-recipe.json ``` The App is engagement-aware. Netezza appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect netezza --json > .airlift/netezza-profile.json fa source plan netezza --variant netezza_performance_server --json > .airlift/netezza-capability-plan.json ``` Expected artifacts: * .airlift/netezza-profile.json * .airlift/netezza-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file netezza-estate.json --idempotency-key netezza-estate-v1 fa connection register --file netezza-connection.json --idempotency-key netezza-connection-v1 fa engagement update --file netezza-scope.json --idempotency-key netezza-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file netezza-assessment-start.json --idempotency-key netezza-assessment-start-v1 fa assessment status --json fa assessment record --file netezza-assessment-record.json --idempotency-key netezza-assessment-record-v1 fa assessment accept --file netezza-assessment-accept.json --idempotency-key netezza-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/netezza` after replacing the placeholder ID with the governed engagement ID. ### 3. Implement the admitted source adapter [#3-implement-the-admitted-source-adapter] > No source-specific executable compiler exists in this release. Continue with assessment, governed work, and an admitted adapter; Airlift does not invent executable output. Expected artifacts: * Capability plan and adapter requirements only Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. > This source is cataloged or assessable but has no source-specific executable compiler in the installed Airlift generation. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file netezza-migration-plan.json --idempotency-key netezza-plan-v1 fa conversion batch create --file netezza-batch.json --idempotency-key netezza-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file netezza-transfer.json --idempotency-key netezza-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file netezza-validation.json --idempotency-key netezza-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file netezza-evidence-export.json --idempotency-key netezza-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to Netezza; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # Oracle # Migrate Oracle with Airlift [#migrate-oracle-with-airlift] The Oracle profile covers SQL and PL/SQL, packages, procedures, materialized views, database links, ODI, scheduler jobs, partitioning, security, and downstream reports. Airlift records each workload class and its dependencies instead of treating a schema export as the whole estate. ## What Airlift adds to an Oracle migration [#what-airlift-adds-to-an-oracle-migration] | Stage | Databricks tool or project adapter | Airlift responsibility | | --------- | ------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | | discover | Lakebridge Profiler and Analyzer scan SQL, PL/SQL, and ODI exports | accept inventory, dependencies, exclusions, source version, licensing constraints, and report digests | | plan | assessment output informs target architecture | assign owners and dependency-aware waves; preserve package, database-link, scheduler, and human-work decisions | | convert | BladeBridge converts supported Oracle SQL and PL/SQL | record attempts, tool generations, artifact digests, warnings, and residue lanes | | move data | consistent snapshot plus SCN or timestamp catch-up | track SCNs, manifests, lag, restart checkpoints, target Delta versions, counts, and reconciliation | | validate | Lakebridge Reconcile plus package, NLS, exception, and business scenarios | admit independent data and behavior evidence against exact source and target snapshots | | certify | Airlift evaluates the active readiness profile | mint a signed certificate identifying artifacts, snapshots, evidence, and policy | | cut over | project service, scheduler, ODI, report, and application effectors | freeze scope, enforce approvals, checkpoint, apply once, verify, and retain rollback evidence | | modernize | Unity Catalog, Lakeflow, Delta, Databricks SQL, and Python work | keep procedural redesign separate from the parity-preserving release | This keeps packages, scheduler chains, database links, ODI workflows, permissions, and report consumers visible alongside tables so schema conversion cannot overstate estate readiness. ## Compile the executable migration pack [#compile-the-executable-migration-pack] Export tables, views, PL/SQL packages, materialized views, database links, scheduler jobs, ODI workflows, grants, and consumers into a credential-free manifest. Select `oracle_database`, `oracle_exadata`, or `oracle_autonomous`, attach the observed hard cases to their owning objects, then compile it: ```bash fa migration-pack inspect --file oracle-manifest.json fa migration-pack plan --file oracle-manifest.json --json > oracle-plan.json fa migration-pack register --file oracle-plan.json \ --engagement-id --estate-id \ --artifact-id --idempotency-key ``` The compiler validates the dependency graph and Oracle hard-case corpus, routes stateful packages and autonomous transactions to human remediation, and emits the SCN-aware transfer contract, required Experiments suites, target mappings, and Runway deployment requirements. See [migration-pack commands](/docs/cli/migration-packs) for the schema and certification workflow. ## Inspect and generate a plan [#inspect-and-generate-a-plan] ```bash fa source inspect oracle fa source plan oracle --json > .airlift/oracle-plan.json ``` ## Profile and analyze [#profile-and-analyze] The current Lakebridge profiler targets multitenant Oracle, connects to `CDB$ROOT`, and may query AWR views. Verify network/authentication constraints and Oracle Diagnostic Pack licensing before enabling those queries. ```bash databricks labs lakebridge configure-database-profiler databricks labs lakebridge execute-database-profiler \ --source-tech oracle \ --output-folder ./artifacts/profile databricks labs lakebridge analyze \ --source-directory ./source-export \ --source-tech "Oracle" \ --report-file ./artifacts/oracle-analysis.xlsx \ --generate-json true ``` Run a separate Analyzer pass with `Oracle Data Integrator` for ODI exports. Register packages, package bodies, scheduler chains, database links, grants, and consumer dependencies as objects or linked evidence. ## Convert [#convert] ```bash databricks labs lakebridge transpile \ --source-dialect oracle \ --input-source ./source-export/sql \ --output-folder ./artifacts/converted ``` BladeBridge is the preferred deterministic path. Route stateful packages, autonomous transactions, database links, advanced queues, Java procedures, proprietary utilities, and unresolved optimizer behavior to residue. Record one artifact and tool version per attempt. ## Transfer and reconcile [#transfer-and-reconcile] Select a consistent snapshot and SCN or timestamp catch-up per table class. Keep the source SCN and target Delta snapshot with all validation evidence. ```bash databricks labs lakebridge configure-reconcile databricks labs lakebridge auto-configure-recon-tables databricks labs lakebridge reconcile ``` Validate `NUMBER` precision, `DATE`/`TIMESTAMP`, empty string versus null, NLS settings, identifier case, exception behavior, and representative package logic. Data parity does not prove procedural side effects; use separate Experiments scenarios. ## Certify and cut over [#certify-and-cut-over] Assign validation profiles separately to tables, views, packages, scheduler work, ODI flows, and reports. Airlift mints certificates only from admitted current evidence. The project cutover effector should switch database services, jobs, ODI schedules, report connections, and application endpoints through rehearsed checkpoint/apply/verify steps. An uncertain endpoint result remains open for reconciliation rather than being retried. ## Modernize after parity [#modernize-after-parity] Replace procedural packages with Databricks SQL or Python where appropriate, move jobs to Lakeflow, map grants to Unity Catalog, and validate Delta constraints, generated columns, and Databricks SQL performance in a separate release. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe oracle fa source recipe oracle --variant oracle_database --json > .airlift/oracle-recipe.json ``` The App is engagement-aware. Oracle appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect oracle --json > .airlift/oracle-profile.json fa source plan oracle --variant oracle_database --json > .airlift/oracle-capability-plan.json ``` Expected artifacts: * .airlift/oracle-profile.json * .airlift/oracle-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file oracle-estate.json --idempotency-key oracle-estate-v1 fa connection register --file oracle-connection.json --idempotency-key oracle-connection-v1 fa engagement update --file oracle-scope.json --idempotency-key oracle-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file oracle-assessment-start.json --idempotency-key oracle-assessment-start-v1 fa assessment status --json fa assessment record --file oracle-assessment-record.json --idempotency-key oracle-assessment-record-v1 fa assessment accept --file oracle-assessment-accept.json --idempotency-key oracle-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/oracle` after replacing the placeholder ID with the governed engagement ID. ### 3. Compile the oracle migration pack [#3-compile-the-oracle-migration-pack] ```bash fa migration-pack inspect --file oracle-manifest.json fa migration-pack plan --file oracle-manifest.json --json > generated/oracle-plan.json ``` Expected artifacts: * Dependency-aware migration pack * Transfer requirements * Validation requirements * Residue lanes Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file oracle-migration-plan.json --idempotency-key oracle-plan-v1 fa conversion batch create --file oracle-batch.json --idempotency-key oracle-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file oracle-transfer.json --idempotency-key oracle-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file oracle-validation.json --idempotency-key oracle-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file oracle-evidence-export.json --idempotency-key oracle-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to Oracle; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # PostgreSQL to Databricks # PostgreSQL to Databricks [#postgresql-to-databricks] ```bash fa source inspect postgres fa source plan postgresql --variant postgresql fa source plan postgresql --variant aurora_postgresql --json ``` The pack covers PostgreSQL, Amazon Aurora PostgreSQL, and Amazon RDS for PostgreSQL. It inventories tables, schemas, views, functions, extensions, logical replication, schedulers, and application or BI consumers. ## What Airlift adds [#what-airlift-adds] * A governed estate with the exact platform variant and opaque connection reference. * Dependency-aware waves for tables, functions, ingestion jobs, and consumers. * Explicit disposition of extensions, PL-language functions, cross-database access, large objects, and transactional side effects. * Snapshot, WAL/CDC or watermark catch-up evidence with retained LSN and restart state. * Independent checks for JSONB, arrays, collation, numeric/timestamp semantics, ordering, deletes, business queries, and accepted performance thresholds. * Signed certificates, frozen wave scope, authenticated approvals, and a verifiable consumer cutover or rollback. Lakeflow Connect or another admitted connector may perform movement; Airlift records what ran and what it proved. The public pack is `cataloged` until immutable live runs support a higher level. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe postgresql fa source recipe postgresql --variant postgresql --json > .airlift/postgresql-recipe.json ``` The App is engagement-aware. PostgreSQL and Aurora PostgreSQL appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect postgresql --json > .airlift/postgresql-profile.json fa source plan postgresql --variant postgresql --json > .airlift/postgresql-capability-plan.json ``` Expected artifacts: * .airlift/postgresql-profile.json * .airlift/postgresql-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file postgresql-estate.json --idempotency-key postgresql-estate-v1 fa connection register --file postgresql-connection.json --idempotency-key postgresql-connection-v1 fa engagement update --file postgresql-scope.json --idempotency-key postgresql-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file postgresql-assessment-start.json --idempotency-key postgresql-assessment-start-v1 fa assessment status --json fa assessment record --file postgresql-assessment-record.json --idempotency-key postgresql-assessment-record-v1 fa assessment accept --file postgresql-assessment-accept.json --idempotency-key postgresql-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/postgresql` after replacing the placeholder ID with the governed engagement ID. ### 3. Implement the admitted source adapter [#3-implement-the-admitted-source-adapter] > No source-specific executable compiler exists in this release. Continue with assessment, governed work, and an admitted adapter; Airlift does not invent executable output. Expected artifacts: * Capability plan and adapter requirements only Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. > This source is cataloged or assessable but has no source-specific executable compiler in the installed Airlift generation. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file postgresql-migration-plan.json --idempotency-key postgresql-plan-v1 fa conversion batch create --file postgresql-batch.json --idempotency-key postgresql-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file postgresql-transfer.json --idempotency-key postgresql-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file postgresql-validation.json --idempotency-key postgresql-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file postgresql-evidence-export.json --idempotency-key postgresql-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to PostgreSQL and Aurora PostgreSQL; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # Presto to Databricks # Presto to Databricks [#presto-to-databricks] ```bash fa source inspect presto fa source plan presto --variant presto ``` Presto often exposes many independent backing systems. Airlift inventories catalogs, schemas, tables, views, queries, connectors, custom functions, session properties, access controls, schedulers, and consumers. Each catalog receives its own disposition: Lakehouse Federation, governed registration, Delta materialization, source-specific ingestion, SQL conversion, or a human lane. Movement remains owned by the adapter for each backing source. Airlift records the artifact, source boundary, target snapshot, restart state, and validation runs that join those routes into one migration wave. Independent scenarios test dialect semantics, connector-specific types, federated joins, functions, access, business results, and performance. After certification, switch queries and consumers through a frozen, approved wave with checkpoint/apply-once/verify/rollback evidence. Modernization can progressively replace federation with governed Delta products without changing the baseline certificate. The pack is `cataloged` pending representative live-catalog evidence. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe presto fa source recipe presto --variant presto --json > .airlift/presto-recipe.json ``` The App is engagement-aware. Presto appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect presto --json > .airlift/presto-profile.json fa source plan presto --variant presto --json > .airlift/presto-capability-plan.json ``` Expected artifacts: * .airlift/presto-profile.json * .airlift/presto-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file presto-estate.json --idempotency-key presto-estate-v1 fa connection register --file presto-connection.json --idempotency-key presto-connection-v1 fa engagement update --file presto-scope.json --idempotency-key presto-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file presto-assessment-start.json --idempotency-key presto-assessment-start-v1 fa assessment status --json fa assessment record --file presto-assessment-record.json --idempotency-key presto-assessment-record-v1 fa assessment accept --file presto-assessment-accept.json --idempotency-key presto-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/presto` after replacing the placeholder ID with the governed engagement ID. ### 3. Implement the admitted source adapter [#3-implement-the-admitted-source-adapter] > No source-specific executable compiler exists in this release. Continue with assessment, governed work, and an admitted adapter; Airlift does not invent executable output. Expected artifacts: * Capability plan and adapter requirements only Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. > This source is cataloged or assessable but has no source-specific executable compiler in the installed Airlift generation. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file presto-migration-plan.json --idempotency-key presto-plan-v1 fa conversion batch create --file presto-batch.json --idempotency-key presto-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file presto-transfer.json --idempotency-key presto-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file presto-validation.json --idempotency-key presto-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file presto-evidence-export.json --idempotency-key presto-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to Presto; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # Amazon Redshift # Migrate Amazon Redshift with Airlift [#migrate-amazon-redshift-with-airlift] The Redshift profile covers SQL objects, stored procedures, WLM queues, Spectrum, `COPY`/`UNLOAD`, materialized views, scheduled queries, and downstream BI. Select the profiler variant that matches the actual deployment; Airlift records that source detail with the estate evidence. ## What Airlift adds to an Amazon Redshift migration [#what-airlift-adds-to-an-amazon-redshift-migration] | Stage | Databricks tool or project adapter | Airlift responsibility | | --------- | -------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- | | discover | the matching Lakebridge Profiler variant and Analyzer scan SQL plus exported estate metadata | accept inventory, dependencies, exclusions, deployment variant, source version, and report digests | | plan | assessment output informs target architecture | assign owners and dependency-aware waves; preserve WLM, Spectrum, UDF, and human-work decisions | | convert | BladeBridge converts supported Redshift SQL | record attempts, tool generations, artifact digests, warnings, and residue lanes | | move data | restartable UNLOAD to S3 plus timestamp, sequence, or application delta | track manifests, watermarks, lag, restart checkpoints, target Delta versions, counts, and reconciliation | | validate | Lakebridge Reconcile plus SUPER, Spectrum, workload, and business scenarios | admit independent functional, data, and performance evidence against exact snapshots | | certify | Airlift evaluates the active readiness profile | mint a signed certificate identifying artifacts, snapshots, evidence, and policy | | cut over | project query, schedule, ingestion, and BI connection effectors | freeze scope, enforce approvals, checkpoint, apply once, verify, and retain rollback evidence | | modernize | Unity Catalog, Lakeflow, Delta, and Databricks SQL work | keep distribution, layout, and workload redesign separate from baseline parity | This keeps Spectrum paths, IAM mappings, scheduled queries, ingestion manifests, and BI consumers attached to their SQL dependencies instead of reporting migration progress from converted files alone. ## Compile the executable migration pack [#compile-the-executable-migration-pack] Create a versioned manifest containing tables and views, Spectrum objects, WLM rules, COPY/UNLOAD jobs, scheduled queries, UDFs, permissions, and consumers. Select the exact provisioned, Multi-AZ, or serverless variant and mark observed hard cases, then run: ```bash fa migration-pack inspect --file redshift-manifest.json fa migration-pack plan --file redshift-manifest.json --json > redshift-plan.json ``` The compiler rejects a mismatched variant, validates dependencies, reports missing Redshift hard cases, and routes unsupported behavior explicitly. Its plan requires an S3 manifest plus watermark restart contract, source-specific Experiments suites, Databricks target mappings, and a Runway deployment manifest. See [migration-pack commands](/docs/cli/migration-packs) for registration and certification. ## Inspect and generate a plan [#inspect-and-generate-a-plan] ```bash fa source inspect redshift fa source plan aws_redshift --json > .airlift/redshift-plan.json ``` ## Profile and analyze [#profile-and-analyze] Profiler variants are `redshift_serverless`, `redshift_provisioned`, and `redshift_provisioned_multi_az`: ```bash databricks labs lakebridge configure-database-profiler databricks labs lakebridge execute-database-profiler \ --source-tech redshift_provisioned \ --output-folder ./artifacts/profile databricks labs lakebridge analyze \ --source-directory ./source-export \ --source-tech "Redshift" \ --report-file ./artifacts/redshift-analysis.xlsx \ --generate-json true ``` Inventory WLM rules, Spectrum paths and IAM dependencies, scheduled queries, load/unload manifests, UDFs, late-binding views, and BI connections in addition to SQL files. ## Convert [#convert] ```bash databricks labs lakebridge transpile \ --source-dialect redshift \ --input-source ./source-export/sql \ --output-folder ./artifacts/converted ``` BladeBridge is the preferred deterministic path. Common residue includes Python UDFs, leader-node-only functions, WLM behavior, Spectrum-specific constructs, late-binding views, and cross-database queries. ## Transfer and reconcile [#transfer-and-reconcile] Use restartable `UNLOAD` manifests to S3 for the snapshot and select a timestamp, sequence, or application delta for catch-up. Preserve source query snapshot, manifest, watermark, and target Delta version. ```bash databricks labs lakebridge configure-reconcile databricks labs lakebridge auto-configure-recon-tables databricks labs lakebridge reconcile ``` Validate `SUPER`, semi-structured values, decimal/timestamp behavior, Spectrum data, sort/distribution-sensitive queries, and WLM-sensitive performance. Reconcile and performance are separate readiness tracks. ## Certify and cut over [#certify-and-cut-over] Require independent profiles for data parity, semi-structured behavior, Spectrum-backed queries, and accepted performance thresholds. The project effector may switch ingestion, scheduled queries, application endpoints, and BI connections only after Airlift verifies fresh certificates and approvals. Retain the final UNLOAD/catch-up watermark, endpoint observations, and rollback result with the wave. ## Modernize after parity [#modernize-after-parity] Replace distribution and sort-key assumptions with measured Delta layout and liquid clustering, move ingestion to Lakeflow, manage S3 through Unity Catalog external locations, and retest workload isolation on Databricks SQL warehouses. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe redshift fa source recipe redshift --variant redshift_serverless --json > .airlift/redshift-recipe.json ``` The App is engagement-aware. Amazon Redshift appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect redshift --json > .airlift/redshift-profile.json fa source plan redshift --variant redshift_serverless --json > .airlift/redshift-capability-plan.json ``` Expected artifacts: * .airlift/redshift-profile.json * .airlift/redshift-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file redshift-estate.json --idempotency-key redshift-estate-v1 fa connection register --file redshift-connection.json --idempotency-key redshift-connection-v1 fa engagement update --file redshift-scope.json --idempotency-key redshift-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file redshift-assessment-start.json --idempotency-key redshift-assessment-start-v1 fa assessment status --json fa assessment record --file redshift-assessment-record.json --idempotency-key redshift-assessment-record-v1 fa assessment accept --file redshift-assessment-accept.json --idempotency-key redshift-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/redshift` after replacing the placeholder ID with the governed engagement ID. ### 3. Compile the redshift migration pack [#3-compile-the-redshift-migration-pack] ```bash fa migration-pack inspect --file redshift-manifest.json fa migration-pack plan --file redshift-manifest.json --json > generated/redshift-plan.json ``` Expected artifacts: * Dependency-aware migration pack * Transfer requirements * Validation requirements * Residue lanes Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file redshift-migration-plan.json --idempotency-key redshift-plan-v1 fa conversion batch create --file redshift-batch.json --idempotency-key redshift-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file redshift-transfer.json --idempotency-key redshift-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file redshift-validation.json --idempotency-key redshift-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file redshift-evidence-export.json --idempotency-key redshift-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to Amazon Redshift; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # Salesforce to Databricks # Salesforce to Databricks [#salesforce-to-databricks] Airlift makes Salesforce ingestion auditable as a migration: scope standard and custom objects, relationships, history, files, deletes, reports, and activations; track the connector cursor and restart; validate the resulting customer data product; then switch consumers in governed waves. ```bash fa source inspect salesforce fa source plan salesforce --variant salesforce_sales_service fa source plan salesforce --variant salesforce_marketing_cloud --json ``` ## Developer workflow [#developer-workflow] * Feed object, field, relationship, history, file, and consumer metadata into the assessment adapter. Record only immutable artifact references. * Register entities and custom transformations as migration objects. Polymorphic relationships, formulas, encrypted fields, large binaries, and Marketing Cloud data extensions remain explicit residue until dispositioned. * Run the admitted connector or bulk API route and return snapshot ID, replay cursor, deletion handling, restart checkpoint, and lag observations. * Validate object counts, relationship integrity, history, deletes, and accepted funnel or service metrics with an independent provider run. * Mint certificates only after the assigned profile passes. Cut over reports, activations, and applications through a frozen wave with rollback. Target modernization can include Lakeflow Connect, Unity Catalog, Delta customer-360 products, Databricks SQL, and feature tables. It is tracked after baseline certification so redesign does not weaken the migration claim. The pack is `cataloged`; live connector availability is not itself Airlift certification. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe salesforce fa source recipe salesforce --variant salesforce_sales_service --json > .airlift/salesforce-recipe.json ``` The App is engagement-aware. Salesforce appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect salesforce --json > .airlift/salesforce-profile.json fa source plan salesforce --variant salesforce_sales_service --json > .airlift/salesforce-capability-plan.json ``` Expected artifacts: * .airlift/salesforce-profile.json * .airlift/salesforce-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file salesforce-estate.json --idempotency-key salesforce-estate-v1 fa connection register --file salesforce-connection.json --idempotency-key salesforce-connection-v1 fa engagement update --file salesforce-scope.json --idempotency-key salesforce-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file salesforce-assessment-start.json --idempotency-key salesforce-assessment-start-v1 fa assessment status --json fa assessment record --file salesforce-assessment-record.json --idempotency-key salesforce-assessment-record-v1 fa assessment accept --file salesforce-assessment-accept.json --idempotency-key salesforce-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/salesforce` after replacing the placeholder ID with the governed engagement ID. ### 3. Implement the admitted source adapter [#3-implement-the-admitted-source-adapter] > No source-specific executable compiler exists in this release. Continue with assessment, governed work, and an admitted adapter; Airlift does not invent executable output. Expected artifacts: * Capability plan and adapter requirements only Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. > This source is cataloged or assessable but has no source-specific executable compiler in the installed Airlift generation. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file salesforce-migration-plan.json --idempotency-key salesforce-plan-v1 fa conversion batch create --file salesforce-batch.json --idempotency-key salesforce-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file salesforce-transfer.json --idempotency-key salesforce-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file salesforce-validation.json --idempotency-key salesforce-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file salesforce-evidence-export.json --idempotency-key salesforce-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to Salesforce; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # SAP to Databricks # SAP to Databricks [#sap-to-databricks] Airlift treats SAP as an ERP migration, not a collection of anonymous tables. The source pack keeps business objects, CDS views and extractors, BW objects, process chains, authorizations, currencies, units, hierarchies, and downstream analytics in one governed scope. ## Generate the route [#generate-the-route] ```bash fa source inspect sap fa source plan sap --variant sap_s4hana fa source plan sap --variant sap_bdc --json > .airlift/sap-bdc-plan.json fa application-pack plan --file sap-bdc-manifest.json --json > sap-bdc-plan.json ``` Supported variants are `sap_bdc`, `sap_s4hana`, `sap_ecc`, `sap_bw4hana`, `sap_hana`, and `sap_datasphere`. Select the deployed system rather than using a generic label; the variant is stored with the governed estate. ## What Airlift adds [#what-airlift-adds] | Stage | Your SAP/Databricks adapter | Airlift record | | --------- | ---------------------------------------------------------------------------------- | --------------------------------------------------------------------------- | | Discover | Export business objects, CDS/extractor metadata, BW lineage, jobs, and consumers | Accepted inventory, dependencies, source version, exclusions, report digest | | Model | Map business keys, currencies, units, hierarchies, and history to target contracts | Migration objects, target dispositions, human residue, owners | | Ingest | Use SAP BDC sharing or an admitted SAP connector/extract route | Snapshot, delta cursor, deletions, restart checkpoint, lag | | Validate | Compare business keys, histories, relationships, and accepted business totals | Immutable provider runs and readiness observations | | Certify | Admit functional, data, security, consumer, and scale evidence | System-minted signed certificates | | Cut over | Change consumers through a certified client effector | Frozen wave, approvals, checkpoint, effect, verify or rollback refs | | Modernize | Build governed SAP data products | Separate release and independent evidence profile | Custom ABAP extractors, BW process-chain logic, authorization-dependent semantics, and variant-specific extensions stay visible as residue. Ingestion success is not treated as business-semantic proof. ## Register the estate [#register-the-estate] ```ts await runtime.invokeAction(AIRLIFT_ACTION_IDS.estateRegister, { ...authenticatedContext, idempotencyKey: 'sap-finance-estate-v1', params: { name: 'SAP finance', sourceSystem: 'sap', sourceVariant: 'sap_s4hana', environment: 'prod', owner: 'finance-data', connectionRef: 'uc-connection://sap-finance', }, }); ``` The SAP BDC business-semantic compiler is executable. The S/4HANA, ECC, BW/4HANA, HANA, and Datasphere profiles remain cataloged until their source-specific compiler and recurring evidence ship. An executable compiler is not live client proof; promote a capability claim only after its inventory, movement, validation, scale, and—when requested—cutover evidence passes the [source-pack contract](/docs/sources/source-packs). See [enterprise application-pack commands](/docs/cli/application-packs) for the SAP BDC manifest and governed registration flow. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe sap fa source recipe sap --variant sap_bdc --json > .airlift/sap-recipe.json ``` The App is engagement-aware. SAP appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect sap --json > .airlift/sap-profile.json fa source plan sap --variant sap_bdc --json > .airlift/sap-capability-plan.json ``` Expected artifacts: * .airlift/sap-profile.json * .airlift/sap-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file sap-estate.json --idempotency-key sap-estate-v1 fa connection register --file sap-connection.json --idempotency-key sap-connection-v1 fa engagement update --file sap-scope.json --idempotency-key sap-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file sap-assessment-start.json --idempotency-key sap-assessment-start-v1 fa assessment status --json fa assessment record --file sap-assessment-record.json --idempotency-key sap-assessment-record-v1 fa assessment accept --file sap-assessment-accept.json --idempotency-key sap-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/sap` after replacing the placeholder ID with the governed engagement ID. ### 3. Compile the enterprise application pack [#3-compile-the-enterprise-application-pack] ```bash fa application-pack inspect --file sap-manifest.json fa application-pack plan --file sap-manifest.json --json > generated/sap-plan.json ``` Expected artifacts: * Business-semantic target plan * Control totals * Security mappings * Consumer transitions Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file sap-migration-plan.json --idempotency-key sap-plan-v1 fa conversion batch create --file sap-batch.json --idempotency-key sap-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file sap-transfer.json --idempotency-key sap-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file sap-validation.json --idempotency-key sap-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file sap-evidence-export.json --idempotency-key sap-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to SAP; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # ServiceNow to Databricks # ServiceNow to Databricks [#servicenow-to-databricks] ```bash fa source inspect servicenow fa source plan servicenow --variant servicenow_platform ``` The source pack covers tables, dictionary metadata, references, choices, audit history, attachments, workflows, access controls, and reporting consumers. An admitted connector or table-API adapter performs extraction; Airlift records its immutable inventory, `sys_updated_on` cursor, deletion policy, pagination boundary, restart checkpoint, and freshness result. Airlift keeps dynamic or extended tables, ACL-filtered records, journal fields, large attachments, and custom scripted APIs visible until a developer or business owner dispositions them. Validation compares table and field counts, reference integrity, choice mappings, audit history, access behavior, ITSM metrics, and performance windows. After the baseline is certified, a separate release can create governed IT operations data products with Lakeflow Connect, Delta history, Unity Catalog, and Databricks SQL. The pack is `cataloged`; connector completion alone cannot promote its support level. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe servicenow fa source recipe servicenow --variant servicenow_platform --json > .airlift/servicenow-recipe.json ``` The App is engagement-aware. ServiceNow appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect servicenow --json > .airlift/servicenow-profile.json fa source plan servicenow --variant servicenow_platform --json > .airlift/servicenow-capability-plan.json ``` Expected artifacts: * .airlift/servicenow-profile.json * .airlift/servicenow-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file servicenow-estate.json --idempotency-key servicenow-estate-v1 fa connection register --file servicenow-connection.json --idempotency-key servicenow-connection-v1 fa engagement update --file servicenow-scope.json --idempotency-key servicenow-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file servicenow-assessment-start.json --idempotency-key servicenow-assessment-start-v1 fa assessment status --json fa assessment record --file servicenow-assessment-record.json --idempotency-key servicenow-assessment-record-v1 fa assessment accept --file servicenow-assessment-accept.json --idempotency-key servicenow-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/servicenow` after replacing the placeholder ID with the governed engagement ID. ### 3. Implement the admitted source adapter [#3-implement-the-admitted-source-adapter] > No source-specific executable compiler exists in this release. Continue with assessment, governed work, and an admitted adapter; Airlift does not invent executable output. Expected artifacts: * Capability plan and adapter requirements only Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. > This source is cataloged or assessable but has no source-specific executable compiler in the installed Airlift generation. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file servicenow-migration-plan.json --idempotency-key servicenow-plan-v1 fa conversion batch create --file servicenow-batch.json --idempotency-key servicenow-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file servicenow-transfer.json --idempotency-key servicenow-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file servicenow-validation.json --idempotency-key servicenow-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file servicenow-evidence-export.json --idempotency-key servicenow-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to ServiceNow; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # Snowflake # Migrate Snowflake with Airlift [#migrate-snowflake-with-airlift] The Snowflake profile covers SQL objects, Snowflake Scripting, tasks and streams, stages and pipes, dynamic tables, shares, dbt projects, external functions, and downstream consumers. Airlift uses one dependency graph so database objects are not certified while their task, stream, or consumer path is still missing. ## What Airlift adds to a Snowflake migration [#what-airlift-adds-to-a-snowflake-migration] | Stage | Databricks tool or project adapter | Airlift responsibility | | --------- | ------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | | discover | Lakebridge Profiler and Analyzer plus account, dbt, task, stream, and sharing exports | accept inventory, dependency graph, exclusions, source version, and report digests | | plan | assessment output informs target architecture | assign owners and dependency-aware waves; preserve account-level, external-function, and human-work decisions | | convert | Morpheus converts supported Snowflake SQL and assists dbt repointing | record attempts, tool generations, target artifacts, warnings, and residue lanes | | move data | bulk unload plus stream, timestamp, or application-watermark catch-up | track manifests, watermarks, lag, restart checkpoints, counts, and reconciliation | | validate | Lakebridge Reconcile plus semi-structured, task, stream, and business scenarios | admit independent evidence for the exact object and source/target snapshots | | certify | Airlift evaluates the active readiness profile | mint a signed certificate identifying artifacts, snapshots, evidence, and policy | | cut over | project connection, task, dbt, share, and consumer effectors | freeze scope, enforce approvals, checkpoint, apply once, verify, and retain rollback evidence | | modernize | Unity Catalog, Lakeflow, Delta, and Databricks SQL work | separate native redesign from the parity-preserving migration release | This prevents a successful table copy from hiding missing task ordering, stream consumption, grants, shares, dbt dependencies, or downstream connection changes. ## Compile the executable migration pack [#compile-the-executable-migration-pack] Create a versioned manifest containing SQL objects, tasks, streams, stages, pipes, dynamic tables, Snowpark and external-function references, shares, dbt metadata, permissions, and consumers. Mark observed hard cases on their owning objects, then run: ```bash fa migration-pack inspect --file snowflake-manifest.json fa migration-pack plan --file snowflake-manifest.json --json > snowflake-plan.json ``` The compiler validates dependencies, reports missing Snowflake hard cases, and preserves deterministic, bounded-repair, and human-only lanes. Its plan includes unload and stream/watermark restart requirements, semi-structured and task-order validation suites, target assets, and a Runway-owned deployment contract. See [migration-pack commands](/docs/cli/migration-packs) for registration and certification. ## Inspect and generate a plan [#inspect-and-generate-a-plan] ```bash fa source inspect snowflake fa source plan snowflake --json > .airlift/snowflake-plan.json ``` ## Profile and analyze [#profile-and-analyze] ```bash databricks labs lakebridge configure-database-profiler databricks labs lakebridge execute-database-profiler \ --source-tech snowflake \ --output-folder ./artifacts/profile databricks labs lakebridge analyze \ --source-directory ./source-export \ --source-tech "Snowflake" \ --report-file ./artifacts/snowflake-analysis.xlsx \ --generate-json true ``` Include account-level integrations, shares, network policies, role grants, task graphs, stream consumption, stage definitions, and dbt metadata in the Airlift inventory even when Lakebridge analyzes only the SQL-bearing artifacts. ## Convert [#convert] ```bash databricks labs lakebridge transpile \ --source-dialect snowflake \ --input-source ./source-export/sql \ --output-folder ./artifacts/converted ``` Morpheus is preferred and supports Snowflake SQL plus dbt repointing. Treat JavaScript procedures, Snowpark code, external functions, account-level objects, shares, and vendor-specific task behavior as explicit residue until a target implementation is recorded. ## Transfer and reconcile [#transfer-and-reconcile] Use bulk unload to the client's cloud object storage and a stream, timestamp, or application watermark catch-up. Federation through a Unity Catalog connection is useful for validation; it does not by itself prove migration or restart behavior. ```bash databricks labs lakebridge configure-reconcile databricks labs lakebridge auto-configure-recon-tables databricks labs lakebridge reconcile ``` Add validations for `VARIANT`, arrays and objects, timestamp/timezone behavior, identifier case, decimal/null semantics, task ordering, and stream consumption. Review auto-configured joins, mappings, filters, transformations, and thresholds before running. ## Certify and cut over [#certify-and-cut-over] Resolve the required readiness tracks for each table, procedure, task, stream, dbt model, share, and consumer before certificate minting. A project cutover effector should switch connections and schedules only after Airlift rechecks frozen scope, current certificates, and distinct approvals. Verify dbt runs, task ordering, stream progress, BI queries, and rollback independently before recording the wave complete. ## Modernize after parity [#modernize-after-parity] Disposition tasks and dynamic tables into Lakeflow, use Unity Catalog for grants and sharing, retarget dbt, and benchmark Databricks SQL serverless plus liquid clustering in a separate release. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe snowflake fa source recipe snowflake --variant snowflake --json > .airlift/snowflake-recipe.json ``` The App is engagement-aware. Snowflake appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect snowflake --json > .airlift/snowflake-profile.json fa source plan snowflake --variant snowflake --json > .airlift/snowflake-capability-plan.json ``` Expected artifacts: * .airlift/snowflake-profile.json * .airlift/snowflake-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file snowflake-estate.json --idempotency-key snowflake-estate-v1 fa connection register --file snowflake-connection.json --idempotency-key snowflake-connection-v1 fa engagement update --file snowflake-scope.json --idempotency-key snowflake-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file snowflake-assessment-start.json --idempotency-key snowflake-assessment-start-v1 fa assessment status --json fa assessment record --file snowflake-assessment-record.json --idempotency-key snowflake-assessment-record-v1 fa assessment accept --file snowflake-assessment-accept.json --idempotency-key snowflake-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/snowflake` after replacing the placeholder ID with the governed engagement ID. ### 3. Compile the snowflake migration pack [#3-compile-the-snowflake-migration-pack] ```bash fa migration-pack inspect --file snowflake-manifest.json fa migration-pack plan --file snowflake-manifest.json --json > generated/snowflake-plan.json ``` Expected artifacts: * Dependency-aware migration pack * Transfer requirements * Validation requirements * Residue lanes Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file snowflake-migration-plan.json --idempotency-key snowflake-plan-v1 fa conversion batch create --file snowflake-batch.json --idempotency-key snowflake-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file snowflake-transfer.json --idempotency-key snowflake-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file snowflake-validation.json --idempotency-key snowflake-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file snowflake-evidence-export.json --idempotency-key snowflake-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to Snowflake; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # Source-pack contract # Source-pack contract [#source-pack-contract] A source pack is a versioned developer contract. It tells your migration project what to inventory, which specialist adapter boundaries to implement, which governed Airlift actions record the work, and which evidence is required. It does not embed credentials, run a hidden connector, or claim that a catalog entry has been certified against your estate. ## Inspect the installed contract [#inspect-the-installed-contract] ```bash fa sources --json | jq '.[] | {id, archetype, evidenceSupportLevel, implementationRoutingLevel, variants}' fa source inspect dynamics_365 --json fa source constructs synapse --variant synapse_dedicated_sql --json fa source plan dynamics_365 \ --variant dynamics_365_finance_operations \ --json > .airlift/dynamics-finance-plan.json ``` The planner emits one archetype-aware schema. There is no legacy schema selector or alternate command spelling. `source constructs` reads the repository-owned normalization and routing catalog. It reports the canonical construct ID, artifact kind, disposition, target pattern, implementation status, constraints, and validation-profile references consumed by assessment and conversion. It is static implementation metadata—not a provider installation, execution, validation, or support claim. Use `fa capability matrix` for the authenticated evidence-derived registry. For ADF execution, only `native_generator` rows may enter the deterministic generated lane; `descriptor_only` and `routing_only` rows remain repair or human work even when the catalog can name a target shape. ## Archetypes [#archetypes] | Archetype | Typical lifecycle | | ---------------------- | ------------------------------------------------------------------------------------ | | `warehouse` | analyze, inventory, convert, transfer, validate, cut over, modernize | | `operational_database` | snapshot and CDC, code disposition, data parity, consumer cutover | | `erp` | business-object inventory, semantic mapping, governed ingestion, business validation | | `saas` | entity metadata, API/change cursor, relationship and deletion validation | | `etl_platform` | export pipelines, map semantics, implement, rehearse, switch schedules | | `mainframe` | encoding-aware inventory, unload/log capture, batch reconciliation, window control | | `query_engine` | catalog inventory, federate/register/materialize decision, SQL validation | | `streaming` | contract inventory, checkpoint bootstrap, parallel run, loss/duplicate/lag proof | ## Support levels [#support-levels] Evidence-derived support levels are ordered claims. A profile's `implementationRoutingLevel` is not one of these claims; it only describes the implementation workflow packaged with Airlift. * `cataloged` means the typed pack, variant registry, plan compiler, and documentation ship; it makes no execution or evidence claim. * `assessable` requires current hermetic proof for assessment and dependency lineage plus an explicit disposition for every construct in the independent required denominator. * `executable` requires every mandatory lifecycle capability and every required construct at `workspace_proven`; no required construct may be unavailable. * `certifiable` requires the same complete lifecycle and construct set at `client_proven`. * `cutover_certified` additionally requires `production_certified` cutover and rollback proof. See the [independent required-construct denominator](/docs/reference/capability-catalog#independent-required-construct-denominator) for the exact source-versioned counts. Missing contracts, cells, proof, or explicit dispositions always fail closed. Never infer a higher level from an upstream connector or converter being available. ## Adapter boundary [#adapter-boundary] Read `profile.adapter` before implementing a route: ```ts import { createSourceMigrationPlan, resolveSourceSystemProfile, } from '@fabricorg/airlift'; const profile = resolveSourceSystemProfile('sap_s4hana'); const plan = createSourceMigrationPlan(profile.id, { variant: 'sap_s4hana' }); console.log(profile.adapter.inventoryInputs); console.log(profile.adapter.movementOptions); console.log(plan.steps.flatMap((step) => step.airliftActions)); ``` Adapters return artifact, cursor, checkpoint, and run references. Store OAuth tokens, passwords, keys, and connection strings in the platform that owns them; Airlift accepts only opaque references. ## Check live evidence [#check-live-evidence] Scheduled certification jobs produce an immutable manifest. Check it locally or in CI: ```bash fa source certification-check source-certification.json --json AIRLIFT_LIVE_SOURCE_MANIFEST=source-certification.json pnpm test:e2e:live ``` The command exits non-zero when the requested level is missing required runs. It never changes the registry or mints a migration certificate. # SQL Server # Migrate SQL Server with Airlift [#migrate-sql-server-with-airlift] Airlift turns SQL Server assessment, conversion, transfer, validation, and cutover into one governed object ledger. Lakebridge supplies the profiler, Analyzer, Morpheus conversion, and Reconcile implementation; Airlift records which runs and evidence were accepted for each object. ## What Airlift adds to a SQL Server migration [#what-airlift-adds-to-a-sql-server-migration] | Stage | Databricks tool or project adapter | Airlift responsibility | | --------- | ------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | | discover | Lakebridge Profiler and Analyzer scan T-SQL, SSIS, and SSRS exports | accept inventory, dependencies, exclusions, source version, workload evidence, and report digests | | plan | assessment output informs target design | assign owners and dependency-aware waves; preserve CLR, linked-server, agent-job, and human-work decisions | | convert | Morpheus converts supported T-SQL | record attempts, tool generations, artifact digests, warnings, and residue lanes | | move data | snapshot plus change tracking, CDC, or application watermarks | track LSNs or watermarks, manifests, lag, restart checkpoints, counts, and reconciliation | | validate | Lakebridge Reconcile plus SSIS, SSRS, and business-query scenarios | admit independent object-specific evidence instead of treating converted SQL as proof | | certify | Airlift evaluates the active readiness profile | mint a signed certificate identifying the artifacts, snapshots, evidence, and policy used | | cut over | project endpoint, job, report, and connection effectors | freeze scope, enforce approvals, checkpoint, apply once, verify, and retain rollback evidence | | modernize | Unity Catalog, Lakeflow, Delta, and Databricks SQL work | keep native redesign in a separate release so baseline parity remains inspectable | This keeps tables, procedures, SQL Agent or SSIS orchestration, SSRS reports, and application consumers in one dependency-aware plan even when each surface uses a different implementation path. ## Choose the Databricks target [#choose-the-databricks-target] Use the standard migration pack when SQL Server analytical workloads are moving to the Databricks Lakehouse. Use the Lakebase target path when an operational application database needs PostgreSQL-compatible transactional serving on Databricks: ```bash fa lakebase inspect --file sql-server-manifest.json fa lakebase plan --file sql-server-manifest.json --json \ > .airlift/sql-server-lakebase-plan.json fa lakebase qualification-check \ --file .airlift/sql-server-lakebase-qualification.json ``` The Lakebase plan maps schemas, tables, views, functions, roles, and grants while keeping SQL Agent, SSIS, SSRS, and other external workload redesign visible as residue. Runway owns target deployment. Airlift binds the accepted source scope, target artifact, dataset, and validation evidence to one plan digest. See the [SQL Server to Lakebase CLI guide](/docs/cli/lakebase) for the complete repeatable test. For workspace proof, do not reuse the compatibility report. Supply a `source_connectivity` row from the connected SQL Server run and a `target_connectivity` row from managed Lakebase, both bound to the same build, plan, and dataset digests. Then admit the evaluated report with `fa artifact register`. The App shows it under **Engagement → Artifacts**. Registration alone leaves the fail-closed corroboration warning visible. An admitted validation provider must record a passing governed validation run whose `subjectArtifactId` names the registered certification row and whose `artifactDigest` equals that row's SHA-256 digest; only then does the App show **Connected workspace evidence**. Reusing another run's digest is not corroboration. Failed runs remain visible but never create that success label. Client and production boundaries remain unchanged. After registration, read the same governed phase projection shown in the App: ```bash fa engagement status fa engagement status --json ``` ## Compile the executable migration pack [#compile-the-executable-migration-pack] Create a versioned manifest containing SQL objects, SQL Agent jobs, SSIS packages, SSRS reports, linked-server references, cross-database dependencies, permissions, and consumers. Mark observed hard cases on the objects where they occur, then run: ```bash fa migration-pack inspect --file sql-server-manifest.json fa migration-pack plan --file sql-server-manifest.json --json > sql-server-plan.json ``` The compiler requires source IDs and complete dependencies, reports missing SQL Server hard-case coverage, and routes each object to deterministic conversion, bounded repair, or human remediation. It generates target mappings, the LSN/watermark transfer contract, SQL Server-specific Experiments suites, and a Runway deployment requirement. See [migration-pack commands](/docs/cli/migration-packs) for registration and certification. ## Inspect the developer plan [#inspect-the-developer-plan] ```bash fa source inspect sql_server fa source plan mssql fa source plan mssql --json > .airlift/sql-server-plan.json ``` Aliases include `mssql`, `sqlserver`, `azure_sql`, and `rds_sql_server`. ## Run assessment [#run-assessment] Export DDL, procedures, functions, SQL Agent jobs, SSIS packages, SSRS definitions, linked-server dependencies, roles, and representative query history. Then run: ```bash databricks labs lakebridge configure-database-profiler databricks labs lakebridge execute-database-profiler \ --source-tech mssql \ --output-folder ./artifacts/profile databricks labs lakebridge analyze \ --source-directory ./source-export \ --source-tech "MS SQL Server" \ --report-file ./artifacts/analysis.xlsx \ --generate-json true ``` Use additional Analyzer runs with `SSIS` and `SSRS` when those exports are in scope. Record the accepted report and digests through `assessmentStart`, `assessmentRecord`, and `objectRegister` actions. ## Convert code [#convert-code] ```bash databricks labs lakebridge transpile \ --source-dialect mssql \ --input-source ./source-export/sql \ --output-folder ./artifacts/converted ``` Morpheus is the preferred deterministic path. Route CLR objects, linked-server calls, cross-database transactions, SQL Agent side effects, and unresolved dynamic SQL into rework. Airlift may record one bounded repair candidate, but independent validation is still required. ## Move data [#move-data] Choose change tracking, SQL Server CDC, or an application watermark per table class. Implement the Airlift transfer contract as snapshot, incremental catch-up, restart, and reconcile steps. Preserve source LSN or watermark and target snapshot identities in the returned evidence. ## Validate [#validate] ```bash databricks labs lakebridge configure-reconcile databricks labs lakebridge auto-configure-recon-tables databricks labs lakebridge reconcile ``` Review generated table pairs before execution. Add scenarios for collation and case, datetime and decimal behavior, identity/sequence semantics, temporary tables, dynamic SQL, SSIS control flow, and SSRS result sets. Record admitted runs with `validationRunRecord` and `readinessRecord`; do not treat transpiler success as parity. ## Certify and cut over [#certify-and-cut-over] Assign profiles that match the object class, admit the required validation runs, and let Airlift mint the certificate from current evidence. For cutover, implement project effectors for connection strings, SQL Agent or SSIS schedules, report data sources, and application endpoints. Rehearse checkpoint, apply-once, verification, and rollback for each wave; a successful table comparison cannot authorize an untested consumer switch. ## Databricks modernization backlog [#databricks-modernization-backlog] After baseline certification, disposition SQL Agent and SSIS into Lakeflow Jobs or Declarative Pipelines, map database roles into Unity Catalog, move serving workloads to Databricks SQL, and evaluate liquid clustering. Keep these changes in a release separate from the parity-preserving migration. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe sql_server fa source recipe sql_server --variant sql_server --json > .airlift/sql_server-recipe.json ``` The App is engagement-aware. SQL Server appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect sql_server --json > .airlift/sql_server-profile.json fa source plan sql_server --variant sql_server --json > .airlift/sql_server-capability-plan.json ``` Expected artifacts: * .airlift/sql\_server-profile.json * .airlift/sql\_server-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file sql_server-estate.json --idempotency-key sql_server-estate-v1 fa connection register --file sql_server-connection.json --idempotency-key sql_server-connection-v1 fa engagement update --file sql_server-scope.json --idempotency-key sql_server-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file sql_server-assessment-start.json --idempotency-key sql_server-assessment-start-v1 fa assessment status --json fa assessment record --file sql_server-assessment-record.json --idempotency-key sql_server-assessment-record-v1 fa assessment accept --file sql_server-assessment-accept.json --idempotency-key sql_server-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/sql_server` after replacing the placeholder ID with the governed engagement ID. ### 3. Compile the sql\_server migration pack [#3-compile-the-sql_server-migration-pack] ```bash fa migration-pack inspect --file sql_server-manifest.json fa migration-pack plan --file sql_server-manifest.json --json > generated/sql_server-plan.json ``` Expected artifacts: * Dependency-aware migration pack * Transfer requirements * Validation requirements * Residue lanes Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file sql_server-migration-plan.json --idempotency-key sql_server-plan-v1 fa conversion batch create --file sql_server-batch.json --idempotency-key sql_server-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file sql_server-transfer.json --idempotency-key sql_server-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file sql_server-validation.json --idempotency-key sql_server-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file sql_server-evidence-export.json --idempotency-key sql_server-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. ### Authenticated hermetic run ledger [#authenticated-hermetic-run-ledger] In the isolated sandbox, open **Engagements → engagement → Run ledger**. Airlift renders **Start SQL Server journey** only from the engagement's canonical SQL Server estate; the browser submits organization and engagement identifiers, never provider identity, workflow generation, or candidate lineage. The server resolves one exact verified SQL Server source binding, planned transfer, requested deployment, and assigned validation scope before it starts or attaches to the deterministic Temporal workflow. Worker activities select the SQL Server hermetic provider from that server-resolved source, record its exact provider generation and private content-digested diagnostic references, and project lifecycle stages into shared PostgreSQL. Missing, duplicate, mixed-source, or unsupported provider composition fails closed. Existing Synapse workflow identities remain replay-compatible; SQL Server receives a distinct source-bound identity. This lane proves authenticated orchestration, provider isolation, durable lifecycle, exact validation continuation, cancellation cleanup, and source-bound idempotency only. It does **not** connect to live SQL Server, provision or inspect managed Lakebase, establish workspace/client proof, certify production readiness, or bypass the owner-bound workspace certification recipe below. These are automated captures from public synthetic engagements. The source workspace and run-ledger control are specific to SQL Server; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # Azure Synapse # Build a Synapse-to-Databricks migration [#build-a-synapse-to-databricks-migration] Airlift’s Synapse path combines a governed warehouse plan with a native ADF export importer and concrete Databricks file generator. The outputs are not a presentation checklist: they include dependency order, target mappings, materialized Workflow, Python, BDD, and Asset Bundle files, transfer proof requirements, explicit repair work, and immutable digests used by the migration ledger. Use it for dedicated SQL pools, serverless SQL pools, or mixed Synapse workspaces. The Lakebridge handles admitted SQL analysis and conversion. Airlift inventories data, external storage, workloads, permissions, pipelines, linked services, triggers, notebooks, and downstream consumers, then controls how implementation and evidence converge. ## What Airlift adds to a Synapse migration [#what-airlift-adds-to-a-synapse-migration] | Input | Airlift output | Where execution happens | | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------- | | normalized Synapse inventory | accepted object counts, dependency order, exclusions, dead assets, distributions, partitions, locations, and owners | Airlift assessment and inventory actions | | ADF/Synapse ARM or Git export | credential-free ETL IR plus concrete Workflow, Python, BDD, configuration, and Asset Bundle files | local or CI generation, then Runway release execution | | unsupported SQL or activities | classified agent-repairable or human-only residue with required skill | Harness-backed repair or the Airlift remediation workbench | | external table locations and dedicated-pool scope | snapshot plus incremental-catch-up contract and required restart/reconciliation proofs | admitted transfer adapter and durable worker workflow | | source object types | Unity Catalog, Delta, Lakeflow, Databricks SQL, notebook, grant, connection, and consumer-runbook target blueprint | immutable release submitted to Runway | | migration context | schema, row, aggregate, checksum, query, business, security, schedule, performance, cost, and Power BI suites | Fabric Experiments; evidence returns to Airlift | Airlift owns the migration ledger and the decisions that advance it. It does not copy a SQL transpiler, deployment engine, or test engine into the application. That boundary is what lets a developer replace an adapter without losing governance or historical evidence. | Stage | Specialist execution | What Airlift governs | | --------- | ---------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- | | discover | Lakebridge Profiler and Analyzer process Synapse SQL and ADF exports | accepted scope, versioned inventory, dependencies, exclusions, owners, and report digests | | plan | the accepted inventory drives Databricks target design | dependency-aware waves, economics, staffing, target blueprint, and human residue | | convert | Morpheus converts supported Synapse T-SQL and the ADF adapter generates admitted orchestration files | attempts, exact tool versions, immutable diffs, warnings, and residue disposition | | move data | admitted adapters execute snapshot and incremental catch-up | watermarks, manifests, lag, restart checkpoints, rejects, and reconciliation | | validate | Experiments executes source/target scenarios | independent object-specific evidence returned by reference and digest | | certify | Airlift evaluates the assigned evidence profile | a system-minted signed certificate bound to artifacts, snapshots, evidence, and policy | | cut over | the admitted effector changes client endpoints and schedules | frozen scope, separation of duties, checkpoint, apply once, verify, compensation, and rollback | | modernize | engineers adopt Unity Catalog, Delta, Lakeflow, and Databricks SQL | a separate release with its own measurements and acceptance evidence | ## 1. Prepare the manifest [#1-prepare-the-manifest] Create `synapse-manifest.json`. Do not put credentials, tokens, or connection strings in this file. ```json { "schemaVersion": 1, "estate": { "name": "Finance warehouse", "variant": "synapse_dedicated_sql", "synapseVersion": "10.0", "sourceSnapshot": "2030-01-15T12:00:00.000Z" }, "inventory": [ { "sourceId": "schema:finance", "name": "finance", "kind": "schema", "sourcePath": "finance", "owner": "finance-data" }, { "sourceId": "table:finance/ledger", "name": "ledger", "kind": "table", "sourcePath": "finance.ledger", "owner": "finance-data", "dependencies": ["schema:finance"], "distribution": "hash", "distributionColumns": ["account_id"], "partitionColumns": ["posting_date"] } ], "adfExport": { "resources": [] } } ``` Supported inventory kinds are `schema`, `table`, `view`, `stored_procedure`, `function`, `external_table`, `copy_statement`, `polybase_object`, `workload_group`, `permission`, `pipeline`, `linked_service`, `trigger`, `notebook`, and `consumer`. Mark retired objects with `dead: true`. Use `excludedReason` for an explicit scope decision. Add `unsupportedReasons` to route known dynamic SQL, cross-database behavior, side effects, or ambiguous distribution logic into remediation. ## 2. Inspect before creating work [#2-inspect-before-creating-work] ```bash fa source inspect synapse fa synapse inspect --file synapse-manifest.json ``` The command validates references, rejects duplicate IDs and missing dependencies, and reports in-scope inventory, ETL task count, residue count, human-residue count, and the prospective bundle digest. A dependency cycle fails the command instead of silently inventing an execution order. To preview task routing without materializing files: ```bash fa synapse compile-adf --file TemplateForWorkspace.json --json ``` That command returns execution-neutral task mappings only. To import a native export and generate actual file bodies, use the migration IR workflow: ```bash fa migration-ir import --source adf-synapse \ --file TemplateForWorkspace.json \ --estate-name "Finance pipelines" \ --snapshot-at 2030-01-15T12:00:00Z \ --output migration-ir.json fa migration-ir generate --file migration-ir.json --out-dir generated fa migration-ir validate --file generated/artifact-set.json --root generated ``` Linked services become credential-binding requirements; credential properties are never copied into IR. Unknown activities remain source-preserved remediation rather than being silently dropped. ## 3. Generate and register the bundle [#3-generate-and-register-the-bundle] ```bash fa synapse plan --file synapse-manifest.json --json > synapse-plan.json ``` Store `synapse-plan.json` in your admitted immutable artifact store. Then register its reference with the active engagement and Synapse estate: ```bash fa synapse register \ --file synapse-plan.json \ --engagement-id eng_01ARZ3NDEKTSV4RRFFQ69G5FAV \ --estate-id est_01ARZ3NDEKTSV4RRFFQ69G5FAV \ --artifact-id volumes/migration/plans/synapse-plan.json \ --idempotency-key finance-synapse-plan-v1 ``` `register` invokes `airlift.artifact_register`; it does not upload the file. The API derives organization and actor identity from the authenticated principal. The artifact reference and the bundle share the same digest, so replacing the referenced content is detectable. ## 4. Execute the factory [#4-execute-the-factory] Before accepting generated workbench artifacts, inspect the exact repository route: ```bash fa source constructs synapse \ --variant synapse_dedicated_sql \ --construct adf.copy \ --artifact-kind target_code ``` The construct catalog is the shared source for Synapse/ADF assessment lanes, Migration IR routing, and v2 artifact admission. A new artifact whose declared construct, disposition, or target disagrees with the exact catalog row is rejected without an event. Historical rows remain replayable and visible, but the App marks their route as diverged or unrecognized so an operator can regenerate or review them. Catalog alignment is implementation metadata; it does not create a capability claim or prove workspace behavior. Open **Active sources → Azure Synapse** inside the active engagement, or use the corresponding CLI surfaces: ```bash fa assessment list --estate-id fa plan list --engagement-id fa conversion batch list --engagement-id fa residue list --engagement-id fa transfer list --estate-id fa deployment list --estate-id fa validation list --estate-id fa cutover list --estate-id fa modernization list --engagement-id fa engagement status fa engagement status --json ``` The engagement status view shows **Azure Synapse → Databricks Lakehouse**, all eight delivery phases, the number of governed objects at each stage, open human residue, validation blockers, active certificates, and the next action. It derives these values from admitted records; developers do not edit the percentage. The generated bundle tells these stages what must exist; the governed actions record what actually happened. Converter success never certifies an object. Deployment success never proves parity. A certificate is minted only after the current validation profile has admitted evidence for every required track. ## Target and movement behavior [#target-and-movement-behavior] Airlift proposes Unity Catalog schemas and grants, Delta tables, external locations, Lakeflow pipelines and jobs, Databricks SQL assets, notebooks, connection bindings, validation assets, and Databricks Asset Bundle resources. Runway owns deployment, promotion, and rollback and returns immutable release references to Airlift. The Synapse transfer profile is snapshot plus incremental catch-up. Before completion, the project must record stable snapshot identity, source watermarks, deletes, late data, schema drift, restart checkpoints, bounded concurrency, throughput, rejects, and reconciliation. These are executable evidence requirements, not prose reminders. ## Human remediation is part of the product [#human-remediation-is-part-of-the-product] Cross-database transactions, external side effects, custom activities, and ambiguous business semantics are intentionally visible. Airlift creates priced, assignable residue instead of hiding that work inside an automation percentage. A specialist can repair the artifact, record independent validation, and submit it for human review without bypassing the certificate policy. ## Security properties [#security-properties] * source exports contain metadata and code, never live credentials; * linked services are represented as opaque connection-binding requirements; * every remote mutation uses authenticated identity, tenant scope, policy, audit, and a stable idempotency key; * Airlift references Experiments and Runway results by ID and digest and never re-owns their state; * cutover remains blocked until frozen scope, approvals, fresh evidence, and effector certification all pass. See [Synapse CLI commands](/docs/cli/synapse) for command and exit semantics. ## Read a development journey [#read-a-development-journey] When a team exercises the Synapse factory before production reviewers and effectors are available, Airlift records the result as **Development evidence**. Use the CLI and App to inspect what actually passed: ```bash fa engagement status fa validation list --engagement-id fa validation runs --object-id fa certificate list --object-id fa transfer list --engagement-id fa cutover status ``` In the App, open the engagement’s **Migration status**, **Artifacts**, and **Run ledger**; then open **Assurance center** and **Cutover control**. A complete development exercise should show converted and development-certified objects, a completed validation execution, a reconciled transfer, and a rehearsed wave. It should also show that production is still blocked. See [Development assurance](/docs/operations/development-assurance) for the exact boundary and expected UI state. ## Prove generation without a source account [#prove-generation-without-a-source-account] Run `import`, `generate`, and `validate` against a credential-free ARM or Git export. This proves parsing, dependency preservation, file materialization, and digest integrity. It is account-independent, so the maximum result is `hermetic_proven`; it does not prove source connectivity or Databricks behavior. Add live workspace execution, Experiments verdicts, a Runway release, transfer restart evidence, and current provider generations before promoting the relevant capability cells to `workspace_proven`. Representative client behavior and production cutover remain separate evidence levels. See [Capability certification](/docs/sources/certification) for the proof model. ### Run the authenticated sandbox journey [#run-the-authenticated-sandbox-journey] The isolated sandbox can execute the same account-independent provider through the real Temporal parent workflow and show durable progress in the engagement **Run ledger**. An administrator must first prepare one exact governed candidate in the shared PostgreSQL ledger: * one active engagement bound to one Synapse estate; * one verified Synapse source connection; * one planned transfer whose non-empty object scope belongs to that estate; * one requested deployment whose artifacts cover exactly the transfer objects; * assigned validation profiles for those objects. Open **Engagements → engagement → Run ledger** and select **Start Synapse journey**. The server accepts only organization and engagement from the form, resolves the provider IDs from canonical projections, verifies the signed-in natural person and governed transfer permission, and starts or attaches to a request-bound workflow on `airlift-sandbox-v1`. An exact retry of the same browser request collapses to the open workflow; a later retry after terminal failure receives a fresh workflow and run-ledger row for the same governed scope. Temporal activities record `discovering`, `assessing`, `converting`, `transferring`, `deploying`, validation, and terminal state through Platform Host. Refreshing or reconnecting reads the same replayable PostgreSQL projection. After deployment the journey pauses at `awaiting_validation_request`. This is deliberate: the worker does not manufacture operator authority. Create the exact governed validation request in **Validation**, then wake the existing workflow through the supported validation control. The hermetic provider cannot mint business acceptance, production certification, or a client cutover decision. The browser lane has the same permanent maximum result as the CLI lane: `hermetic_proven`. Seeing a completed workflow in the sandbox is not live Synapse or Databricks workspace evidence. ## Certify and cut over [#certify-and-cut-over] Use object-type-specific profiles for tables, SQL routines, pipelines, permissions, and consumers. Airlift mints signed certificates only from current admitted validation evidence. Cutover then rechecks the frozen wave, certificates, approvals, deployment digests, operational window, and certified effector before it permits an apply-once effect. Failure and uncertainty retain a governed rollback or reconciliation path. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe synapse fa source recipe synapse --variant synapse_dedicated_sql --json > .airlift/synapse-recipe.json ``` The App is engagement-aware. Azure Synapse appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect synapse --json > .airlift/synapse-profile.json fa source plan synapse --variant synapse_dedicated_sql --json > .airlift/synapse-capability-plan.json ``` Expected artifacts: * .airlift/synapse-profile.json * .airlift/synapse-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file synapse-estate.json --idempotency-key synapse-estate-v1 fa connection register --file synapse-connection.json --idempotency-key synapse-connection-v1 fa engagement update --file synapse-scope.json --idempotency-key synapse-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file synapse-assessment-start.json --idempotency-key synapse-assessment-start-v1 fa assessment status --json fa assessment record --file synapse-assessment-record.json --idempotency-key synapse-assessment-record-v1 fa assessment accept --file synapse-assessment-accept.json --idempotency-key synapse-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/synapse` after replacing the placeholder ID with the governed engagement ID. ### 3. Compile Synapse and ADF artifacts [#3-compile-synapse-and-adf-artifacts] ```bash fa synapse inspect --file synapse-manifest.json fa synapse plan --file synapse-manifest.json --json > generated/synapse-plan.json fa migration-ir import --source adf-synapse --file TemplateForWorkspace.json --estate-name "Source pipelines" --snapshot-at --output generated/migration-ir.json fa migration-ir generate --file generated/migration-ir.json --out-dir generated fa migration-ir validate --file generated/artifact-set.json --root generated ``` Expected artifacts: * Synapse plan * Databricks Asset Bundle * Workflow/Python files * BDD specifications * Artifact set digest Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file synapse-migration-plan.json --idempotency-key synapse-plan-v1 fa conversion batch create --file synapse-batch.json --idempotency-key synapse-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file synapse-transfer.json --idempotency-key synapse-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file synapse-validation.json --idempotency-key synapse-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file synapse-evidence-export.json --idempotency-key synapse-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to Azure Synapse; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # Teradata # Migrate Teradata with Airlift [#migrate-teradata-with-airlift] The Teradata profile covers SQL, macros, stored procedures, BTEQ, FastLoad, MultiLoad, TPT, volatile tables, workload management, security, and downstream consumers. Airlift keeps utility and orchestration objects visible so a converted SQL percentage cannot hide the work required to run the target estate. ## What Airlift adds to a Teradata migration [#what-airlift-adds-to-a-teradata-migration] | Stage | Databricks tool or project adapter | Airlift responsibility | | --------- | ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------ | | discover | Lakebridge Profiler and Analyzer scan SQL plus BTEQ and utility exports | accept inventory, dependencies, exclusions, profiler variant, source version, and report digests | | plan | assessment and unload benchmarks inform target architecture | assign owners and dependency-aware waves; preserve utility, workload, procedural, and human-work decisions | | convert | BladeBridge converts supported Teradata SQL | record attempts, tool generations, artifact digests, warnings, and residue lanes | | move data | utility-backed unload plus a source-specific incremental strategy | track manifests, watermarks, throughput, lag, restart checkpoints, rejected rows, counts, and reconciliation | | validate | hash-aware Lakebridge Reconcile plus utility, workload, and business scenarios | admit independent evidence with the exact hash configuration, source watermark, and target snapshot | | certify | Airlift evaluates the active readiness profile | mint a signed certificate identifying artifacts, snapshots, evidence, hash policy, and validation profile | | cut over | project utility, BTEQ, schedule, application, and BI effectors | freeze scope, enforce approvals, checkpoint, apply once, verify, and retain rollback evidence | | modernize | Unity Catalog, Lakeflow, Delta, liquid clustering, and Databricks SQL work | separate appliance redesign from baseline parity certification | This makes unload throughput, utility replacement, BTEQ control flow, workload behavior, and downstream consumers part of readiness instead of hiding them behind converted SQL. ## Compile the executable migration pack [#compile-the-executable-migration-pack] Export SQL, macros, procedures, BTEQ scripts, utility jobs, workload rules, grants, and consumers into a credential-free manifest. Select `teradata_vantage` or `teradata_appliance`, record the observed hard cases, then compile it: ```bash fa migration-pack inspect --file teradata-manifest.json fa migration-pack plan --file teradata-manifest.json --json > teradata-plan.json fa migration-pack register --file teradata-plan.json \ --engagement-id --estate-id \ --artifact-id --idempotency-key ``` The compiler preserves BTEQ and utility dependencies, routes proprietary behavior to an explicit remediation owner, and emits utility restart, watermark, reconciliation, validation, target, and deployment requirements. See [migration-pack commands](/docs/cli/migration-packs) for the complete contract. ## Inspect and generate a plan [#inspect-and-generate-a-plan] ```bash fa source inspect teradata fa source plan td --json > .airlift/teradata-plan.json ``` ## Profile and analyze [#profile-and-analyze] Choose the `core` profiler variant when PDCR is unavailable; use `pdcr` only when the client has the required environment and access. ```bash databricks labs lakebridge configure-database-profiler databricks labs lakebridge test-profiler-connection --source-tech teradata databricks labs lakebridge execute-database-profiler \ --source-tech teradata \ --variant core \ --output-folder ./artifacts/profile databricks labs lakebridge analyze \ --source-directory ./source-export \ --source-tech "Teradata" \ --report-file ./artifacts/teradata-analysis.xlsx \ --generate-json true ``` ## Convert [#convert] ```bash databricks labs lakebridge transpile \ --source-dialect teradata \ --input-source ./source-export/sql \ --output-folder ./artifacts/converted ``` BladeBridge is the preferred deterministic path. BTEQ control flow, load utilities, volatile-table assumptions, query bands, workload rules, and proprietary procedural logic commonly require explicit target implementations. ## Transfer and reconcile [#transfer-and-reconcile] Benchmark utility-backed unload throughput before promising a cutover window. Return restart checkpoints, file manifests, source watermarks, lag, counts, and rejected rows through the transfer adapter. ```bash databricks labs lakebridge configure-reconcile databricks labs lakebridge auto-configure-recon-tables databricks labs lakebridge reconcile ``` Teradata has no portable cryptographic hash in pure SQL. Lakebridge `row`, `data`, and `all` reports require a source hash UDF and `hash_expression_overrides.source` in the reconcile configuration. Without that setup, use schema reconciliation plus admitted Experiments parity scenarios; do not claim full row/data reconciliation. Validate SET/MULTISET behavior, primary-index assumptions, `QUALIFY`, ordered analytics, format/character semantics, utility restart, and workload-sensitive performance. ## Certify and cut over [#certify-and-cut-over] Do not resolve full data parity from a row/data report unless the required source hash UDF and override configuration were used and recorded. Assign separate profiles to BTEQ, utilities, procedures, workload behavior, and consumers. The project effector switches loads, schedules, queries, applications, and BI connections only after fresh certificates, approved scope, measured transfer completion, and rollback readiness are confirmed. ## Modernize after parity [#modernize-after-parity] Replace BTEQ and appliance utilities with Lakeflow or Databricks jobs, use Delta and liquid clustering based on measured workloads, and map security into Unity Catalog under a new release profile. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe teradata fa source recipe teradata --variant teradata_vantage --json > .airlift/teradata-recipe.json ``` The App is engagement-aware. Teradata appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect teradata --json > .airlift/teradata-profile.json fa source plan teradata --variant teradata_vantage --json > .airlift/teradata-capability-plan.json ``` Expected artifacts: * .airlift/teradata-profile.json * .airlift/teradata-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file teradata-estate.json --idempotency-key teradata-estate-v1 fa connection register --file teradata-connection.json --idempotency-key teradata-connection-v1 fa engagement update --file teradata-scope.json --idempotency-key teradata-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file teradata-assessment-start.json --idempotency-key teradata-assessment-start-v1 fa assessment status --json fa assessment record --file teradata-assessment-record.json --idempotency-key teradata-assessment-record-v1 fa assessment accept --file teradata-assessment-accept.json --idempotency-key teradata-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/teradata` after replacing the placeholder ID with the governed engagement ID. ### 3. Compile the teradata migration pack [#3-compile-the-teradata-migration-pack] ```bash fa migration-pack inspect --file teradata-manifest.json fa migration-pack plan --file teradata-manifest.json --json > generated/teradata-plan.json ``` Expected artifacts: * Dependency-aware migration pack * Transfer requirements * Validation requirements * Residue lanes Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file teradata-migration-plan.json --idempotency-key teradata-plan-v1 fa conversion batch create --file teradata-batch.json --idempotency-key teradata-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file teradata-transfer.json --idempotency-key teradata-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file teradata-validation.json --idempotency-key teradata-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file teradata-evidence-export.json --idempotency-key teradata-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to Teradata; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # Trino and Starburst to Databricks # Trino and Starburst to Databricks [#trino-and-starburst-to-databricks] ```bash fa source inspect trino fa source plan trino --variant trino fa source plan trino --variant starburst --json ``` Inventory catalogs, schemas, views, queries, connector configuration, custom functions, session properties, access controls, schedulers, and consumers. For every backing source, choose federation, registration, copy/ingestion, Delta materialization, SQL conversion, or a human-owned route. Airlift adds a single accepted inventory and dependency graph across those routes, versioned artifacts and residue, source and target snapshot references, independent semantic and business validation, and dependency-aware consumer waves. It does not copy connector credentials or claim a generic data-transfer route for federated catalogs. Validate connector-specific types, federated joins, functions, access behavior, business queries, and accepted performance thresholds. Certify from immutable provider runs, then switch consumers with authenticated approvals and verifiable rollback. The pack is `cataloged`; Trino and Starburst live routes must be certified separately. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe trino fa source recipe trino --variant trino --json > .airlift/trino-recipe.json ``` The App is engagement-aware. Trino appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect trino --json > .airlift/trino-profile.json fa source plan trino --variant trino --json > .airlift/trino-capability-plan.json ``` Expected artifacts: * .airlift/trino-profile.json * .airlift/trino-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file trino-estate.json --idempotency-key trino-estate-v1 fa connection register --file trino-connection.json --idempotency-key trino-connection-v1 fa engagement update --file trino-scope.json --idempotency-key trino-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file trino-assessment-start.json --idempotency-key trino-assessment-start-v1 fa assessment status --json fa assessment record --file trino-assessment-record.json --idempotency-key trino-assessment-record-v1 fa assessment accept --file trino-assessment-accept.json --idempotency-key trino-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/trino` after replacing the placeholder ID with the governed engagement ID. ### 3. Implement the admitted source adapter [#3-implement-the-admitted-source-adapter] > No source-specific executable compiler exists in this release. Continue with assessment, governed work, and an admitted adapter; Airlift does not invent executable output. Expected artifacts: * Capability plan and adapter requirements only Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. > This source is cataloged or assessable but has no source-specific executable compiler in the installed Airlift generation. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file trino-migration-plan.json --idempotency-key trino-plan-v1 fa conversion batch create --file trino-batch.json --idempotency-key trino-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file trino-transfer.json --idempotency-key trino-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file trino-validation.json --idempotency-key trino-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file trino-evidence-export.json --idempotency-key trino-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to Trino; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # Vertica to Databricks # Vertica to Databricks [#vertica-to-databricks] ```bash fa source inspect vertica fa source plan vertica ``` The Vertica pack inventories SQL objects, projections, external tables, UDx extensions, load and export jobs, resource pools, and consumers. It records deterministic conversion attempts where available and keeps projection design, UDx code, resource policies, COPY extensions, and epoch-specific behavior in owned residue. Implement movement as a parallel, restartable export with file manifests and a stable epoch or watermark for catch-up. Validate query results, numeric and timestamp behavior, external formats, projection-derived semantics, business scenarios, and representative concurrency independently from conversion. Airlift freezes the certified object and consumer scope before cutover, enforces authenticated approvals, and records checkpoint, apply-once, verification, or rollback. Modernize to Delta, liquid clustering, Lakeflow, Unity Catalog, and Databricks SQL only after retaining the parity-preserving baseline. The pack is `cataloged` pending live route evidence. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe vertica fa source recipe vertica --variant vertica --json > .airlift/vertica-recipe.json ``` The App is engagement-aware. Vertica appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect vertica --json > .airlift/vertica-profile.json fa source plan vertica --variant vertica --json > .airlift/vertica-capability-plan.json ``` Expected artifacts: * .airlift/vertica-profile.json * .airlift/vertica-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file vertica-estate.json --idempotency-key vertica-estate-v1 fa connection register --file vertica-connection.json --idempotency-key vertica-connection-v1 fa engagement update --file vertica-scope.json --idempotency-key vertica-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file vertica-assessment-start.json --idempotency-key vertica-assessment-start-v1 fa assessment status --json fa assessment record --file vertica-assessment-record.json --idempotency-key vertica-assessment-record-v1 fa assessment accept --file vertica-assessment-accept.json --idempotency-key vertica-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/vertica` after replacing the placeholder ID with the governed engagement ID. ### 3. Implement the admitted source adapter [#3-implement-the-admitted-source-adapter] > No source-specific executable compiler exists in this release. Continue with assessment, governed work, and an admitted adapter; Airlift does not invent executable output. Expected artifacts: * Capability plan and adapter requirements only Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. > This source is cataloged or assessable but has no source-specific executable compiler in the installed Airlift generation. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file vertica-migration-plan.json --idempotency-key vertica-plan-v1 fa conversion batch create --file vertica-batch.json --idempotency-key vertica-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file vertica-transfer.json --idempotency-key vertica-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file vertica-validation.json --idempotency-key vertica-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file vertica-evidence-export.json --idempotency-key vertica-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to Vertica; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */} # Workday to Databricks # Workday to Databricks [#workday-to-databricks] ```bash fa source inspect workday fa source plan workday --variant workday_hcm fa source plan workday --variant workday_reports --json ``` Airlift inventories Workday business objects, custom reports and RaaS, EIB integrations, effective-dated history, security domains, calculated fields, and downstream consumers. The ingestion adapter uses approved reports, APIs, or managed routes and returns pagination, watermark, correction, deletion, restart, and freshness evidence. ## Developer acceptance path [#developer-acceptance-path] * Map business-object identifiers, references, effective dates, and security-filter behavior to target contracts. * Register custom reports, calculated fields, Studio integrations, attachments, and effective-dated correction behavior as explicit objects or residue. * Validate object counts, history completeness, reference integrity, access-filter behavior, and accepted HR or finance totals independently. * Mint certificates only from admitted provider runs; cut over reports and consumers in a frozen, approved wave. * Build governed workforce data products, Delta history, Unity Catalog policies, Lakeflow jobs, and Databricks SQL models as a separate modernization release. The pack is `cataloged`. Credentials remain in the connector platform; Airlift stores only an opaque reference. {/* GENERATED:SOURCE-RECIPE:START */} ## Complete developer command sequence [#complete-developer-command-sequence] Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync: ```bash fa source recipe workday fa source recipe workday --variant workday_hcm --json > .airlift/workday-recipe.json ``` The App is engagement-aware. Workday appears under **Active sources** only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages. Every remote mutation below requires `--host`, `--org`, authenticated workspace identity, and a stable `--idempotency-key`. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run `fa --help` for the current schema and exit semantics. ### 0. Inspect the source contract [#0-inspect-the-source-contract] ```bash fa source inspect workday --json > .airlift/workday-profile.json fa source plan workday --variant workday_hcm --json > .airlift/workday-capability-plan.json ``` Expected artifacts: * .airlift/workday-profile.json * .airlift/workday-capability-plan.json Open **Engagements → active engagement** in the App. This stage is visible at `/engagements` after replacing the placeholder ID with the governed engagement ID. ### 1. Create governed scope and connection references [#1-create-governed-scope-and-connection-references] ```bash fa engagement create --file engagement.json --idempotency-key migration-create-v1 fa estate register --file workday-estate.json --idempotency-key workday-estate-v1 fa connection register --file workday-connection.json --idempotency-key workday-connection-v1 fa engagement update --file workday-scope.json --idempotency-key workday-scope-v1 fa engagement preflight ``` Expected artifacts: * Governed engagement * Source estate * Opaque connection binding Open **Engagements → active engagement** in the App. This stage is visible at `/engagements/` after replacing the placeholder ID with the governed engagement ID. ### 2. Assess and accept inventory [#2-assess-and-accept-inventory] ```bash fa assessment start --file workday-assessment-start.json --idempotency-key workday-assessment-start-v1 fa assessment status --json fa assessment record --file workday-assessment-record.json --idempotency-key workday-assessment-record-v1 fa assessment accept --file workday-assessment-accept.json --idempotency-key workday-assessment-accept-v1 fa inventory list --estate-id --json ``` Expected artifacts: * Assessment report reference * Normalized inventory * Dependency graph Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//sources/workday` after replacing the placeholder ID with the governed engagement ID. ### 3. Implement the admitted source adapter [#3-implement-the-admitted-source-adapter] > No source-specific executable compiler exists in this release. Continue with assessment, governed work, and an admitted adapter; Airlift does not invent executable output. Expected artifacts: * Capability plan and adapter requirements only Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//artifacts` after replacing the placeholder ID with the governed engagement ID. > This source is cataloged or assessable but has no source-specific executable compiler in the installed Airlift generation. ### 4. Convert, move, and remediate [#4-convert-move-and-remediate] ```bash fa plan generate --file workday-migration-plan.json --idempotency-key workday-plan-v1 fa conversion batch create --file workday-batch.json --idempotency-key workday-batch-v1 fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1 fa residue list --engagement-id fa transfer plan --file workday-transfer.json --idempotency-key workday-transfer-v1 fa transfer run --idempotency-key transfer-run-v1 fa transfer reconcile --idempotency-key transfer-reconcile-v1 ``` Expected artifacts: * Target artifacts * Residue cases * Transfer checkpoints * Reconciliation evidence Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 5. Validate independently and inspect discrepancies [#5-validate-independently-and-inspect-discrepancies] ```bash fa validation run --file workday-validation.json --idempotency-key workday-validation-v1 fa validation status --json fa discrepancy list --engagement-id fa artifact list --engagement-id ``` Expected artifacts: * Provider run references * Readiness evidence * Discrepancies Open **Engagements → active engagement** in the App. This stage is visible at `/engagements//runs` after replacing the placeholder ID with the governed engagement ID. ### 6. Certify, cut over, and export evidence [#6-certify-cut-over-and-export-evidence] ```bash fa certificate list --object-id fa cutover status --json fa evidence list --engagement-id fa evidence export --file workday-evidence-export.json --idempotency-key workday-evidence-export-v1 ``` Expected artifacts: * Migration certificates * Cutover evidence * Content-digested evidence export Open **Engagements → active engagement** in the App. This stage is visible at `/assurance` after replacing the placeholder ID with the governed engagement ID. > Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy. ## What developers see in the App [#what-developers-see-in-the-app] The contextual source workspace shows the accepted estate and the factory stages for this engagement. **Artifacts** displays immutable references, content digests, media types, and provider lineage. **Runs** displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict. These are automated captures from public synthetic engagements. The source workspace is specific to Workday; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images. {/* GENERATED:SOURCE-RECIPE:END */}