Databricks-native modernization

Airlift is also useful after a customer has adopted Databricks. A native engagement records the existing estate as the baseline and governs a separate modernization release with independent evidence, immutable promotion and rollback, and measured value.

fa source inspect databricks
fa source plan databricks --variant databricks_unity_catalog
fa databricks-native inspect --file native-manifest.json
fa databricks-native plan --file native-manifest.json --json > native-plan.json

What Airlift inventories

• workspaces, metastores, Hive Metastore objects, and Unity Catalog assets;
• jobs, pipelines, notebooks, warehouses, clusters, and policies;
• permissions, shares, lineage, and external paths;
• dashboards, models, endpoints, and downstream consumers;
• cost, performance, reliability, and operational dependencies.

What Airlift helps change

• Unity Catalog upgrades and workspace consolidation;
• permission, identity, and path modernization;
• serverless and Lakeflow adoption;
• Delta layout and liquid clustering;
• runtime and dependency upgrades;
• dashboard and consumer transitions; and
• cost and performance optimization.

The workflow is baseline → design → implement → validate → release → measure. The baseline remains immutable. Each change gets a typed recommendation and evidence profile. A promoted release needs Runway identity, Experiments verdicts, operational evidence, acceptance, and a rollback path. External productivity claims require a reviewed Airlift value report.

See Databricks-native commands for the exact manifest and CLI contract.

Complete developer command sequence

Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync:

fa source recipe databricks
fa source recipe databricks --variant databricks_unity_catalog --json > .airlift/databricks-recipe.json

The App is engagement-aware. Databricks native estate appears under Active sources only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages.

Every remote mutation below requires --host, --org, authenticated workspace identity, and a stable --idempotency-key. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run fa <resource> <operation> --help for the current schema and exit semantics.

0. Inspect the source contract

fa source inspect databricks --json > .airlift/databricks-profile.json
fa source plan databricks --variant databricks_unity_catalog --json > .airlift/databricks-capability-plan.json

Expected artifacts:

• .airlift/databricks-profile.json
• .airlift/databricks-capability-plan.json

Open Engagements → active engagement in the App. This stage is visible at /engagements after replacing the placeholder ID with the governed engagement ID.

1. Create governed scope and connection references

fa engagement create --file engagement.json --idempotency-key migration-create-v1
fa estate register --file databricks-estate.json --idempotency-key databricks-estate-v1
fa connection register --file databricks-connection.json --idempotency-key databricks-connection-v1
fa engagement update --file databricks-scope.json --idempotency-key databricks-scope-v1
fa engagement preflight <engagement-id>

Expected artifacts:

• Governed engagement
• Source estate
• Opaque connection binding

Open Engagements → active engagement in the App. This stage is visible at /engagements/<engagement-id> after replacing the placeholder ID with the governed engagement ID.

2. Assess and accept inventory

fa assessment start --file databricks-assessment-start.json --idempotency-key databricks-assessment-start-v1
fa assessment status <assessment-id> --json
fa assessment record --file databricks-assessment-record.json --idempotency-key databricks-assessment-record-v1
fa assessment accept --file databricks-assessment-accept.json --idempotency-key databricks-assessment-accept-v1
fa inventory list --estate-id <estate-id> --json

Expected artifacts:

• Assessment report reference
• Normalized inventory
• Dependency graph

Open Engagements → active engagement in the App. This stage is visible at /engagements/<engagement-id>/sources/databricks after replacing the placeholder ID with the governed engagement ID.

3. Compile the Databricks-native modernization plan

fa databricks-native inspect --file databricks-manifest.json
fa databricks-native plan --file databricks-manifest.json --json > generated/databricks-plan.json

Expected artifacts:

• Unity Catalog modernization plan
• Operational application candidates
• Measurement requirements

Open Engagements → active engagement in the App. This stage is visible at /engagements/<engagement-id>/artifacts after replacing the placeholder ID with the governed engagement ID.

4. Convert, move, and remediate

fa plan generate --file databricks-migration-plan.json --idempotency-key databricks-plan-v1
fa conversion batch create --file databricks-batch.json --idempotency-key databricks-batch-v1
fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1
fa residue list --engagement-id <engagement-id>
fa transfer plan --file databricks-transfer.json --idempotency-key databricks-transfer-v1
fa transfer run <transfer-id> --idempotency-key transfer-run-v1
fa transfer reconcile <transfer-id> --idempotency-key transfer-reconcile-v1

Expected artifacts:

• Target artifacts
• Residue cases
• Transfer checkpoints
• Reconciliation evidence

Open Engagements → active engagement in the App. This stage is visible at /engagements/<engagement-id>/runs after replacing the placeholder ID with the governed engagement ID.

5. Validate independently and inspect discrepancies

fa validation run --file databricks-validation.json --idempotency-key databricks-validation-v1
fa validation status <validation-execution-id> --json
fa discrepancy list --engagement-id <engagement-id>
fa artifact list --engagement-id <engagement-id>

Expected artifacts:

• Provider run references
• Readiness evidence
• Discrepancies

Open Engagements → active engagement in the App. This stage is visible at /engagements/<engagement-id>/runs after replacing the placeholder ID with the governed engagement ID.

6. Certify, cut over, and export evidence

fa certificate list --object-id <object-id>
fa cutover status <wave-id> --json
fa evidence list --engagement-id <engagement-id>
fa evidence export --file databricks-evidence-export.json --idempotency-key databricks-evidence-export-v1

Expected artifacts:

• Migration certificates
• Cutover evidence
• Content-digested evidence export

Open Engagements → active engagement in the App. This stage is visible at /assurance after replacing the placeholder ID with the governed engagement ID.

Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy.

What developers see in the App

The contextual source workspace shows the accepted estate and the factory stages for this engagement. Artifacts displays immutable references, content digests, media types, and provider lineage. Runs displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict.

View full size

These are automated captures from public synthetic engagements. The source workspace is specific to Databricks native estate; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images.