Snowflake
Implement Snowflake migrations including SQL, tasks, streams, stages, dbt, semi-structured data, transfer, and reconciliation.
Migrate Snowflake with Airlift
The Snowflake profile covers SQL objects, Snowflake Scripting, tasks and streams, stages and pipes, dynamic tables, shares, dbt projects, external functions, and downstream consumers. Airlift uses one dependency graph so database objects are not certified while their task, stream, or consumer path is still missing.
What Airlift adds to a Snowflake migration
| Stage | Databricks tool or project adapter | Airlift responsibility |
|---|---|---|
| discover | Lakebridge Profiler and Analyzer plus account, dbt, task, stream, and sharing exports | accept inventory, dependency graph, exclusions, source version, and report digests |
| plan | assessment output informs target architecture | assign owners and dependency-aware waves; preserve account-level, external-function, and human-work decisions |
| convert | Morpheus converts supported Snowflake SQL and assists dbt repointing | record attempts, tool generations, target artifacts, warnings, and residue lanes |
| move data | bulk unload plus stream, timestamp, or application-watermark catch-up | track manifests, watermarks, lag, restart checkpoints, counts, and reconciliation |
| validate | Lakebridge Reconcile plus semi-structured, task, stream, and business scenarios | admit independent evidence for the exact object and source/target snapshots |
| certify | Airlift evaluates the active readiness profile | mint a signed certificate identifying artifacts, snapshots, evidence, and policy |
| cut over | project connection, task, dbt, share, and consumer effectors | freeze scope, enforce approvals, checkpoint, apply once, verify, and retain rollback evidence |
| modernize | Unity Catalog, Lakeflow, Delta, and Databricks SQL work | separate native redesign from the parity-preserving migration release |
This prevents a successful table copy from hiding missing task ordering, stream consumption, grants, shares, dbt dependencies, or downstream connection changes.
Compile the executable migration pack
Create a versioned manifest containing SQL objects, tasks, streams, stages, pipes, dynamic tables, Snowpark and external-function references, shares, dbt metadata, permissions, and consumers. Mark observed hard cases on their owning objects, then run:
fa migration-pack inspect --file snowflake-manifest.json
fa migration-pack plan --file snowflake-manifest.json --json > snowflake-plan.jsonThe compiler validates dependencies, reports missing Snowflake hard cases, and preserves deterministic, bounded-repair, and human-only lanes. Its plan includes unload and stream/watermark restart requirements, semi-structured and task-order validation suites, target assets, and a Runway-owned deployment contract. See migration-pack commands for registration and certification.
Inspect and generate a plan
fa source inspect snowflake
fa source plan snowflake --json > .airlift/snowflake-plan.jsonProfile and analyze
databricks labs lakebridge configure-database-profiler
databricks labs lakebridge execute-database-profiler \
--source-tech snowflake \
--output-folder ./artifacts/profile
databricks labs lakebridge analyze \
--source-directory ./source-export \
--source-tech "Snowflake" \
--report-file ./artifacts/snowflake-analysis.xlsx \
--generate-json trueInclude account-level integrations, shares, network policies, role grants, task graphs, stream consumption, stage definitions, and dbt metadata in the Airlift inventory even when Lakebridge analyzes only the SQL-bearing artifacts.
Convert
databricks labs lakebridge transpile \
--source-dialect snowflake \
--input-source ./source-export/sql \
--output-folder ./artifacts/convertedMorpheus is preferred and supports Snowflake SQL plus dbt repointing. Treat JavaScript procedures, Snowpark code, external functions, account-level objects, shares, and vendor-specific task behavior as explicit residue until a target implementation is recorded.
Transfer and reconcile
Use bulk unload to the client's cloud object storage and a stream, timestamp, or application watermark catch-up. Federation through a Unity Catalog connection is useful for validation; it does not by itself prove migration or restart behavior.
databricks labs lakebridge configure-reconcile
databricks labs lakebridge auto-configure-recon-tables
databricks labs lakebridge reconcileAdd validations for VARIANT, arrays and objects, timestamp/timezone behavior,
identifier case, decimal/null semantics, task ordering, and stream consumption. Review
auto-configured joins, mappings, filters, transformations, and thresholds before running.
Certify and cut over
Resolve the required readiness tracks for each table, procedure, task, stream, dbt model, share, and consumer before certificate minting. A project cutover effector should switch connections and schedules only after Airlift rechecks frozen scope, current certificates, and distinct approvals. Verify dbt runs, task ordering, stream progress, BI queries, and rollback independently before recording the wave complete.
Modernize after parity
Disposition tasks and dynamic tables into Lakeflow, use Unity Catalog for grants and sharing, retarget dbt, and benchmark Databricks SQL serverless plus liquid clustering in a separate release.
Complete developer command sequence
Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync:
fa source recipe snowflake
fa source recipe snowflake --variant snowflake --json > .airlift/snowflake-recipe.jsonThe App is engagement-aware. Snowflake appears under Active sources only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages.
Every remote mutation below requires --host, --org, authenticated workspace
identity, and a stable --idempotency-key. JSON request files contain identifiers,
artifact references, and opaque credential references—never passwords, tokens, or
connection strings. Run fa <resource> <operation> --help for the current schema and
exit semantics.
0. Inspect the source contract
fa source inspect snowflake --json > .airlift/snowflake-profile.json
fa source plan snowflake --variant snowflake --json > .airlift/snowflake-capability-plan.jsonExpected artifacts:
- .airlift/snowflake-profile.json
- .airlift/snowflake-capability-plan.json
Open Engagements → active engagement in the App. This stage is visible at
/engagements after replacing the placeholder ID with the governed engagement ID.
1. Create governed scope and connection references
fa engagement create --file engagement.json --idempotency-key migration-create-v1
fa estate register --file snowflake-estate.json --idempotency-key snowflake-estate-v1
fa connection register --file snowflake-connection.json --idempotency-key snowflake-connection-v1
fa engagement update --file snowflake-scope.json --idempotency-key snowflake-scope-v1
fa engagement preflight <engagement-id>Expected artifacts:
- Governed engagement
- Source estate
- Opaque connection binding
Open Engagements → active engagement in the App. This stage is visible at
/engagements/<engagement-id> after replacing the placeholder ID with the governed engagement ID.
2. Assess and accept inventory
fa assessment start --file snowflake-assessment-start.json --idempotency-key snowflake-assessment-start-v1
fa assessment status <assessment-id> --json
fa assessment record --file snowflake-assessment-record.json --idempotency-key snowflake-assessment-record-v1
fa assessment accept --file snowflake-assessment-accept.json --idempotency-key snowflake-assessment-accept-v1
fa inventory list --estate-id <estate-id> --jsonExpected artifacts:
- Assessment report reference
- Normalized inventory
- Dependency graph
Open Engagements → active engagement in the App. This stage is visible at
/engagements/<engagement-id>/sources/snowflake after replacing the placeholder ID with the governed engagement ID.
3. Compile the snowflake migration pack
fa migration-pack inspect --file snowflake-manifest.json
fa migration-pack plan --file snowflake-manifest.json --json > generated/snowflake-plan.jsonExpected artifacts:
- Dependency-aware migration pack
- Transfer requirements
- Validation requirements
- Residue lanes
Open Engagements → active engagement in the App. This stage is visible at
/engagements/<engagement-id>/artifacts after replacing the placeholder ID with the governed engagement ID.
4. Convert, move, and remediate
fa plan generate --file snowflake-migration-plan.json --idempotency-key snowflake-plan-v1
fa conversion batch create --file snowflake-batch.json --idempotency-key snowflake-batch-v1
fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1
fa residue list --engagement-id <engagement-id>
fa transfer plan --file snowflake-transfer.json --idempotency-key snowflake-transfer-v1
fa transfer run <transfer-id> --idempotency-key transfer-run-v1
fa transfer reconcile <transfer-id> --idempotency-key transfer-reconcile-v1Expected artifacts:
- Target artifacts
- Residue cases
- Transfer checkpoints
- Reconciliation evidence
Open Engagements → active engagement in the App. This stage is visible at
/engagements/<engagement-id>/runs after replacing the placeholder ID with the governed engagement ID.
5. Validate independently and inspect discrepancies
fa validation run --file snowflake-validation.json --idempotency-key snowflake-validation-v1
fa validation status <validation-execution-id> --json
fa discrepancy list --engagement-id <engagement-id>
fa artifact list --engagement-id <engagement-id>Expected artifacts:
- Provider run references
- Readiness evidence
- Discrepancies
Open Engagements → active engagement in the App. This stage is visible at
/engagements/<engagement-id>/runs after replacing the placeholder ID with the governed engagement ID.
6. Certify, cut over, and export evidence
fa certificate list --object-id <object-id>
fa cutover status <wave-id> --json
fa evidence list --engagement-id <engagement-id>
fa evidence export --file snowflake-evidence-export.json --idempotency-key snowflake-evidence-export-v1Expected artifacts:
- Migration certificates
- Cutover evidence
- Content-digested evidence export
Open Engagements → active engagement in the App. This stage is visible at
/assurance after replacing the placeholder ID with the governed engagement ID.
Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy.
What developers see in the App
The contextual source workspace shows the accepted estate and the factory stages for this engagement. Artifacts displays immutable references, content digests, media types, and provider lineage. Runs displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict.
Snowflake has its own engagement-scoped workspace; unrelated source systems are not shown.
This public synthetic capture demonstrates Airlift setup and navigation for Snowflake; it is not evidence of a live connection or certified migration.
Register the client estate, bind a credential reference, run the Snowflake assessment recipe, and admit the resulting evidence.
These are automated captures from public synthetic engagements. The source workspace is specific to Snowflake; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images.
Azure Synapse
Import Synapse and ADF metadata, generate Databricks implementation files, and govern migration evidence from assessment through cutover.
Amazon Redshift
Implement Redshift serverless and provisioned migrations with profiling variants, BladeBridge conversion, S3 transfer, and reconciliation.