Azure Event Hubs to Databricks

fa source inspect event_hubs
fa source plan event_hubs --variant azure_event_hubs

Inventory event hubs, partitions, consumer groups, schemas, Capture destinations, retention, access policies, producers, and consumers. The movement adapter uses checkpointed reads or Capture-file ingestion and returns starting offsets or sequence numbers, enqueue-time boundaries, checkpoints, restarts, lag, and throughput.

Airlift requires independent loss, duplicate, schema compatibility, partition ordering, business aggregate, and performance evidence. Protocol-specific producers, capture format assumptions, shared-access policy design, checkpoint coupling, and cross-hub ordering remain visible as residue.

Once evidence passes, freeze the producer and consumer scope, collect authenticated approvals, switch through a certified checkpoint/apply-once/verify effector, and retain the rollback result. Structured Streaming, Lakeflow, Delta event tables, Unity Catalog, and streaming tables are separate modernization targets.

The pack is cataloged; a configured Event Hubs connector is not by itself a certified migration route.

Complete developer command sequence

Generate this exact recipe from the installed CLI so the guide and executable surface stay in sync:

fa source recipe event_hubs
fa source recipe event_hubs --variant azure_event_hubs --json > .airlift/event_hubs-recipe.json

The App is engagement-aware. Azure Event Hubs appears under Active sources only after the source estate is added to an active engagement. The menu is derived from governed engagement scope; installing Airlift does not expose unrelated source pages.

Every remote mutation below requires --host, --org, authenticated workspace identity, and a stable --idempotency-key. JSON request files contain identifiers, artifact references, and opaque credential references—never passwords, tokens, or connection strings. Run fa <resource> <operation> --help for the current schema and exit semantics.

0. Inspect the source contract

fa source inspect event_hubs --json > .airlift/event_hubs-profile.json
fa source plan event_hubs --variant azure_event_hubs --json > .airlift/event_hubs-capability-plan.json

Expected artifacts:

• .airlift/event_hubs-profile.json
• .airlift/event_hubs-capability-plan.json

Open Engagements → active engagement in the App. This stage is visible at /engagements after replacing the placeholder ID with the governed engagement ID.

1. Create governed scope and connection references

fa engagement create --file engagement.json --idempotency-key migration-create-v1
fa estate register --file event_hubs-estate.json --idempotency-key event_hubs-estate-v1
fa connection register --file event_hubs-connection.json --idempotency-key event_hubs-connection-v1
fa engagement update --file event_hubs-scope.json --idempotency-key event_hubs-scope-v1
fa engagement preflight <engagement-id>

Expected artifacts:

• Governed engagement
• Source estate
• Opaque connection binding

Open Engagements → active engagement in the App. This stage is visible at /engagements/<engagement-id> after replacing the placeholder ID with the governed engagement ID.

2. Assess and accept inventory

fa assessment start --file event_hubs-assessment-start.json --idempotency-key event_hubs-assessment-start-v1
fa assessment status <assessment-id> --json
fa assessment record --file event_hubs-assessment-record.json --idempotency-key event_hubs-assessment-record-v1
fa assessment accept --file event_hubs-assessment-accept.json --idempotency-key event_hubs-assessment-accept-v1
fa inventory list --estate-id <estate-id> --json

Expected artifacts:

• Assessment report reference
• Normalized inventory
• Dependency graph

Open Engagements → active engagement in the App. This stage is visible at /engagements/<engagement-id>/sources/event_hubs after replacing the placeholder ID with the governed engagement ID.

3. Implement the admitted source adapter

No source-specific executable compiler exists in this release. Continue with assessment, governed work, and an admitted adapter; Airlift does not invent executable output.

Expected artifacts:

• Capability plan and adapter requirements only

Open Engagements → active engagement in the App. This stage is visible at /engagements/<engagement-id>/artifacts after replacing the placeholder ID with the governed engagement ID.

This source is cataloged or assessable but has no source-specific executable compiler in the installed Airlift generation.

4. Convert, move, and remediate

fa plan generate --file event_hubs-migration-plan.json --idempotency-key event_hubs-plan-v1
fa conversion batch create --file event_hubs-batch.json --idempotency-key event_hubs-batch-v1
fa conversion batch start --file conversion-batch-start.json --idempotency-key conversion-start-v1
fa residue list --engagement-id <engagement-id>
fa transfer plan --file event_hubs-transfer.json --idempotency-key event_hubs-transfer-v1
fa transfer run <transfer-id> --idempotency-key transfer-run-v1
fa transfer reconcile <transfer-id> --idempotency-key transfer-reconcile-v1

Expected artifacts:

• Target artifacts
• Residue cases
• Transfer checkpoints
• Reconciliation evidence

Open Engagements → active engagement in the App. This stage is visible at /engagements/<engagement-id>/runs after replacing the placeholder ID with the governed engagement ID.

5. Validate independently and inspect discrepancies

fa validation run --file event_hubs-validation.json --idempotency-key event_hubs-validation-v1
fa validation status <validation-execution-id> --json
fa discrepancy list --engagement-id <engagement-id>
fa artifact list --engagement-id <engagement-id>

Expected artifacts:

• Provider run references
• Readiness evidence
• Discrepancies

Open Engagements → active engagement in the App. This stage is visible at /engagements/<engagement-id>/runs after replacing the placeholder ID with the governed engagement ID.

6. Certify, cut over, and export evidence

fa certificate list --object-id <object-id>
fa cutover status <wave-id> --json
fa evidence list --engagement-id <engagement-id>
fa evidence export --file event_hubs-evidence-export.json --idempotency-key event_hubs-evidence-export-v1

Expected artifacts:

• Migration certificates
• Cutover evidence
• Content-digested evidence export

Open Engagements → active engagement in the App. This stage is visible at /assurance after replacing the placeholder ID with the governed engagement ID.

Runway executes releases; Experiments owns validation verdicts; Airlift owns migration readiness and cutover policy.

What developers see in the App

The contextual source workspace shows the accepted estate and the factory stages for this engagement. Artifacts displays immutable references, content digests, media types, and provider lineage. Runs displays assessment, conversion, transfer, validation, and deployment executions without treating a provider's success as an Airlift verdict.

View full size

These are automated captures from public synthetic engagements. The source workspace is specific to Azure Event Hubs; no unrelated source is presented as its migration journey. For sources without an evidence-backed journey, the image demonstrates setup, navigation, and developer entry points only—not a live connection, converted output, or certified migration. No client data, credentials, workspace hostnames, or internal deployment identifiers are embedded in the images.