Unity Catalog evidence spine

DBX2 status: this release implements and tests the Unity Catalog projection and reconciliation contracts in the repository. It does not claim live closure until the exact candidate digest has same-digest workspace Job and reconciliation evidence.

The Databricks-native evidence surface is a governed projection of the Airlift event log into the Unity Catalog analytics schema, plus the reconciliation of every Airlift target artifact to its exact target UC identity. It is read-only and fail-closed: missing privilege, a missing system table, lag, or a failed/partial refresh renders a named blocker or not_observed — never a misleading zero and never a misleading 100% coverage. A stale projection never displays a current metric.

Two governed actions build the evidence:

• airlift.uc_projection_run_record — records one run of the analytics projection. The run carries the source-event cursor watermark it advanced to; freshness derives from that watermark, never from the run's completion time, so a permanently-failing refresh loop can never render "fresh". A failed or partial run never advances the displayed watermark. A watermark that regresses below the newest admitted watermark is rejected. App-triggered refreshes use a Databricks Jobs idempotency token, per-org engagement-scoped single-flight, a 20-second request budget, and cursor-windowed event/artifact scope that never splits equal millisecond timestamps. A still-running or unfinished backlog refresh records partial; a terminal job failure records failed.
• airlift.uc_target_reconciliation_record — records the reconciliation of one target artifact, keyed per artifact. An observed resolution binds the artifactId, the exact candidate digest, and the exact target UC identity; digest-binding (not name-matching) is required. Every required surface is either native-observed for that exact target or explicitly notApplicable; those sets are disjoint and exhaustive. Missing DESCRIBE rows, or an external location/connection that does not match the target, remains blocked. This is what makes target drift detectable.

Evidence-state law

Freshness derives from the watermark (clock − watermark), never from run recency. Coverage is suppressed whenever any declared scope is unreadable, so it can never render a misleading 100%. The freshness window is a module constant, never organization-configurable. The derived row names its selected engagementId; organization-level callers never display a scope-free watermark or ratio.

Target identity resolution

resolveDatabricksTargetIdentity is a join over admitted reconciliations — never a function of the artifact alone, because an artifact carries no UC identity and any convention-derived name would be configuration rendering as proof. An observed resolution requires an admitted reconciliation binding the artifactId, the exact candidate digest, and the exact target UC identity (catalog, schema, name, objectType, and the stable entityRef/entityDigest when the workspace discloses them). Artifact bytes stay in their provider-owned store; Airlift stores only references and digests.

Recording contract

Recording is an admitted system action through an in-process admitted system principal; the remote transport carries no system actor, so there is currently no transport by which these commands succeed in a governed deployment. The commands below are the admitted probe's in-process path, never an operator's.

fa uc-evidence record --kind projection-run \
  --file .airlift/uc-projection-run.json \
  --idempotency-key uc-projection-<run-id>

fa uc-evidence record --kind reconciliation \
  --file .airlift/uc-reconciliation.json \
  --idempotency-key uc-reconciliation-<run-id>

fa uc-evidence show                       # exit 0 only when evidence is current
fa uc-evidence show --engagement-id <id>  # scope to one engagement
fa uc-evidence show --json

analytics-projection is the closed DBX2 projection identifier. Admission rejects caller-invented projection IDs, future watermarks, watermarks later than run completion, and any watermark below the highest admitted cursor for the engagement.

Refresh migration readiness (App-triggered)

The deployed Databricks App exposes the same pipeline as one operator action in the Databricks-native evidence section: Refresh migration readiness. From the operator's perspective the contract is:

• Select an engagement. The action runs per organization and engagement against an active or frozen engagement; the derived row always names that engagement. A role must carry airlift:uc_evidence:observe; without it the action is denied with "Your Airlift role cannot refresh migration readiness."
• Single-flight. One refresh per organization and engagement runs at a time. A second press while a refresh is in flight joins the running job rather than starting a duplicate.
• What runs in Databricks. The App writes a metadata-only projection snapshot of the admitted Airlift events to the configured UC Volume and triggers the native Databricks Workflow job with a Jobs idempotency token derived from the snapshot digest — a retry of the same snapshot collapses to the same job run, so a refresh can never double-apply.
• What gets recorded. The run and each artifact reconciliation are admitted through airlift.uc_projection_run_record and airlift.uc_target_reconciliation_record by the admitted worker principal, with idempotency keys bound to the evidence digest. The button never mints evidence directly.
• Bounded scope. A refresh projects the cursor-windowed event batch since the previous watermark (never splitting equal millisecond timestamps) and reconciles the engagement's current artifacts — superseded candidates are excluded. An engagement with more than 20 current artifacts is refused up front with an instruction to split it into smaller engagements.

The button answers with the honest terminal state:

Consistent with the evidence-state law above, a failed or partial run never renders as current: freshness derives from the newest completed watermark, so a broken refresh loop degrades to stale or failed rather than displaying a misleading current metric.

Honest support boundary

Governed deployments currently fail closed: no production runtime wires an admitted UC evidence verifier yet, so no governed deployment can record a projection run or reconciliation until one is admitted. That is the deliberate, honest default. A current evidence state is never integration proof and promotes nothing — not a capability cell, a certificate, a gate, or a target integration's observed state.