DP-750 Certification Practice Question #65

Question

A retail analytics team operates a Lakeflow Spark Declarative Pipeline (Python) that builds `silver.orders` from a bronze streaming source. Their data-quality SLA has three tiers, and they want each tier handled by the appropriate Lakeflow **expectation action**:

1. **Fatal tier** — `order_id` must never be null. If any null `order_id` arrives, the update must halt immediately and roll back so nothing dirty is committed.
2. **Cleanse tier** — Rows where `order_total` is negative are known-bad noise from a buggy source; they should be silently discarded before the table is written, but the pipeline must keep running.
3. **Observe tier** — Rows where `currency` is not `'USD'` are allowed through (downstream converts them), but the team wants the count of such rows tracked over time in the event log so they can watch the trend.

You must also reuse the **same dictionary of multiple checks with one collective fail action** for the fatal tier across several datasets.

Which expectation operators correctly satisfy these requirements? (Choose THREE.)

Accepted Answer

The three Lakeflow expectation actions map directly onto the three SLA tiers. The fatal tier needs the update to stop and roll back on any violation, which is `expect_or_fail`; grouping several fatal checks into one collective fail action and reusing it across datasets is exactly what `expect_all_or_fail` provides, so A is correct. The cleanse tier wants known-bad rows silently removed while the pipeline keeps running, which is `expect_or_drop` (the dropped count is still logged), so B is correct. The observe tier wants rows retained but counted, which is the default warn behavior of `expect` — invalid rows are written to the target and pass/fail metrics appear in the event log's data-quality metrics, so C is correct. D over-escalates noisy rows to a pipeline-halting failure, E discards rows the team wants kept, and F swaps the requested expectation mechanism for a table-level enforced constraint that does not emit the per-expectation pipeline metrics the SLA tracking relies on.

More DP-750 practice questions