DP-750 Certification Practice Question #52

Question

You must stand up incremental file ingestion from an Azure Data Lake Storage container into a Unity Catalog managed table using Auto Loader, run as a triggered batch job. Order the configuration and authoring steps into the correct sequence to produce a governed, fault-tolerant ingestion job. Drag each step into the correct order. ```mermaid flowchart TD A["Slot 1"] --> B["Slot 2"] --> C["Slot 3"] --> D["Slot 4"] --> E["Slot 5"] subgraph TILES["Steps (unordered)"] T1["Create a Unity Catalog external location over the ADLS container
and grant READ FILES to the executing identity"] T2["Define spark.readStream.format('cloudFiles') with cloudFiles.format
and a persistent cloudFiles.schemaLocation"] T3["Configure the writeStream with a unique checkpointLocation
and .trigger(availableNow=True)"] T4["Start the query with .toTable('main.bronze.orders')"] T5["Schedule the notebook as a Lakeflow Job so the stream restarts
and processes only newly arrived files"] end ```

Accepted Answer

A governed Auto Loader job requires the Unity Catalog external location and `READ FILES` grant to exist first, otherwise the read is unauthorized. You then define the `cloudFiles` streaming reader with a persistent `cloudFiles.schemaLocation` to track schema evolution, configure the `writeStream` with a unique `checkpointLocation` (for exactly-once fault tolerance) and `Trigger.AvailableNow` (process all files that arrived before start, then stop), and start the query with the terminal `toTable` sink. Finally, scheduling the notebook as a Lakeflow Job restarts the stream on a cadence so only newly arrived files are processed each run.

More DP-750 practice questions