Study with Certified-Data-Engineer-Professional Actual Test Questions

A data engineer wants to enforce the principle of least privilege when configuring ACLs for Databricks jobs in a collaborative workspace. Which approach should the data engineer use?

• A. Use only folder-level permissions and avoid setting permissions on individual jobs.
• B. Assign users only the minimum permission level (e.g., CAN RUN or CAN VIEW) required for their role on each job.
• C. Grant CAN RUN permission to everyone and CAN MANAGE to a single admin group.
• D. Grant all users CAN MANAGE permission on all jobs to avoid access issues.

The following table consists of items found in user carts within an e-commerce website.

The following MERGE statement is used to update this table using an updates view, with schema evolution enabled on this table.

How would the following update be handled?

• A. The update is moved to separate ''restored'' column because it is missing a column expected in the target schema.
• B. The new nested field is added to the target schema, and files underlying existing records are updated to include NULL values for the new field.
• C. The update throws an error because changes to existing columns in the target schema are not supported.
• D. The new restored field is added to the target schema, and dynamically read as NULL for existing unmatched records.

Assuming that the Databricks CLI has been installed and configured correctly, which Databricks CLI command can be used to upload a custom Python Wheel to object storage mounted with the DBFS for use with a production job?

• A. workspace
• B. libraries
• C. configure
• D. fs
• E. jobs

An upstream system is emitting change data capture (CDC) logs that are being written to a cloud object storage directory. Each record in the log indicates the change type (insert, update, or delete) and the values for each field after the change. The source table has a primary key identified by the field pk_id.
For auditing purposes, the data governance team wishes to maintain a full record of all values that have ever been valid in the source system. For analytical purposes, only the most recent value for each record needs to be recorded. The Databricks job to ingest these records occurs once per hour, but each individual record may have changed multiple times over the course of an hour.
Which solution meets these requirements?

• A. Use Delta Lake's change data feed to automatically process CDC data from an external system, propagating all changes to all dependent tables in the Lakehouse.
• B. Ingest all log information into a bronze table; use merge into to insert, update, or delete the most recent entry for each pk_id into a silver table to recreate the current table state.
• C. Create a separate history table for each pk_id resolve the current state of the table by running a union all filtering the history tables for the most recent state.
• D. Iterate through an ordered set of changes to the table, applying each in turn; rely on Delta Lake's versioning ability to create an audit log.
• E. Use merge into to insert, update, or delete the most recent entry for each pk_id into a bronze table, then propagate all changes throughout the system.

What describes a primary technical challenge in ensuring consistent PII masking across all nodes in large-scale, distributed Databricks batch and streaming pipelines?

• A. Dynamic data masking is applied only at rest, so it does not affect query performance.
• B. Masking functions must be standardized and managed through Unity Catalog, with enforcement applied across all relevant datasets to avoid any data inconsistency.
• C. Native masking in Databricks automatically synchronizes with all downstream external Databricks systems.
• D. PII masking is only required for direct identifiers.