← Back to exam page

Free CERTIFIED-DATA-ENGINEER-PROFESSIONAL – Databricks CERTIFIED-DATA-ENGINEER-PROFESSIONAL Practice Test Question

Loading demo links...

Showing 1–3 of 26 questions

Question 1 (Topic 1)

An upstream system is emitting change data capture (CDC) logs that are being written to a cloud object storage directory. Each record in the log indicates the change type (insert, update, or delete) and the values for each field after the change. The source table has a primary key identified by the field pk_id. For auditing purposes, the data governance team wishes to maintain a full record of all values that have ever been valid in the source system. For analytical purposes, only the most recent value for each record needs to be recorded. The Databricks job to ingest these records occurs once per hour, but each individual record may have changed multiple times over the course of an hour. Which solution meets these requirements?

Select an option, then click Submit answer.

  • ○ Create a separate history table for each pk_id resolve the current state of the table by running a union all filtering the history tables for the most recent state.
  • ○ Use MERGE INTO to insert, update, or delete the most recent entry for each pk_id into a bronze table, then propagate all changes throughout the system.
  • ○ Iterate through an ordered set of changes to the table, applying each in turn; rely on Delta Lake's versioning ability to create an audit log.
  • ○ Use Delta Lake's change data feed to automatically process CDC data from an external system, propagating all changes to all dependent tables in the Lakehouse.
  • ○ Ingest all log information into a bronze table; use MERGE INTO to insert, update, or delete the most recent entry for each pk_id into a silver table to recreate the current table state.
Question 2 (Topic 1)

A Spark job is taking longer than expected. Using the Spark UI, a data engineer notes that the Min, Median, and Max Durations for tasks in a particular stage show the minimum and median time to complete a task as roughly the same, but the max duration for a task to be roughly 100 times as long as the minimum. Which situation is causing increased duration of the overall job?

Select an option, then click Submit answer.

  • ○ Task queueing resulting from improper thread pool assignment.
  • ○ Spill resulting from attached volume storage being too small.
  • ○ Network latency due to some cluster nodes being in different regions from the source data
  • ○ Skew caused by more data being assigned to a subset of spark-partitions.
  • ○ Credential validation errors while pulling data from an external system.
Question 3 (Topic 1)

The data engineering team is migrating an enterprise system with thousands of tables and views into the Lakehouse. They plan to implement the target architecture using a series of bronze, silver, and gold tables. Bronze tables will almost exclusively be used by production data engineering workloads, while silver tables will be used to support both data engineering and machine learning workloads. Gold tables will largely serve business intelligence and reporting purposes. While personal identifying information (PII) exists in all tiers of data, pseudonymization and anonymization rules are in place for all data at the silver and gold levels. The organization is interested in reducing security concerns while maximizing the ability to collaborate across diverse teams. Which statement exemplifies best practices for implementing this system?

Select an option, then click Submit answer.

  • ○ Isolating tables in separate databases based on data quality tiers allows for easy permissions management through database ACLs and allows physical separation of default storage locations for managed tables.
  • ○ Because databases on Databricks are merely a logical construct, choices around database organization do not impact security or discoverability in the Lakehouse.
  • ○ Storing all production tables in a single database provides a unified view of all data assets available throughout the Lakehouse, simplifying discoverability by granting all users view privileges on this database.
  • ○ Working in the default Databricks database provides the greatest security when working with managed tables, as these will be created in the DBFS root.
  • ○ Because all tables must live in the same storage containers used for the database they're created in, organizations should be prepared to create between dozens and thousands of databases depending on their data isolation requirements.