Databricks Exam Databricks-Certified-Professional-Data-Engineer Topic 1 Question 4 Discussion

Actual exam question for Databricks's Databricks Certified Data Engineer Professional exam

Question #: 4
Topic #: 1

[All Databricks Certified Data Engineer Professional Questions]

A junior data engineer is migrating a workload from a relational database system to the Databricks Lakehouse. The source system uses a star schema, leveraging foreign key constrains and multi-table inserts to validate records on write.

Which consideration will impact the decisions made by the engineer while migrating this workload?

AAll Delta Lake transactions are ACID compliance against a single table, and Databricks does not enforce foreign key constraints.

BDatabricks only allows foreign key constraints on hashed identifiers, which avoid collisions in highly-parallel writes.

CForeign keys must reference a primary key field; multi-table inserts must leverage Delta Lake's upsert functionality.

DCommitting to multiple tables simultaneously requires taking out multiple table locks and can lead to a state of deadlock.

Show Suggested Answer

Suggested Answer: A

In Databricks and Delta Lake, transactions are indeed ACID-compliant, but this compliance is limited to single table transactions. Delta Lake does not inherently enforce foreign key constraints, which are a staple in relational database systems for maintaining referential integrity between tables. This means that when migrating workloads from a relational database system to Databricks Lakehouse, engineers need to reconsider how to maintain data integrity and relationships that were previously enforced by foreign key constraints. Unlike traditional relational databases where foreign key constraints help in maintaining the consistency across tables, in Databricks Lakehouse, the data engineer has to manage data consistency and integrity at the application level or through careful design of ETL processes. Reference:

Databricks Documentation on Delta Lake: Delta Lake Guide

Databricks Documentation on ACID Transactions in Delta Lake: ACID Transactions in Delta Lake

by Aliza at Mar 16, 2024, 02:57 AM

Limited Time Offer

25%

Off

Get Premium Databricks-Certified-Professional-Data-Engineer Questions as Interactive Web-Based Practice Test or PDF

Contribute your Thoughts:

Submit Cancel

Tommy

2 days ago

I think the key consideration here is how Delta Lake transactions differ from the source system's handling of foreign key constraints.

upvoted 0 times

...