Databricks Exam Databricks-Machine-Learning-Associate Topic 3 Question 26 Discussion

Actual exam question for Databricks's Databricks-Machine-Learning-Associate exam

Question #: 26
Topic #: 3

[All Databricks-Machine-Learning-Associate Questions]

A data scientist wants to use Spark ML to one-hot encode the categorical features in their PySpark DataFrame features_df. A list of the names of the string columns is assigned to the input_columns variable.

They have developed this code block to accomplish this task:

The code block is returning an error.

Which of the following adjustments does the data scientist need to make to accomplish this task?

AThey need to specify the method parameter to the OneHotEncoder.

BThey need to remove the line with the fit operation.

CThey need to use Stringlndexer prior to one-hot encodinq the features.

DThey need to use VectorAssembler prior to one-hot encoding the features.

Show Suggested Answer

Suggested Answer: C

For large datasets, Spark ML uses iterative optimization methods to distribute the training of a linear regression model. Specifically, Spark MLlib employs techniques like Stochastic Gradient Descent (SGD) and Limited-memory Broyden--Fletcher--Goldfarb--Shanno (L-BFGS) optimization to iteratively update the model parameters. These methods are well-suited for distributed computing environments because they can handle large-scale data efficiently by processing mini-batches of data and updating the model incrementally.

Databricks documentation on linear regression: Linear Regression in Spark ML

by Brock at Mar 04, 2025, 05:06 AM

Limited Time Offer

25%

15 days ago

I think the data scientist needs to specify the method parameter to the OneHotEncoder.

upvoted 0 times

...