CDP Data Engineer - Certification Exam - CDP-3002 Exam Practice Test

Question 1

How does setting the "priority_weight" parameter in Airflow tasks influence the scheduler's behavior?

A. It determines the order in which tasks are executed within the same DAG.

B. It allocates more CPIJ and memory resources to tasks with higher priority weights.

C. It sets a fixed time slot during which the task must be executed, overriding the scheduler's automatic timing.

D. It influences the order in which tasks are selected for execution across multiple DAGs competing for resources.

Correct Answer: D

Explanation: Only visible for Actualtests4sure members. You can sign-up / login (it's free).

Question 2

When leveraging caching in Spark, which scenario illustrates the use of the MEMORY ONLY SER storage level most effectively?

A. Caching datasets that are frequently accessed and modified.

B. Caching small, static datasets used in lookup operations.

C. Caching datasets that require fast, sequential access without the need for serialization.

D. Caching large datasets that do not fit into memory when stored in deserialized form.

Correct Answer: D

Explanation: Only visible for Actualtests4sure members. You can sign-up / login (it's free).

Question 3

You're tasked with scheduling an ETL pipeline in Airflow that extracts data from a database, transforms it, and loads it into a data warehouse. How can you achieve this using Airflow operators?

A. Utilize Airflow's built-in ETL operators like BigQueryOperator or MySqlOperator (these operators are specific to certain data sources and not generally applicable for all ETL pipelines).

B. Implement a single custom Python script containing the entire ETL logic and call it within a BashOperator.

C. Use three separate Python operators for each stage (extract, transform, loaD. and chain them together in the DAG.

D. Leverage dedicated operators like PostgresHook for extraction, custom Python operators for transformation, and S3Hook for loading.

Correct Answer: D

Explanation: Only visible for Actualtests4sure members. You can sign-up / login (it's free).

Question 4

You're building an Airflow DAG to extract data from a database table that is constantly updated. How can you implement incremental extraction to avoid processing the entire table each time the DAG runs?

A. Implement a custom Python script to track the last processed record and use it to filter the data during subsequent runs.

B. Use the File sensor to check for the presence of a new data file and trigger the DAG only when the file appears.

C. Leverage the Previous execution operator to access the execution date of the previous DAG run and use it as a filter in the extraction query.

D. Configure the database connection to only retrieve newly added data.

Correct Answer: C

Explanation: Only visible for Actualtests4sure members. You can sign-up / login (it's free).

Question 5

You are working with a large, skewed dataset in Spark. How would you optimize processing to mitigate the impact of skew and improve performance?

A. Use salting on the skewed column during data partitioning.

B. Addressing skewed data requires

C. Broadcast the skewed data to all executors.

D. Implement custom partitioners to evenly distribute skewed values.

Correct Answer: B

Explanation: Only visible for Actualtests4sure members. You can sign-up / login (it's free).

Question 6

When would it be advantageous to use both partitioning and bucketing on a Hive table?

A. When dealing with large datasets that require efficient querying and data sampling

B. When managing small datasets to reduce complexity

C. When data security is a primary concern

D. When data needs to be stored in a single file for archival purposes

Correct Answer: A

Explanation: Only visible for Actualtests4sure members. You can sign-up / login (it's free).

Question 7

You need to design a DAG that can be easily triggered based on external events or data availability. How can you achieve this functionality?

A. Schedule the DAG to run continuously at a fixed interval.

B. Rely on manual intervention to trigger the DAG whenever necessary.

C. Utilize Airflow sensors like FileSensor or S3KeySensor to wait for specific conditions before triggering the DAG.

D. Implement custom code within the DAG to actively check for external events or data availability.

Correct Answer: C

Question 8

You are developing a PySpark application that processes a large dataset. Your application involves a transformation that requires shuffling data across the Executors. Which of the following responsibilities of the Spark Driver is most directly involved in this process?

A. Interpreting the user application and creating an execution plan.

B. Managing the lifecycle of Executor pods in Kubernetes.

C. Managing the shuffling and aggregation of data across Executors.

D. Monitoring the progress and status of tasks on the Executors.

Correct Answer: C

Explanation: Only visible for Actualtests4sure members. You can sign-up / login (it's free).

Question 9

You're working with a real-time streaming application using Spark Streaming. How can you ensure that your application gracefully handles late-arriving data and maintains data consistency?

A. Recompute the entire stream from scratch for each late record

B. Ignore late-arriving data altogether

C. Implement micro-batching with windowing and watermarking techniques

D. Use Spark's checkpointing functionality to recover from failures

Correct Answer: C

Explanation: Only visible for Actualtests4sure members. You can sign-up / login (it's free).

Question 10

Consider a PySpark application that calculates the count of rows in a CSV file. The main application file is 'count_rows.py', and it uses a custom library 'data_utils.py'. After packaging, which 'spark-submit' command correctly submits this job including the custom library?

A. 'spark-submit -py-files data utils.py count_rows.py'

B. 'spark-submit --archives data_utils.py count_rows.py'

C. 'spark-submit --files data_utils.py count_rows.py'

D. 'spark-submit --jars data utils.py count_rows.py'

Correct Answer: A

Explanation: Only visible for Actualtests4sure members. You can sign-up / login (it's free).

Question 11

What is the role of an Operator in an Apache Airflow DAG?
A To schedule DAG runs based on time or external triggers

A. To represent a single task or a step within a DAG

B. To monitor the execution of DAGs and provide notifications

C. To store metadata and logs for tasks and DAGs

Correct Answer: B

Explanation: Only visible for Actualtests4sure members. You can sign-up / login (it's free).

Question 12

You're working with a complex data pipeline involving Spark and Hive, and you need to monitor its performance and identify potential bottlenecks. Which tools and techniques can you employ for effective monitoring?

A. Implement custom instrumentation code within your Spark application

B. Manually analyze Spark and Hive logs after job completion

C. Leverage Spark's web UI and Hive logs for basic information

D. Utilize YARN resource manager and Spark/Hive metrics for detailed monitoring

Correct Answer: D

Explanation: Only visible for Actualtests4sure members. You can sign-up / login (it's free).

Question 13

Which tool or API is primarily used for monitoring and inspecting the performance of Spark applications in real-time?
A Spark History Server

A. Spark Web UI

B. Apache Ambari

C. Hadoop YARN ResourceManager UI

Correct Answer: C

Explanation: Only visible for Actualtests4sure members. You can sign-up / login (it's free).

Cloudera CDP Data Engineer - Certification - CDP-3002 Exam Practice Test

Latest Update

Useful Links

Contact Us