Skip to Content

Amazon DEA-C01: How to Link Customer Records Across Databases with Inconsistent Fields?

By: Author Alex Lim

Posted on  - Last updated:

Categories Exam

Discover the most efficient solution to link customer records across databases with inconsistent fields. Learn how to manage data inconsistencies and find duplicate records using AWS services like AWS Glue, Amazon EMR, and Amazon SageMaker. Minimize operational overhead and streamline your data processing workflow.

Table of Contents

Question

A company reads data from customer databases that run on Amazon RDS. The databases contain many inconsistent fields. For example, a customer record field that iPnamed place_id in one database is named location_id in another database. The company needs to link customer records across different databases, even when customer record fields do not match.

Which solution will meet these requirements with the LEAST operational overhead?

A. Create a provisioned Amazon EMR cluster to process and analyze data in the databases. Connect to the Apache Zeppelin notebook. Use the FindMatches transform to find duplicate records in the data.
B. Create an AWS Glue crawler to craw the databases. Use the FindMatches transform to find duplicate records in the data. Evaluate and tune the transform by evaluating the performance and results.
C. Create an AWS Glue crawler to craw the databases. Use Amazon SageMaker to construct Apache Spark ML pipelines to find duplicate records in the data.
D. Create a provisioned Amazon EMR cluster to process and analyze data in the databases. Connect to the Apache Zeppelin notebook. Use an Apache Spark ML model to find duplicate records in the data. Evaluate and tune the model by evaluating the performance and results.

Answer

B. Create an AWS Glue crawler to craw the databases. Use the FindMatches transform to find duplicate records in the data. Evaluate and tune the transform by evaluating the performance and results.

Explanation

The best solution to link customer records across databases with inconsistent fields while minimizing operational overhead is to use AWS Glue (Option B). Here’s why:

  1. AWS Glue Crawler: AWS Glue provides a crawler that can automatically discover and catalog metadata from the customer databases running on Amazon RDS. The crawler can handle inconsistencies in field names across different databases, making it easier to process and analyze the data.
  2. FindMatches Transform: AWS Glue offers a built-in FindMatches transform that can efficiently find duplicate records in the data. This transform uses machine learning algorithms to identify and link records that refer to the same entity, even when the field names or values don’t exactly match.
  3. Evaluation and Tuning: With AWS Glue, you can evaluate and tune the FindMatches transform by assessing its performance and the quality of the results. This allows you to optimize the deduplication process and ensure accurate linking of customer records.
  4. Minimal Operational Overhead: AWS Glue is a fully managed service, which means it handles the underlying infrastructure and automatically scales the resources based on the workload. This minimizes the operational overhead compared to using Amazon EMR (Options A and D), where you need to provision and manage the cluster manually.
  5. No Need for Additional Services: While Amazon SageMaker (Option C) is a powerful machine learning platform, it adds unnecessary complexity to this specific use case. AWS Glue’s FindMatches transform is sufficient for finding duplicate records, and using SageMaker would introduce additional operational overhead.

In summary, using AWS Glue with its crawler and FindMatches transform is the most efficient and effective solution to link customer records across databases with inconsistent fields. It minimizes operational overhead while providing the necessary functionality to handle data inconsistencies and find duplicate records.

Amazon AWS Certified Data Engineer – Associate DEA-C01 certification exam assessment practice question and answer (Q&A) dump including multiple choice questions (MCQ) and objective type questions, with detail explanation and reference available free, helpful to pass the Amazon AWS Certified Data Engineer – Associate DEA-C01 exam and earn Amazon AWS Certified Data Engineer – Associate DEA-C01 certification.