Learn how to design a robust and scalable solution for processing sensor data on AWS by migrating to Amazon Managed Streaming for Apache Kafka (Amazon MSK), using a Network Load Balancer (NLB) with health checks to distribute traffic, and ensuring high availability to prevent data loss.
Table of Contents
Question
A company has more than 10,000 sensors that send data to an on-premises Apache Kafka server by using the Message Queuing Telemetry Transport (MQTT) protocol. The on-premises Kafka server transforms the data and then stores the results as objects in an Amazon S3 bucket.
Recently, the Kafka server crashed. The company lost sensor data while the server was being restored. A solutions architect must create a new design on AWS that is highly available and scalable to prevent a similar occurrence.
Which solution will meet these requirements?
A. Launch two Amazon EC2 instances to host the Kafka server in an active/standby configuration across two Availability Zones. Create a domain name in Amazon Route 53. Create a Route 53 failover policy. Route the sensors to send the data to the domain name.
B. Migrate the on-premises Kafka server to Amazon Managed Streaming for Apache Kafka (Amazon MSK). Create a Network Load Balancer (NLB) that points to the Amazon MSK broker. Enable NLB health checks. Route the sensors to send the data to the NLB.
C. Deploy AWS IoT Core, and connect it to an Amazon Kinesis Data Firehose delivery stream. Use an AWS Lambda function to handle data transformation. Route the sensors to send the data to AWS IoT Core.
D. Deploy AWS IoT Core, and launch an Amazon EC2 instance to host the Kafka server. Configure AWS IoT Core to send the data to the EC2 instance. Route the sensors to send the data to AWS IoT Core.
Answer
B. Migrate the on-premises Kafka server to Amazon Managed Streaming for Apache Kafka (Amazon MSK). Create a Network Load Balancer (NLB) that points to the Amazon MSK broker. Enable NLB health checks. Route the sensors to send the data to the NLB.
Explanation
To meet the requirements of a highly available and scalable design for the sensor data processing workload, you should choose option B: Migrate the on-premises Kafka server to Amazon Managed Streaming for Apache Kafka (Amazon MSK), create a Network Load Balancer (NLB) that points to the Amazon MSK broker, enable NLB health checks, and route the sensors to send the data to the NLB.
Here’s why this option is the correct choice:
- Amazon MSK: Amazon MSK is a fully managed service that makes it easy to build and run applications using Apache Kafka. By migrating the on-premises Kafka server to Amazon MSK, you can leverage its managed infrastructure, which provides high availability, scalability, and durability.
- Network Load Balancer (NLB): Create an NLB that points to the Amazon MSK broker. An NLB distributes incoming connections across multiple targets, in this case, the Amazon MSK brokers. It ensures that the workload is evenly distributed and can handle high traffic volumes.
- Enable NLB health checks: By enabling health checks for the NLB, the load balancer can monitor the health of the Amazon MSK brokers. If a broker becomes unhealthy or unavailable, the NLB will automatically redirect traffic to the healthy brokers, ensuring continuous operation and preventing data loss.
- Route sensors to the NLB: Configure the sensors to send the data to the NLB’s endpoint. The NLB will distribute the data across the available Amazon MSK brokers, allowing for scalability and fault tolerance.
By migrating to Amazon MSK, using an NLB with health checks, and routing the sensor data to the NLB, you can achieve a highly available and scalable design that prevents data loss in the event of server failures, while benefiting from the managed infrastructure provided by Amazon MSK.
Amazon AWS Certified Solutions Architect – Professional SAP-C02 certification exam practice question and answer (Q&A) dump with detail explanation and reference available free, helpful to pass the Amazon AWS Certified Solutions Architect – Professional SAP-C02 exam and earn Amazon AWS Certified Solutions Architect – Professional SAP-C02 certification.