Learn the most efficient method to transform and ingest millions of daily real-time events from Amazon Kinesis into OpenSearch Service. Minimize operational overhead with this solution.
Table of Contents
Question
A media company wants to use Amazon OpenSearch Service to analyze rea-time data about popular musical artists and songs. The company expects to ingest millions of new data events every day. The new data events will arrive through an Amazon Kinesis data stream. The company must transform the data and then ingest the data into the OpenSearch Service domain.
Which method should the company use to ingest the data with the LEAST operational overhead?
A. Use Amazon Kinesis Data Firehose and an AWS Lambda function to transform the data and deliver the transformed data to OpenSearch Service.
B. Use a Logstash pipeline that has prebuilt filters to transform the data and deliver the transformed data to OpenSearch Service.
C. Use an AWS Lambda function to call the Amazon Kinesis Agent to transform the data and deliver the transformed data OpenSearch Service.
D. Use the Kinesis Client Library (KCL) to transform the data and deliver the transformed data to OpenSearch Service.
Answer
The best method for the media company to ingest millions of real-time data events every day from Amazon Kinesis into Amazon OpenSearch Service with the least operational overhead is:
A. Use Amazon Kinesis Data Firehose and an AWS Lambda function to transform the data and deliver the transformed data to OpenSearch Service.
Explanation
Amazon Kinesis Data Firehose is a fully managed service that reliably loads streaming data into data lakes, data stores and analytics services like OpenSearch Service. It can automatically scale to match the throughput of your data and requires no ongoing administration.
You can configure Kinesis Data Firehose to transform your data before delivering it. It provides a serverless data transformation option through AWS Lambda. With Lambda, you can write custom transformation code and Kinesis Data Firehose will call your Lambda function to transform the data before delivering it to OpenSearch Service.
This serverless architecture minimizes operational overhead because AWS handles resource management, automatic scaling, retry logic, and service availability for Kinesis Data Firehose and Lambda. You simply provide the data and transformation code.
The other options would require more hands-on management:
- B) Running your own Logstash pipeline
- C) Building a Lambda function to call the Kinesis Agent
- D) Implementing the Kinesis Client Library to pull data from the stream
So in summary, using Kinesis Data Firehose with a Lambda transformation function provides an efficient, serverless way to reliably ingest and transform a high volume of streaming data into OpenSearch Service while minimizing the operational burden on the media company.
Amazon AWS Certified Data Engineer – Associate DEA-C01 certification exam assessment practice question and answer (Q&A) dump including multiple choice questions (MCQ) and objective type questions, with detail explanation and reference available free, helpful to pass the Amazon AWS Certified Data Engineer – Associate DEA-C01 exam and earn Amazon AWS Certified Data Engineer – Associate DEA-C01 certification.