Learn the best approach to create near real-time data visualization dashboards on AWS with minimal latency, using services like Amazon MSK, OpenSearch, S3, Athena, and QuickSight.
Table of Contents
Question
A company is creating near real-time dashboards to visualize time series data. The company ingests data into Amazon Managed Streaming for Apache Kafka (Amazon MSK). A customized data pipeline consumes the data. The pipeline then writes data to Amazon Keyspaces (for Apache Cassandra), Amazon OpenSearch Service, and Apache Avro objects in Amazon S3.
Which solution will make the data available for the data visualizations with the LEAST latency?
A. Create OpenSearch Dashboards by using the data from OpenSearch Service.
B. Use Amazon Athena with an Apache Hive metastore to query the Avro objects in Amazon S3. Use Amazon Managed Grafana to connect to Athena and to create the dashboards.
C. Use Amazon Athena to query the data from the Avro objects in Amazon S3. Configure Amazon Keyspaces as the data catalog. Connect Amazon QuickSight to Athena to create the dashboards.
D. Use AWS Glue to catalog the data. Use S3 Select to query the Avro objects in Amazon S3. Connect Amazon QuickSight to the S3 bucket to create the dashboards.
Answer
A. Create OpenSearch Dashboards by using the data from OpenSearch Service.
Explanation
The question states that the company is already ingesting data into Amazon MSK (Managed Streaming for Apache Kafka) and then writing that data to several destinations, including Amazon OpenSearch Service, after processing it through a customized data pipeline.
OpenSearch Service is purpose-built for log analytics, full-text search, and application monitoring, providing near real-time access to the ingested data. It includes OpenSearch Dashboards, a visualization tool that allows you to create interactive dashboards directly from the data in OpenSearch Service. This approach offers the lowest latency because the data is already in OpenSearch Service and can be immediately queried and visualized using the built-in OpenSearch Dashboards.
The other options involve additional steps and services that would introduce more latency:
- Option B uses Amazon Athena to query Avro data in S3, with Managed Grafana to create dashboards. While this can work, it adds query latency from Athena and does not leverage the data already in OpenSearch Service.
- Option C also uses Athena to query S3 data, with Amazon Keyspaces as the catalog and QuickSight for dashboards. This introduces additional latency and complexity compared to directly using the data in OpenSearch Service.
- Option D catalogs data with AWS Glue, queries S3 with S3 Select, and creates dashboards in QuickSight. This approach also adds latency and does not take advantage of the data already available in OpenSearch Service.
Therefore, creating OpenSearch Dashboards using the data from Amazon OpenSearch Service (Option A) provides the lowest latency solution for near real-time data visualization, given the company’s current architecture.
Amazon AWS Certified Data Engineer – Associate DEA-C01 certification exam assessment practice question and answer (Q&A) dump including multiple choice questions (MCQ) and objective type questions, with detail explanation and reference available free, helpful to pass the Amazon AWS Certified Data Engineer – Associate DEA-C01 exam and earn Amazon AWS Certified Data Engineer – Associate DEA-C01 certification.