A company has three subsidiaries. Each subsidiary uses a different data warehousing solution. The first subsidiary hosts its data warehouse in Amazon Redshift. The second subsidiary uses Teradata Vantage on AWS. The third subsidiary uses Google BigQuery. The company wants to aggregate all the data into a central Amazon S3 data lake. The company wants to use Apache Iceberg as the table format. A data engineer needs to build a new pipeline to connect to all the data sources, run transformations by using each source engine, join the data, and write the data to Iceberg. Which solution will meet these requirements with the LEAST operational effort?
Select an option, then click Submit answer.
Reference / correct answer:
Use the Amazon Athena federated query connectors for Amazon Redshift, Teradata, and BigQuery to build the pipeline in Athena. Write a SQL query to read from all the data sources, join the data, and run a Merge operation on the data lake Iceberg table.
Most accepted answer: B. Use the Amazon Athena federated query connectors for Amazon Redshift, Teradata, and BigQuery to build the pipeline in Athena. Write a SQL query to read from all the data sources, join the data, and run a Merge operation on the data lake Iceberg table.
Community votes: A=5, B=6, D=1
Selected Answer: B Uses Amazon Athena with federated connectors. Advantages: -Athena can query: Redshift, Teradata, BigQuery -Executes SQL across sources. -Each engine processes its own data where possible. -Athena supports Apache Iceberg tables in S3. -No cluster management. -No ETL infrastructure. -This is serverless and lowest operational overhead. upvoted 1 times
Selected Answer: B B) Answer B: 1) Athena with federated connectors ( Redshift, Teradata via JDBC connector, BigQuery connector). No cluster to run. 2) You write one SQL to transform/join across all three. 3) Athena supports MERGE into Inceberg tables so that you can materialise results directly in the S3 Data Lake. upvoted 2 times
Selected Answer: B Athena supports federated queries, including connectors for Redshift, Teradata, and BigQuery. Queries can push down operations to each source engine (meeting the “run transformations using each source engine” requirement). Athena natively supports Iceberg tables on Amazon S3, including MERGE INTO operations. Very low operational overhead: only SQL queries, no infrastructure to manage. ✅ Best fit. upvoted 1 times
Selected Answer: A Glue, for sure. Athena is an ad-hoc querying tool, not and ETL tool and besides doesn't have connectors for Bigquery and Terradata! upvoted 3 times
Selected Answer: A Native Connectors: AWS Glue provides built-in connectors for Amazon Redshift, Teradata, and Google BigQuery. This eliminates the need for custom-built connectors, reducing development and maintenance overhead. upvoted 2 times