AWS Certified Data Engineer - Associate

Category/Data Store Management

A company needs to partition the Amazon S3 storage that the company uses for a data lake. The partitioning will use a path of the S3 object keys in the following format: s3://bucket/prefix/year=2023/month=01/day=01 A data engineer must ensure that the AWS Glue Data Catalog synchronizes with the S3 storage when the company adds new partitions to the bucket. Which solution will meet these requirements with the LEAST latency?

Schedule an AWS Glue crawler to run every morning.

Manually run the AWS Glue CreatePartition API twice each day.

Use code that writes data to Amazon S3 to invoke the Boto3 AWS Glue create_partition API call.

Run the MSCK REPAIR TABLE command from the AWS Glue console.