Create S3 Export

In this section, we’ll export our QLDB ledger data into S3 so that we may crawl it with Glue and access it with Athena. First, we’ll create the S3 bucket that our ledger data will be exported to.

Create S3 Bucket

We’ll create all of our infrastucture with CloudFormation in your desired region.

This template will be used to create the following items:

  • S3 Bucket
  • Lambda functions
  • IAM Roles
Region Launch CloudFormation Template
US East (Virginia) Launch Stack in us-east-1
US East (Ohio) Launch Stack in us-east-2
US West (Oregon) Launch Stack in us-west-2
Europe (Frankfurt) Launch Stack in eu-central-1

or, download the file to your local workstation and create a CloudFormation stack by uploading the template.

You will see the Quick create stack page as shown below. In the Stack name block, leave the Stack name as qldb-athena-lab-create-s3.

Leave the Parameters block as-is as no parameters are needed for this template.

Check the “I acknowledge that CloudFormation might create IAM resources.” box and click Create stack.

Create Stack

The stack will take several minutes to create. Its status will be updated to CREATE_COMPLETE.

Click the Outputs tab and note the name of the S3 bucket that was created, denoted by the QLDBExportBucketName output value.

Stack Outputs

Create Export Job

Return to the main page of the AWS console by clicking the AWS logo in the upper-left hand corner of any console page. Navigate to Amazon QLDB by finding it under Databases in the list of services or you can find it by typing it in the search bar at the top.

Find Services

In left-hand navigation of the QLDB homepage, click Export.

QLDB Console

Click the Create export job button to get started.

Export Page

The Create export job screen will appear as shown below.

In the Journal blocks section of the export page, select the “vehicle-registration” ledger.

Enter 1900/01/01 and 00:00:00 in the **End date and time (UTC)** fields.

Enter the current date and time in the **End date and time (UTC)** fields. The time should be provided as a UTC time and cannot be in the future. You can get the current date and time in UTC from this website: https://www.timeanddate.com/worldclock/timezone/utc.

In the Write journal blocks to S3 section, click the Browse S3 button. Select the S3 bucket you created with your CloudFormation template and click the Choose button. If you’re not sure of the bucket that you created, check the Outputs section of the CloudFormation stack you created, as described above.

Journal Blocks

In the Grant service access to s3 section. Click on Create and use a new service role. Ensure that the S3 bucket name is populated with the correct s3 bucket.

Journal Blocks

Click Create export job button.

After a few moments, you will see that the export job has completed.

Export Completed

View export files

Return to the main page of the AWS console by clicking the AWS logo in the upper-left hand corner of any console page. Navigate to S3 by finding it under Storage in the list of services or you can find it by typing S3 in the search bar at the top.

Find Services

In the list of buckets, click on the bucket created in the CloudFormation stack above. If you’re not sure of the name of the bucket that you created, check the Outputs section of the CloudFormation stack you created, as described above.

Click through the directories in the export bucket and explore the files exported by QLDB.