Build An Ion Document From Scratch

In this section, we’ll take a document-oriented approach to building and manipulating Ion documents. We’ll start with an empty document and build it piece by piece using the Amazon Ion Java library. This approach is useful for situations where a Java object model would be overkill. We’ll end the section by writing the document to QLDB.

Build Ion Document

We’ll start where we left off in the last section. The App.java file should be open in the Cloud9 IDE and should look like this:

package lab2;

public class App {

    public static void main(String[] args) {
        System.out.println("Let's play with Ion!");
    }
}

We’ll use the App class for all of the exercises in this workshop. Let’s start off by creating a method for this exercise called buildAndWriteIon() that is invoked from the main() method. We’ll also remove the print statement from the previous exercise.

package lab2;

public class App {

    public static void main(String[] args) throws Exception {
        buildAndWriteIon();
    }


    private static void buildAndWriteIon() throws Exception {
    }
}

Now import classes from the Ion library. Note that we’ll take shortcuts like importing wildcards and throwing Exception throughout the workshop for convenience. The workshop code is not intended for production use.

package lab2;

import com.amazon.ion.*;
import com.amazon.ion.system.*;
import com.amazon.ion.util.*;


public class App {
  ...
}

Compile and run the program just to make sure we’re starting from a good baseline.

gradle run

To begin using the Ion APIs to create a new document, we must first initialize Ion using the IonSystem class. The Ion Javadoc describes IonSystem as:

The central interface in ion-java is IonSystem, which is the main factory and facade for all Ion processing.

Insert the following line into the buildAndWriteIon() method:

IonSystem ionSys = IonSystemBuilder.standard().build();

Start a new Ion document by using the IonSystem to create an IonStruct. The IonStruct is a container element and this first element will represent our entire Ion document. Insert the following line of code at the end of the buildAndWriteIon() method:

IonStruct personDocument = ionSys.newEmptyStruct();

Let’s print our new Ion document to the console. IonStruct inherits a toPrettyString() method from its ancestor IonValue, the root class for all Ion data nodes. We’ll use toPrettyString() to print a nicely-formatted copy of our document. Insert the following line of code at the end of the buildAndWriteIon() method.

System.out.println(personDocument.toPrettyString());

Our buildAndWriteIon() method should now look like this:

private static void buildAndWriteIon() throws Exception {
    IonSystem ionSys = IonSystemBuilder.standard().build();

    IonStruct personDocument = ionSys.newEmptyStruct();
    System.out.println(personDocument.toPrettyString());
}

Compile and run the program by entering gradle run into the terminal window in Cloud9. In addition to a lot of output (and possible deprecation warnings) from the build process, the program should have printed an empty set of curly braces for our empty Ion document.

Admin:~/environment/lab2 $ gradle run

> Task :run

{
}

...

As you may have inferred from the name of our IonStruct variable, we’re creating an Ion document that contains data about a Person (you may also remember that we created a Person table in our QLDB ledger in a previous exercise). Let’s add a few string fields to our document that contain more information about the Person. We do this by creating IonString objects and adding them to our IonStruct container. There are a couple of ways to do this, but we’ll use a convenience offered by IonStruct that lets us create and append the new string in one line of code. Add the following lines of code just before System.out.println() call.

personDocument.put("PersonId").newString("123456789");
personDocument.put("FirstName").newString("John");
personDocument.put("LastName").newString("Doe");

Note that we created an index on the PersonId field in the Person table of our QLDB ledger.

Compile and run the program to see what our document looks like. You should see something like this:

{
  PersonId:"123456789",
  FirstName:"John",
  LastName:"Doe"
}

Let’s use a few more data types in our Person document. We’ll track the amount of money in our Person’s wallet using an IonDecimal value. To do this, add the following to the import section at the top of the App.java program:

import java.math.BigDecimal;

We’ll use the same convenience method to add our IonDecimal to our Ion document. Add the following line to our program:

personDocument.put("MoneyInWallet").newDecimal(new BigDecimal("31.24"));

Compile and run the program to see our Ion document.

{
  PersonId:"123456789",
  FirstName:"John",
  LastName:"Doe",
  MoneyInWallet:31.24
}

Add a date to our document. The IonValue sub-class for dates and times is IonTimestamp. Ion has its own Timestamp class to represent a timestamp, so we’ll create one of those first.

IonTimestamp is a data type offered by Ion that is not available in JSON, so understanding to create and manipulate them will be important if you need to convert to and from JSON. We’ll cover this in later exercises. For now, add the following line to our buildAndWriteIon() method and compile and run the program to see what it does.

personDocument.put("DateOfBirth").newTimestamp(Timestamp.forDay(1970, 7, 4));

Note the timestamp in our output. It is not enclosed in quotes, indicating that it is a timestamp value, not a string.

{
  PersonId:"123456789",
  FirstName:"John",
  LastName:"Doe",
  MoneyInWallet:31.24,
  DateOfBirth:1970-07-04
}

Now add integer (IonInt) and boolean (IonBool) values to our document.

personDocument.put("NumberOfLegs").newInt(2);
personDocument.put("LikesGreenBeans").newBool(false);

Run the program and check the output.

{
  PersonId:"123456789",
  FirstName:"John",
  LastName:"Doe",
  MoneyInWallet:31.24,
  DateOfBirth:1970-07-04,
  NumberOfLegs:2,
  LikesGreenBeans:false
}

Now we’ll add a list element to our Ion document. IonList is a container type like IonStruct. You can add any object that inherits from IonValue to an IonList, including other lists and structures. We’ll add some simple strings to our list.

Here we create an empty list and add it to our Person document. Then we add several string values to the list.

IonList items = personDocument.put("ThingsInPocket").newEmptyList();
items.add().newString("keys");
items.add().newString("pocketknife");
items.add().newString("lint");
items.add().newString("pack of gum");

This gives us the following output:

{
  PersonId:"123456789",
  FirstName:"John",
  LastName:"Doe",
  MoneyInWallet:31.24,
  DateOfBirth:1970-07-04,
  NumberOfLegs:2,
  LikesGreenBeans:false,
  ThingsInPocket:[
    "keys",
    "pocketknife",
    "lint",
    "pack of gum"
  ]
}

Note that lists can contain values of different types.

Now we’ll add some nested structured elements to our Person document. These elements will be of type IonStruct, the same as our root Person document. We’ll create two nested structures and we’ll use two slightly different mechanisms to create and add them to our Person document.

Add the code below to the buildAndWriteIon() method. This method uses the same convenience method that we used to add the other value types to our document. We first create and add an empty IonStruct to the Person document (homeAddress) before adding values to it.

IonStruct homeAddress = personDocument.put("HomeAddress").newEmptyStruct();
homeAddress.put("Street1").newString("123 Main Street");
homeAddress.put("City").newString("Beverly Hills");
homeAddress.put("State").newString("CA");
homeAddress.put("Zip").newString("90210");

Now we’ll add another IonStruct (workAddress) to our Person document, but we’ll use a slightly different technique. First, we’ll create an empty IonStruct using the IonSystem object that we initialized at the top of our buildAndWriteIon() method. This IonStruct is not yet attached to our Person document. We then attach some values to our workAddress sub-document. Finally, we add the populated workAddress sub-document to the Person document with a put() method that accepts an IonValue parameter.

IonStruct workAddress = ionSys.newEmptyStruct();
workAddress.put("Street1").newString("12 Elm Street");
workAddress.put("City").newString("Los Angeles");
workAddress.put("State").newString("CA");
workAddress.put("Zip").newString("90001");
personDocument.put("WorkAddress", workAddress);

Running the program gives us our complete document.

{
  PersonId:"123456789",
  FirstName:"John",
  LastName:"Doe",
  MoneyInWallet:31.24,
  DateOfBirth:1970-07-04,
  NumberOfLegs:2,
  LikesGreenBeans:false,
  ThingsInPocket:[
    "keys",
    "pocketknife",
    "lint",
    "pack of gum"
  ],
  HomeAddress:{
    Street1:"123 Main Street",
    City:"Beverly Hills",
    State:"CA",
    Zip:"90210"
  },
  WorkAddress:{
    Street1:"12 Elm Street",
    City:"Los Angeles",
    State:"CA",
    Zip:"90001"
  }
}

Our buildAndWriteIon() method should now look like this:

private static void buildAndWriteIon() throws Exception {
    IonSystem ionSys = IonSystemBuilder.standard().build();

    IonStruct personDocument = ionSys.newEmptyStruct();

    personDocument.put("PersonId").newString("123456789");
    personDocument.put("FirstName").newString("John");
    personDocument.put("LastName").newString("Doe");
    personDocument.put("MoneyInWallet").newDecimal(new BigDecimal("31.24"));
    personDocument.put("DateOfBirth").newTimestamp(Timestamp.forDay(1970, 7, 4));
    personDocument.put("NumberOfLegs").newInt(2);
    personDocument.put("LikesGreenBeans").newBool(false);

    IonList items = personDocument.put("ThingsInPocket").newEmptyList();
    items.add().newString("keys");
    items.add().newString("pocketknife");
    items.add().newString("lint");
    items.add().newString("pack of gum");

    IonStruct homeAddress = personDocument.put("HomeAddress").newEmptyStruct();
    homeAddress.put("Street1").newString("123 Main Street");
    homeAddress.put("City").newString("Beverly Hills");
    homeAddress.put("State").newString("CA");
    homeAddress.put("Zip").newString("90210");

    IonStruct workAddress = ionSys.newEmptyStruct();
    workAddress.put("Street1").newString("12 Elm Street");
    workAddress.put("City").newString("Los Angeles");
    workAddress.put("State").newString("CA");
    workAddress.put("Zip").newString("90001");
    personDocument.put("WorkAddress", workAddress);

    System.out.println(personDocument.toPrettyString());
}

Write to QLDB

Now that we’ve created an Ion document representing a Person, we’ll write it to Person table in our QLDB ledger.

We’ll need to import all of the classes we need from the QLDB driver and the AWS SDK. Add the statements below to the import section at the top of our App.java file.

import software.amazon.awssdk.services.qldbsession.*;
import software.amazon.qldb.*;

import java.util.Iterator;

Initialize the QLDB driver with the lines of code below. Place the code in our buildAndWriteIon() method beneath where we pretty-print our Ion document. We named our QLDB ledger “ion-lab” in a previous exercise. If you used a different name for your ledger, change the value used in the driver initialization below. This code uses a credentials file on the Cloud9 EC2 instance for authentication so that you don’t have to explicitly provide credentials. Cloud9 populates the credentials file for you. To learn more about explicitly setting credentials, see the AWS Java SDK documentation.

QldbSessionClientBuilder sessionClientBuilder = QldbSessionClient.builder();
QldbDriver driver = QldbDriver
  .builder()
  .ledger("ion-lab")
  .sessionClientBuilder(sessionClientBuilder)
  .build();

Now execute the query to insert the Ion document into QLDB. The code below uses a Java lambda expression to simplify the execute() call on the QLDBDriver (NOTE: this is not the same as an AWS Lambda function). The expression itself implements the ‘Executor’ interface. The txn parameter implements TransactionExecutor, which provides execute() methods that return a Result object.

The code below calls an execute() method that accepts a parameterized query statement and an IonValue parameter that may contain one or more parameter values. We’re using the syntax of the PartiQL INSERT statement that accepts a single, complete Ion document and we’re passing the entire document we’ve created as a parameter.

Result result = driver.execute(txn -> {
      return txn.execute("INSERT INTO Person VALUE ?", personDocument);
  });

The Result object may contain multiple IonValue objects. We access those objects by iterating over them with an Iterator. The INSERT statement returns a single Ion document, an IonStruct, that contains a document ID in a string field. The document ID is a unique identifier metadata field that QLDB creates for each document in the ledger. The code below extracts the document ID from the Result object and prints it to the console.

String documentId = null;
Iterator<IonValue> iter = result.iterator();
while (iter.hasNext()) {
    IonValue obj = iter.next();
    if (obj instanceof IonStruct) {
        IonStruct val = (IonStruct) obj;
        IonString str = (IonString) val.get("documentId");
        documentId = str.stringValue();
        break;
    }
}

System.out.println("\n\nInserted Person document:  " + documentId + "\n\n");

Compile and run the program. The output will contain the entire Ion document pretty-printed as before as well as output containing the ID of the document that was inserted into QLDB. That output should look something like this (the actual document ID will be different).

Inserted Person document:  0FyBytEhbbOKOFnvyMr4Kq

Your App.java file should now look like this:

package lab2;

import com.amazon.ion.*;
import com.amazon.ion.system.*;
import com.amazon.ion.util.*;

import software.amazon.awssdk.services.qldbsession.*;
import software.amazon.qldb.*;

import java.math.BigDecimal;
import java.util.Iterator;


public class App {

    public static void main(String[] args) throws Exception {
        buildAndWriteIon();
    }


    private static void buildAndWriteIon() throws Exception {
        IonSystem ionSys = IonSystemBuilder.standard().build();

        IonStruct personDocument = ionSys.newEmptyStruct();

        personDocument.put("PersonId").newString("123456789");
        personDocument.put("FirstName").newString("John");
        personDocument.put("LastName").newString("Doe");
        personDocument.put("MoneyInWallet").newDecimal(new BigDecimal("31.24"));
        personDocument.put("DateOfBirth").newTimestamp(Timestamp.forDay(1970, 7, 4));
        personDocument.put("NumberOfLegs").newInt(2);
        personDocument.put("LikesGreenBeans").newBool(false);

        IonList items = personDocument.put("ThingsInPocket").newEmptyList();
        items.add().newString("keys");
        items.add().newString("pocketknife");
        items.add().newString("lint");
        items.add().newString("pack of gum");

        IonStruct homeAddress = personDocument.put("HomeAddress").newEmptyStruct();
        homeAddress.put("Street1").newString("123 Main Street");
        homeAddress.put("City").newString("Beverly Hills");
        homeAddress.put("State").newString("CA");
        homeAddress.put("Zip").newString("90210");

        IonStruct workAddress = ionSys.newEmptyStruct();
        workAddress.put("Street1").newString("12 Elm Street");
        workAddress.put("City").newString("Los Angeles");
        workAddress.put("State").newString("CA");
        workAddress.put("Zip").newString("90001");
        personDocument.put("WorkAddress", workAddress);

        System.out.println(personDocument.toPrettyString());

        QldbSessionClientBuilder sessionClientBuilder = QldbSessionClient.builder();
        QldbDriver driver = QldbDriver
          .builder()
          .ledger("ion-lab")
          .sessionClientBuilder(sessionClientBuilder)
          .build();   

        Result result = driver.execute(txn -> {
              return txn.execute("INSERT INTO Person VALUE ?", personDocument);
        });

        String documentId = null;
        Iterator<IonValue> iter = result.iterator();
        while (iter.hasNext()) {
            IonValue obj = iter.next();
            if (obj instanceof IonStruct) {
                IonStruct val = (IonStruct) obj;
                IonString str = (IonString) val.get("documentId");
                documentId = str.stringValue();
                break;
            }
        }

        System.out.println("\n\nInserted Person document:  " + documentId + "\n\n");        
    }
}

If you are having trouble getting your program to run, click here to download the complete App.java file for this lab.