Using Jackson to keep JSON data up to date

The story started when you decided to save some data as JSON in the database. Your JSON data may change over time. Now what you do to the existing records that you have in a table? Well, you can really read all your JSON records and update it and then save it back to the database. This can be a solution, but you need to work with scripting that and dealing with JSON queries and other things you have to learn, which I have not found it fun myself.

Another complexity level is if you have couple of million records saved in the database as JSON. Lets say you save some user preferences as JSON in the database, and now this JSON structure have changes. Now what are our options? Bulk update all these records and deal with database scripting. But, are there any other options? In this post I will run you through doing this in a different way.

So the other way is an adhoc update of JSON upon request. Sounds interesting? It could, but it can be challenging as well if you do not have the tools to do it. I found the jackson library ( com.fasterxml.jackson) in java is exactly what are we looking for in a very structured way to accomplish our task.

An example of a JSON data that we have may start like this:

{date: "March,30 2018", .....}

Then the new version might be that we want to breakdown the date field to day, month and year into 3 separate fields. Something like this:

{day: "30", month: "3", year: "2018", .....}

Later on, we added an extra field called status and we want to give it a default value open by default when upgrading.

Next, I will be showing you how you can accomplish this, but you can also open your imagination in what kind of changes may happen to your JSON over time. Maybe add more fields, or remove fields, rename some fields, or even merge back our date example into one field, etc,…

We will create our parent core class Resource, then we will create three versions of this class as follows: ResourceV1, ResourceV2, ResourceV3, They represent the three versions of our JSON we mentioned above. Lets start with the Resource class

package versioning;
import com.fasterxml.jackson.annotation.JsonSubTypes;
import com.fasterxml.jackson.annotation.JsonSubTypes.Type;
import com.fasterxml.jackson.annotation.JsonTypeInfo;
import com.fasterxml.jackson.annotation.JsonTypeInfo.As;
@JsonTypeInfo(use=JsonTypeInfo.Id.NAME, include=As.PROPERTY, property="version", defaultImpl = Resource.class)
@JsonSubTypes({
    @Type(name="v1", value=ResourceV1.class),
    @Type(name="v2", value=ResourceV2.class),
    @Type(name="v3", value=ResourceV3.class)
})
public class Resource {}

You may have noticed we have an empty class, but in a few we will be writing the upgrade methods in this class. I just want to show you what is inside other classes.

ResourceV1

public class ResourceV1 extends Resource {
   String date;
   public String getDate() {
      return date;
   }
   public void setDate(String date) {
      this.date = date;
   }
}

ResourceV2

public class ResourceV2 extends Resource {
   Integer day;
   String month;
   Integer year;
   public Integer getDay() {
      return day;
   }
   public void setDay(Integer day) {
      this.day = day;
   }
   public String getMonth() {
      return month;
   }
   public void setMonth(String month) {
      this.month = month;
   }
   public Integer getYear() {
      return year;
   }
   public void setYear(Integer year) {
      this.year = year;
   }
}

Now if we look at ResourceV3, we will notice that it inherits from ResourceV2 as they are the same with an extra field status, which was not the case when we upgraded from ResourceV1.

ResourceV3

public class ResourceV3 extends ResourceV2{
   String status;
   public String getStatus() {
      return status;
   }
   public void setStatus(String status) {
      this.status = status;
   }
}

Lets make a flashback at the Resource class which have these annotations:

@JsonTypeInfo(use=JsonTypeInfo.Id.NAME, include=As.PROPERTY, property="version", defaultImpl = Resource.class)
@JsonSubTypes({
    @Type(name="v1", value=ResourceV1.class),
    @Type(name="v2", value=ResourceV2.class),
    @Type(name="v3", value=ResourceV3.class)
})

These annotations is the magic that is doing the work for us. First annotation @JsonTypeInfo tells jackson to find a property in the JSON string called version and default the mapping to the Resource class. The @JsonSubTypes tells the Resource class that it have three implementations. Once Jackson finds a value of version, it will map it to the right child. Lets see some examples:

String json1 = "{\"date\" : \"March,30 2018\", \"version\": \"v1\"}";
ObjectMapper mapper = new ObjectMapper();
Resource resource = mapper.readValue(json1, Resource.class);
if (resource instanceof ResourceV1) // TRUE

If JSON value was :

String json2 = "{\"day\" : \"30\",\"month\": \"3\", \"year\": \"2018\",\"version\": \"v2\"}";

It will be mapped to ResourceV2 and resource instanceof ResourceV2 will be true.

The last case is when JSON was:

String json3 = "{\"day\" : \"30\",\"month\": \"3\", \"year\": \"2018\",\"status\": \"Open\" ,\"version\": \"v3\"}";

It will be mapped to ResourceV3 and resource instanceof ResourceV3 will be true.

Now this is great, Jackson was able to map different versions of our JSON to the correct class implementation. This is actually 70% of what we need to accomplish. What is left is on us to provide a mechanism to upgrade to the latest version. If what we are reading from the database is v1, we will need to upgrade it to v2 then upgrade it to v3. This is an incremental upgrade that we write over the time when our JSON change. For example, when we talk about the couple of millions user preferences that is saves as JSON, we will only upgrade when our user read their preferences. We will then save the latest version to their record. This helped us to not having to do a mass bulk upgrade for all of our users. Other than doing this, we only upgrade per request.

Now lets see how we wrote the upgrade methods. We place them in the main Resource class. We will have 3 methods. Lets start with upgradeToV21 method. The method will break the one string date into three properties.

private ResourceV2 upgradeToV2(ResourceV1 resource){
   ResourceV2 resource2 = new ResourceV2();
   String[] breakdown = resource.getDate().split(",");
   resource2.setMonth(breakdown[0].trim());
   breakdown = breakdown[1].split("\\s+");
   resource2.setDay(Integer.parseInt(breakdown[0].trim()));
   resource2.setYear(Integer.parseInt(breakdown[1].trim()));
   return resource2;
}

Then the upgradeToV3 method that will set a default value to the new property status:

private ResourceV3 upgradeToV3(ResourceV2 resource2){
   ResourceV3 resource3 = new ResourceV3();
   resource3.setDay(resource2.getDay());
   resource3.setMonth(resource2.getMonth());
   resource3.setYear(resource2.getYear());
   resource3.setStatus("Open"); // Default value
   return resource3;
}

No we will write the getLatest method. It responsible for finding which JSON version we have, and upgrade it to the latest ResourceV3 version:

public ResourceV3 getLatest() throws UnsupportedEncodingException {
   if (this instanceof ResourceV3) {
      return (ResourceV3) this;
   } else if (this instanceof ResourceV2) {
      return this.upgradeToV3((ResourceV2) this);
   }  else if (this instanceof ResourceV1) {
      return this.upgradeToV3(this.upgradeToV2((ResourceV1) this));
   } else {
      throw new UnsupportedEncodingException();
   }
}

Now even if we passed a very old v1 of JSON, we can really get to v3 very easily and maybe save it to the current user. It can be as simple as the following:

String json1 = "{\"date\" : \"March,30 2018\", \"version\": \"v1\"}";
ObjectMapper mapper = new ObjectMapper();
Resource resource = mapper.readValue(json1, Resource.class);
ResourceV3 resource3 = resource.getLatest();

I hope this post was helpful and that is an answer to anyone who have faced a similar problem, or someone who is looking into saving any data as JSON and worried about it getting out of date.