The story started when you decided to save some data as JSON in the database. Your JSON data may change over time. Now what you do to the existing records that you have in a table? Well, you can really read all your JSON records and update it and then save it back to the database. This can be a solution, but you need to work with scripting that and dealing with JSON queries and other things you have to learn, which I have not found it fun myself.
Another complexity level is if you have couple of million records saved in the database as JSON. Lets say you save some user preferences as JSON in the database, and now this JSON structure have changes. Now what are our options? Bulk update all these records and deal with database scripting. But, are there any other options? In this post I will run you through doing this in a different way.
So the other way is an adhoc update of JSON upon request. Sounds interesting? It could, but it can be challenging as well if you do not have the tools to do it. I found the jackson library ( com.fasterxml.jackson) in java is exactly what are we looking for in a very structured way to accomplish our task.
An example of a JSON data that we have may start like this:
{date: "March,30 2018", .....}
Then the new version might be that we want to breakdown the date field to day, month and year into 3 separate fields. Something like this:
{day: "30", month: "3", year: "2018", .....}
Later on, we added an extra field called status
and we want to give it a default value open
by default when upgrading.
Next, I will be showing you how you can accomplish this, but you can also open your imagination in what kind of changes may happen to your JSON over time. Maybe add more fields, or remove fields, rename some fields, or even merge back our date example into one field, etc,…
We will create our parent core class Resource
, then we will create three versions of this class as follows: ResourceV1
, ResourceV2
, ResourceV3
, They represent the three versions of our JSON we mentioned above. Lets start with the Resource
class
package versioning;
import com.fasterxml.jackson.annotation.JsonSubTypes;
import com.fasterxml.jackson.annotation.JsonSubTypes.Type;
import com.fasterxml.jackson.annotation.JsonTypeInfo;
import com.fasterxml.jackson.annotation.JsonTypeInfo.As;@JsonTypeInfo(use=JsonTypeInfo.Id.NAME, include=As.PROPERTY, property="version", defaultImpl = Resource.class)@JsonSubTypes({
@Type(name="v1", value=ResourceV1.class),
@Type(name="v2", value=ResourceV2.class),
@Type(name="v3", value=ResourceV3.class)
})
public class Resource {}
You may have noticed we have an empty class, but in a few we will be writing the upgrade methods in this class. I just want to show you what is inside other classes.
ResourceV1
public class ResourceV1 extends Resource {
String date;
public String getDate() {
return date;
}
public void setDate(String date) {
this.date = date;
}
}
ResourceV2
public class ResourceV2 extends Resource {
Integer day;
String month;
Integer year;
public Integer getDay() {
return day;
}
public void setDay(Integer day) {
this.day = day;
}
public String getMonth() {
return month;
}
public void setMonth(String month) {
this.month = month;
}
public Integer getYear() {
return year;
}
public void setYear(Integer year) {
this.year = year;
}
}
Now if we look at ResourceV3
, we will notice that it inherits from ResourceV2
as they are the same with an extra field status
, which was not the case when we upgraded from ResourceV1
.
ResourceV3
public class ResourceV3 extends ResourceV2{
String status;
public String getStatus() {
return status;
}
public void setStatus(String status) {
this.status = status;
}
}
Lets make a flashback at the Resource
class which have these annotations:
@JsonTypeInfo(use=JsonTypeInfo.Id.NAME, include=As.PROPERTY, property="version", defaultImpl = Resource.class)@JsonSubTypes({
@Type(name="v1", value=ResourceV1.class),
@Type(name="v2", value=ResourceV2.class),
@Type(name="v3", value=ResourceV3.class)
})
These annotations is the magic that is doing the work for us. First annotation @JsonTypeInfo
tells jackson to find a property in the JSON string called version
and default the mapping to the Resource
class. The @JsonSubTypes
tells the Resource
class that it have three implementations. Once Jackson finds a value of version
, it will map it to the right child. Lets see some examples:
String json1 = "{\"date\" : \"March,30 2018\", \"version\": \"v1\"}";
ObjectMapper mapper = new ObjectMapper();
Resource resource = mapper.readValue(json1, Resource.class);
if (resource instanceof ResourceV1) // TRUE
If JSON value was :
String json2 = "{\"day\" : \"30\",\"month\": \"3\", \"year\": \"2018\",\"version\": \"v2\"}";
It will be mapped to ResourceV2
and resource instanceof ResourceV2
will be true.
The last case is when JSON was:
String json3 = "{\"day\" : \"30\",\"month\": \"3\", \"year\": \"2018\",\"status\": \"Open\" ,\"version\": \"v3\"}";
It will be mapped to ResourceV3
and resource instanceof ResourceV3
will be true.
Now this is great, Jackson was able to map different versions of our JSON to the correct class implementation. This is actually 70% of what we need to accomplish. What is left is on us to provide a mechanism to upgrade to the latest version. If what we are reading from the database is v1
, we will need to upgrade it to v2
then upgrade it to v3
. This is an incremental upgrade that we write over the time when our JSON change. For example, when we talk about the couple of millions user preferences that is saves as JSON, we will only upgrade when our user read their preferences. We will then save the latest version to their record. This helped us to not having to do a mass bulk upgrade for all of our users. Other than doing this, we only upgrade per request.
Now lets see how we wrote the upgrade methods. We place them in the main Resource
class. We will have 3 methods. Lets start with upgradeToV21
method. The method will break the one string date into three properties.
private ResourceV2 upgradeToV2(ResourceV1 resource){
ResourceV2 resource2 = new ResourceV2();
String[] breakdown = resource.getDate().split(",");
resource2.setMonth(breakdown[0].trim());
breakdown = breakdown[1].split("\\s+");
resource2.setDay(Integer.parseInt(breakdown[0].trim()));
resource2.setYear(Integer.parseInt(breakdown[1].trim()));
return resource2;
}
Then the upgradeToV3
method that will set a default value to the new property status:
private ResourceV3 upgradeToV3(ResourceV2 resource2){
ResourceV3 resource3 = new ResourceV3();
resource3.setDay(resource2.getDay());
resource3.setMonth(resource2.getMonth());
resource3.setYear(resource2.getYear());
resource3.setStatus("Open"); // Default value
return resource3;
}
No we will write the getLatest
method. It responsible for finding which JSON version we have, and upgrade it to the latest ResourceV3
version:
public ResourceV3 getLatest() throws UnsupportedEncodingException {
if (this instanceof ResourceV3) {
return (ResourceV3) this;
} else if (this instanceof ResourceV2) {
return this.upgradeToV3((ResourceV2) this);
} else if (this instanceof ResourceV1) {
return this.upgradeToV3(this.upgradeToV2((ResourceV1) this));
} else {
throw new UnsupportedEncodingException();
}
}
Now even if we passed a very old v1
of JSON, we can really get to v3
very easily and maybe save it to the current user. It can be as simple as the following:
String json1 = "{\"date\" : \"March,30 2018\", \"version\": \"v1\"}";
ObjectMapper mapper = new ObjectMapper();
Resource resource = mapper.readValue(json1, Resource.class);
ResourceV3 resource3 = resource.getLatest();
I hope this post was helpful and that is an answer to anyone who have faced a similar problem, or someone who is looking into saving any data as JSON and worried about it getting out of date.