Creating Java classes from JSON dynamically.
The need,history…
We were designing a new microservice recently and decided to try a new approach instead of using a shared lib for data structures we use commonly. As you know from many projects, some interfaces and classes are being shared as proxy classes mostly with a naming convention as shared.jar/sharedlib… and similar. DTOs (Data Transfer Objects) are typical examples.
It’s another discussion in DDD (Domain Driven Design) having shared libs, but it’s also an undeniable fact as we need shared things at intersections, and it’s acceptable in some level. Duplication is an option but we thought we could generate the classes dynamically for our service. I admit it’s not possible or the best practice for all projects but it’s an exact cut for our need.
Shared libs could be a bit troublesome for this particular mircoservice. Possible changes in proxy libs needs to be versioned and for every release there must be a concord among all. Moreover, we really wished this micro-service be able to serve to any other service or app written in any language, working with any data-set...
An idea, one solution..
REST API is a standard for almost all apps today, and it can be implemented in different serialization ways.However, we also see that JSON is the most common format community preferred. So, if we can convert JSON serialized objects to Java classes dynamically. This might help us in our problem.
Let’s try,
There are libraries serializing JSON objects to known classes in many programming languages, jackson is the dominant one in Java and JSON.NET is the popular one in .net stack.
There are even some websites converting JSON to Java or C# online. If you check these sites, you’ll figure out they create java classes either from a backend service or by using javascript directly. In both cases there is only string manipulation… That is it. Why can’t we do the same !
The solution is quite simple then, we can create the java source code, compile it on the fly and add it into JVM context. Of course we’ll need a custom class loader, a compiler app and a little reflection trick. After all, there is no need to share a sharedlib.jar, but instead the json serialized class will be converted to java source and define our class structure dynamically. In short, {JSON Obj} -> className.java -> className.class.
If you check json to java source converting websites you see that you need to enter package name and class name for your class generation. Import namespace and class name called for the .net world. As we all know, Java Script Object Notation doesn’t give us any class name. We map it to our known classes. So, what will be our className ?
From the beginning we designed our base objects self descriptive. We have className attribute in our base object and its value is always the full name of the class.
{ className:”com.medium.write.up”}
You got the idea, we always have the package name and class name with this base attribute for all objects in our environment already. It’s basically self descriptive . You can improve it with class version to support versioning based on your need.
Our Dynamic Java Source Map finally looked like this thing below. A source code object and its composite nested members. Source code is a map, because it can refer to other classes and they can refer to others and so on. The limitation here is any member must have self descriptive attribute (here className) or must be an already known type by JVM.
Another important point is you need to start from the deepest node to start creating unknown objects dynamically in your environment and make sure it’s not generated and added to your JVM environment before. To achieve that we need a thread safe object map to store class name and class object pairs.
Precisely Map<String, javax.tools.SimpleJavaFileObject>.
If our className is in this map we instantiate from it otherwise we generate required class and go to upper levels in the node tree to generate the desired root object.
LIMITATION, JDK,..
Since we dynamically build java sources into classes we need javac java compiler. It comes with JDK, but not JRE. So, we need to run this app with JDK. Otherwise the classLoaders we use and the things written under interfaces won’t reach to concrete classes and we’ll face the most famous NullPointerException. This means there is also proxy between JRE and JDK. So, Just run it with a JDK. You will see that it works! I don’t mean it’s not compiling with JRE, it’s compiling in both cases but runs with JDK !
Don’t forget to put className attribute for every object in your JSON, including nested objects.
Example JSON,
Here we have a.b.c.DynaClass, has nested object a.b.c.NestedClass, and that has 3 other sub objects.
Our project needs to create NestedClassFoo,NestedClassBar an NestedClassBaz at first place. Because NestedClass depends on them, then NestedClass and finally DynaClass will be generated. While using a recursive function we need to ensure any reference is not duplicating in the JVM. Because one of the member can be referenced in different classes.
{
“type”:”object”,
“className”:”a.b.c.DynaClass”,
“properties”:{
“className”:”a.b.c.NestedClass”,
“foo”:{
“className”:”a.b.c.NestedClassFoo”,
“var1”:”string”
},
“bar”:{
“className”:”a.b.c.NestedClassBar”,
“var2”:”integer”
},
“baz”:{
“className”:”a.b.c.NestedClassBaz”,
“var3”:”boolean”
}
}
}
Test Flight,
Here is the open source page of the project, give it a try.
It’s a SpringBoot project. You can either post above JSON to your http://localhost:9090/json2java/json endpoint or simply run the JUnit test case to see how JSON turn to java source and be compiled to a class in the runtime.
Looking forward for your pull requests as well,
P.S. Don’t forget to run with JDK and never miss className attribs for objects ;)
Happy coding !