A Java microservice for image classification using Thorntail and DeepLearning4J

If you want to consume a trained neural network from an application you will probably not want to package the model within your app. Models usually occupies ~50mb of disk size, see these keras trained models for example.
A good approach for allowing other applications to use your model is a separated microservice and, if you are a Java developer, Thorntail is the best choice:

• Native CDI;
• Native Eclipse Microprofile support;
• Seamless Integration with JAX-RS;
• Plugin for creating docker images

For image classification nowadays we use CNN neural networks and for Java developers DeepLearning4J is the API you are looking for. You can consume some of the multiple pre-trained models, build your own model or even use a Keras Model!

So yes, we need a microservice for image classification. The microservice should be responsible to run a model that we configure and if we don't provide a model, it should use a default one. It should be accessible using REST and it should cache the model.

Enough cheap talk, let's build our microservice..

The REST interface

We will have only two methods for the API:

• GET  /:  Returns the model information  in JSON format. Sample Response:

• POST /: Accept the binaries of an image and return the classification in JSON format; sample response:

I used jq to highlight curl's output

The Java project

We will use Maven and pom.xml will have the following dependencies:

The dependencies are simplified mainly thanks to Thorntail bring natively microprofile IO and CDI. For dl4j we have the nd4j, the base library for dl4j APIs, core and zoo, from where we will take the default model from. We also use the thorntail plugin which builds a structure that can be used to run our application in production or in a Docker image, this structure will be useful for us later as well.

Our project will have only 3 parts:

• Model: The model objects that contains the JSON representation
• Service: The project heart that will run the model
• REST: The part where we expose 

Users can use their own model, so we use Microprofile Config to inject properties. Service is a CDI bean that will load the model and run the classification. REST is a single JAX-RS resource and the model classes are explained by the JSON outputs you saw later. See the full code on my github.

Testing and Running

There are two tests. By default the microservice will use the Resnet50 DeepLerning4J built in model from the Zoo, which is trained using ImageNet dataset, so users have recognition of 1000 classes without using a custom model.The first test is simply to make sure that the default model is working. The second test is against a custom model, we use a small MNIST model.

Running the application is simple, first you build it, you can even skip tests:

$ mvn clean install -DskipTests

First time you run the tests will take a long time because it will download the default model. Once it is built run the script in $PROJECT_HOME/target/image-classifier-microservice-1.0-bin/bin. Use bat or sh according to your operating system.

Now, to run with a custom model this gets interesting. Adding your custom model requires you to paste it in $PROJECT_HOME/target/image-classifier-microservice-1.0-bin/lib and then set the following java properties to configure it:

model.path: The file name which should be in the application classpath. If you don't provide a model the built it one is used;

model.labels: The labels for the model separated by comma (,). It is highly recommend to provide a value for it, the application will work if you don't provide, but the labels will be like "label 1", etc;

model.type:  A name that you can give to this model;

model.input.width: The input width for the model. Default is 224.

model.input.height: The input height for the model. Default is 224.

model.input.channels: The number of channels. Default is 3.

If you is calling the application from Java you can set these properties using System.setProperty, you can fork the code, modify it as you want and have a main class to call Thorntail.run from a main method and set the system properties according to your application. If you don't want to touch Java code you need to follow the steps below:

•  Have a built version of the project;
• Paste your model into $PROJECT_HOME/target/image-classifier-microservice-1.0-bin/lib
• Set the system property JAVA_OPTS before running the script to start the server for example:

export JAVA_OPTS="-Dmodel.path=/mnist-model.zip -Dmodel.labels=0,1,2,3,4,5,6,7,8,9 -Dmodel.type=MNIST -Dmodel.input.width=28 -Dmodel.input.height=28 -Dmodel.input.channels=1"

If everything is fine you should see the following in logs when starting the server:

Conclusion 

We described a simple, but useful, microservice for image classification. There are improvements that could be done:

• Performance: The performance could be improved in many different ways;
• Configuration: Make easier to configure the model I/O. Currently we consider that the image will be converted to an INDArray without any additional transformation and the output is considered to be of shape [1 X ] where X is the number of labels, which may not be always the case;
• Enrich the interface: Right now we just provide the minimal information to use the model. This could be improved.

Currently the project is not ready for every type of model, in future we may extend it to support all the famous pre-trained models out there, who knows.We are also open for collaboration, send me your PR if you think anything that could be improved.

The source is on my github.