Pular para o conteúdo principal

Postagens

Visualize a neural network activation using JavaFX

Sometimes you want to visualize what is going on inside your neural network to identify what you could improve. This is not an easy task and may require patience. To help on this task I am sharing today a small JavaFX application that will go through all the activated layers and show the activated output and the ones that were not updated. The results are interesting, see some screenshots I took using the Brazilian coin model , which was trained on a deep pre-trained Resnet50 neural network: At the early layers we can see some sort of the shadow of the coin In the middle we can see some features that were activated Close to the end only some outputs were activated It is confusing when we check these huge third party neural networks. It is more interesting when we inspect some neural network we created ourselves. Remember when we used a trained MNIST model in a JavaFX application ? These are the layers after an image classification: This was possible using the ac...

Generic Telegram bot for image classification using DL4J

This might be useful for someone else. Using the TelegramBots Java API I build a generic bot which you can use for any ComputationGraph you exported from a DeepLearning4J application. The bot simple gets any image it receives and use the model to predict an output. All you have to do is ask for a BotFather  key and set a few system properties: bot.username : Your bot userame bot.token : The bot token you got from  BotFather classifier.labels : The model labels separated by comma classifier.modelpath : The full filesystem path to the model classifier.inputformat : Input image height, width and number of channels separated by comma How it works: First you train your model using DeepLearning4J API and export it . See as example what we did in our Brazilian coin classification post ; Second you get a bot key using  BotFather ; Now you can clone the code from github and build it using mvn clean package (I am considering you already have maven); Finally...

Brazilian Coins Classification Using Deep Learning and Java

It is exciting to learn deep learning. The courses and the results are exciting and you can see it in your machine using popular libraries like Keras for Python and DeepLearning4j for Java. These libraries even have utility methods that will download famous dataset for you, you just have to copy and paste the code and run it, after training (probably a few hours on a CPU) you will have a model that is trained and ready to classify new data. I did that. I did that a lot of times. I also created small neural nets from scratch, but still, I was missing solving a problem with this technology and collect my own dataset. In this post I will share how I used DeepLearning4J to classify brazilian coins. I will try also to share how I failed a lot before getting my current 77% accuracy model. Brazilian coin for one real Collecting the data set We need a large dataset to train a neural network. They say that at least 1k images per classes . I started much less than that, about 50 ima...

Web app to easily collect images for your dataset

It is easy to train your neural network when you get data from the web. Datasets such as ImageNet and Mnist even have utility classes to load the data, such as MnistDataSetIterator from DeepLearning4J. The problem is when you have to collect the dataset yourself. There are many ways to do that, we can try to automatize how we collect the data, but then you may have a dataset that does not reflect the real world. Right now I have to collect images form real world. I can't take pictures from the internet, that are only a few and I can't take in my house or important features won't be learned. So I created a small web application which I think that may be useful for others trying to create their own dataset. The idea is simple: You set the output directory and the possible labels in pom.xml; Then you start the application and start taking pictures with your mobile; The application then saves the images using the label as the parent directory When training yo...

How java.io.File.deleteOnExit can cause leaks

If you didn't know this, now you know: calling   java.io.File.deleteOnExit can cause leaks. This is described by  Yan in his java.io.File.deleteOnExit() is evil post. You may think that you develop applications that won't create too much files and does not runs for many hours, but in an application server running for days this may be a real problem. In this post I just wanted to share how it can quickly grow your heap memory when dealing a lot of files. See the code below: Notice that the code deletes the reference to the created file object (nullify it) so it can be collected by the garbage collector. At some point the program will explodes with OutOfMemory error probably due the high amount of String created (when running with small heap). If you increase the heap and use some tool to get a heap dump of your running application you will see memory content on the *static* Linked used in DeleteOnExitHook class. We can do this in runtime by using jvisualvm (a too...

JavaFX with Bean Validation and CDI 2.0

When JavaFX 2 was released I remember people looking for frameworks that would make easy to integrate JavaFX with a validation framework, a dependency injection container and other enterprise services. Currently we can integrate CDI 2.0 and Bean Validation 2.0 with a JavaFX application without the need of any external framework. In this post I will show you how to validate your javaFX screen fields using bean validation in a CDI managed application. Before talking about all these things and JavaFX I recommend you to check the following posts from this blog: Using CDI 2.0 in a JavaFX application Testing Bean Validation 2.0 Validating JavaFX controls using Bean Validation With Bean Validation 2.0 we have the automatic unwrap of JavaFX properties. Which means that if you have a field that is of type of ObservableValue so the validator implementation will be able to retrieve the actual value to perform the validation. However, JavaFX control fields are still not supported, so...

Testing Bean Validation 2.0

Bean Validation 2.0 is among all the new Java EE 8 good new features: In this post I will share a sample application I used to see in action the following Bean Validation 2.0 features:  List items validation Temporal validation using Java 8 Project setup Hibernate Validator is the RI for Bean Validation 2.0 and the project, being a maven or gradle project, must use the following dependencies (or later versions):  org.hibernate:hibernate-validator:6.0.2.Final and you should also add org.glassfish:javax.el:3.0.1-X   as a hibernate validator dependency, where X is the build, check the maven repository for more. This is what my pom.xml looks like: Using bean validation annotations in a bean fields To test the new annotations I used the following bean: Notice the new annotations in subscribers and in manufactureDate and expirationDate fields: subscribers : I am validating users emails, all the Strings added to this list should b...