Recent research has shown that popular computer vision algorithms can struggle with recognizing certain types of stylized or artistic photos posted on services such as Instagram because they alter key properties of the image such as color, shape, texture, and other informative features needed for accurate recognition. MSiA students Kapil Bhatt, Ke Huang, Mengshan Jin, Balamurali Natarajan, and Elena Smith devised a way to significantly improve the recognition rate on stylized images by devising a system that removes certain kinds of style transformations before the image is recognized by a conventional system. Their work was featured at the annual Deep Learning and Data Visualization Poster Session.
The team used a sophisticated analytics technology called Deep Learning to remove artificial style transformations from photographs that were distorted by artistic effects such as Instagram’s Kelvin and Nashville filters. To accomplish this task, the team developed a custom version of a denoising autoencoder that takes a distorted image and produces a clean version of it. Using this technique, the team then fed the denoised images into a standard convolutional neural network for image recognition and managed to classify many stylized images correctly.
The team used Keras, a popular deep learning library, to develop a custom convolutional denoising autoencoder based on one of the samples published on the Keras blog. They made many helpful improvements to the example code including: adding leaky ReLU activations, switching to the Adagrad optimizer and augmenting their training data by adding more training samples specific to the filters used.
This work comes at a very transformative time for NU, as new initiatives in data science are transforming the way that NU does research in broad areas including the arts. The project was part of a new course on Deep Learning (MSiA 490-30) taught by Dr. Ellick Chan. Technology developed from this project could be very useful in a time and age when people are increasingly posting stylized photos on the Internet. Many deep learning image recognition systems are currently trained on a variety of natural images, but as this project shows, new techniques may need to be developed to handle unexpected variations in images including the use of stylizers and other ways to manipulate images. This effort is just one step in developing more robust recognizers.
Please see below for the team’s poster:
Timoth Isean
March 5, 2018 at 4:44 amIts good initiative for students and as well as for the institute. Students can learns the core if they go through it and NU will be greatly benefited by the doing this research.