MSiA students Jen Kung, Rene Li, Kedi Wu, and Luyao Yang satisfied deep cravings for understanding food and restaurant images using Deep Learning and took home the second-place prize at the Deep Learning and Data Visualization Poster Session.
The team used a sophisticated analytics technology called Deep Learning to extract information out of restaurant photos posted on Yelp. Unlike tradition computer vision approaches which try to classify images of food such as pizza and hot dogs, their model predicted more contextual information about a restaurant such as price, outdoor seating, and suitability for lunch or dinner. These attributes are not easily inferred from a single image, and often times even a human requires information from multiple images to successfully determine if a restaurant has these attributes. The team developed a custom version of a deep neural network using the Keras deep learning library that considers nine images at a time and performs a joint classification based off multiple data points. Using this technique, the team managed to extract semantic context from restaurant reviews automatically.
This work comes at a very transformative time for NU, as new initiatives in data science are transforming the way that NU does research in broad areas including the arts. The project was part of a new course on Deep Learning (MSiA 490-30) taught by Dr. Ellick Chan. Technology developed from this project could be very useful in a time and age when people are posting Yelp photos on their mobile phones instead of writing lengthy reviews on desktop PCs. Such images are hard to process using traditional textual search algorithms, and new approaches to inferring such information from images will be necessary.
Please see below for the team’s research poster:
No Comments